Module org.elasticsearch.xcore
Class MPNetTokenization
java.lang.Object
org.elasticsearch.xpack.core.ml.inference.trainedmodel.Tokenization
org.elasticsearch.xpack.core.ml.inference.trainedmodel.MPNetTokenization
- All Implemented Interfaces:
NamedWriteable,Writeable,org.elasticsearch.xcontent.ToXContent,org.elasticsearch.xcontent.ToXContentObject,NamedXContentObject
-
Nested Class Summary
Nested classes/interfaces inherited from class org.elasticsearch.xpack.core.ml.inference.trainedmodel.Tokenization
Tokenization.SpanSettings, Tokenization.TruncateNested classes/interfaces inherited from interface org.elasticsearch.xcontent.ToXContent
org.elasticsearch.xcontent.ToXContent.DelegatingMapParams, org.elasticsearch.xcontent.ToXContent.MapParams, org.elasticsearch.xcontent.ToXContent.ParamsNested classes/interfaces inherited from interface org.elasticsearch.common.io.stream.Writeable
Writeable.Reader<V>, Writeable.Writer<V> -
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final Stringstatic final org.elasticsearch.xcontent.ParseFieldFields inherited from class org.elasticsearch.xpack.core.ml.inference.trainedmodel.Tokenization
DEFAULT_MAX_SEQUENCE_LENGTH, DO_LOWER_CASE, doLowerCase, MAX_SEQUENCE_LENGTH, maxSequenceLength, span, SPAN, truncate, TRUNCATE, UNSET_SPAN_VALUE, WITH_SPECIAL_TOKENS, withSpecialTokensFields inherited from interface org.elasticsearch.xcontent.ToXContent
EMPTY, EMPTY_PARAMS -
Constructor Summary
ConstructorsConstructorDescriptionMPNetTokenization(Boolean doLowerCase, Boolean withSpecialTokens, Integer maxSequenceLength, Tokenization.Truncate truncate, Integer span) -
Method Summary
Modifier and TypeMethodDescriptionstatic org.elasticsearch.xcontent.ConstructingObjectParser<MPNetTokenization, Void> createParser(boolean ignoreUnknownFields) booleanstatic MPNetTokenizationfromXContent(org.elasticsearch.xcontent.XContentParser parser, boolean lenient) getName()inthashCode()voidwriteTo(StreamOutput out) Methods inherited from class org.elasticsearch.xpack.core.ml.inference.trainedmodel.Tokenization
createDefault, doLowerCase, getMaxSequenceLength, getSpan, getTruncate, maxSequenceLength, toXContent, updateWindowSettings, validateSpanAndMaxSequenceLength, validateSpanAndTruncate, validateVocabulary, withSpecialTokensMethods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, toString, wait, wait, waitMethods inherited from interface org.elasticsearch.xcontent.ToXContentObject
isFragment
-
Field Details
-
NAME
public static final org.elasticsearch.xcontent.ParseField NAME -
MASK_TOKEN
- See Also:
-
-
Constructor Details
-
MPNetTokenization
public MPNetTokenization(@Nullable Boolean doLowerCase, @Nullable Boolean withSpecialTokens, @Nullable Integer maxSequenceLength, @Nullable Tokenization.Truncate truncate, @Nullable Integer span) -
MPNetTokenization
- Throws:
IOException
-
-
Method Details
-
createParser
public static org.elasticsearch.xcontent.ConstructingObjectParser<MPNetTokenization,Void> createParser(boolean ignoreUnknownFields) -
fromXContent
public static MPNetTokenization fromXContent(org.elasticsearch.xcontent.XContentParser parser, boolean lenient) -
writeTo
- Specified by:
writeToin interfaceWriteable- Overrides:
writeToin classTokenization- Throws:
IOException
-
getMaskToken
- Specified by:
getMaskTokenin classTokenization
-
getWriteableName
-
getName
- Returns:
- The name of the XContentObject that is to be serialized
-
equals
- Overrides:
equalsin classTokenization
-
hashCode
public int hashCode()- Overrides:
hashCodein classTokenization
-