Module org.elasticsearch.xcore
Class BertJapaneseTokenization
java.lang.Object
org.elasticsearch.xpack.core.ml.inference.trainedmodel.Tokenization
org.elasticsearch.xpack.core.ml.inference.trainedmodel.BertJapaneseTokenization
- All Implemented Interfaces:
NamedWriteable,Writeable,org.elasticsearch.xcontent.ToXContent,org.elasticsearch.xcontent.ToXContentObject,NamedXContentObject
-
Nested Class Summary
Nested classes/interfaces inherited from class org.elasticsearch.xpack.core.ml.inference.trainedmodel.Tokenization
Tokenization.SpanSettings, Tokenization.TruncateNested classes/interfaces inherited from interface org.elasticsearch.xcontent.ToXContent
org.elasticsearch.xcontent.ToXContent.DelegatingMapParams, org.elasticsearch.xcontent.ToXContent.MapParams, org.elasticsearch.xcontent.ToXContent.ParamsNested classes/interfaces inherited from interface org.elasticsearch.common.io.stream.Writeable
Writeable.Reader<V>, Writeable.Writer<V> -
Field Summary
FieldsModifier and TypeFieldDescriptionstatic final Stringstatic final org.elasticsearch.xcontent.ParseFieldFields inherited from class org.elasticsearch.xpack.core.ml.inference.trainedmodel.Tokenization
DEFAULT_MAX_SEQUENCE_LENGTH, DO_LOWER_CASE, doLowerCase, MAX_SEQUENCE_LENGTH, maxSequenceLength, span, SPAN, truncate, TRUNCATE, UNSET_SPAN_VALUE, WITH_SPECIAL_TOKENS, withSpecialTokensFields inherited from interface org.elasticsearch.xcontent.ToXContent
EMPTY, EMPTY_PARAMS -
Constructor Summary
ConstructorsConstructorDescriptionBertJapaneseTokenization(Boolean doLowerCase, Boolean withSpecialTokens, Integer maxSequenceLength, Tokenization.Truncate truncate, Integer span) -
Method Summary
Modifier and TypeMethodDescriptionstatic org.elasticsearch.xcontent.ConstructingObjectParser<BertJapaneseTokenization, Void> createJpParser(boolean ignoreUnknownFields) static BertJapaneseTokenizationfromXContent(org.elasticsearch.xcontent.XContentParser parser, boolean lenient) getName()Methods inherited from class org.elasticsearch.xpack.core.ml.inference.trainedmodel.Tokenization
createDefault, doLowerCase, equals, getMaxSequenceLength, getSpan, getTruncate, hashCode, maxSequenceLength, toXContent, updateWindowSettings, validateSpanAndMaxSequenceLength, validateSpanAndTruncate, validateVocabulary, withSpecialTokens, writeToMethods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, toString, wait, wait, waitMethods inherited from interface org.elasticsearch.xcontent.ToXContentObject
isFragment
-
Field Details
-
NAME
public static final org.elasticsearch.xcontent.ParseField NAME -
MASK_TOKEN
- See Also:
-
-
Constructor Details
-
BertJapaneseTokenization
public BertJapaneseTokenization(@Nullable Boolean doLowerCase, @Nullable Boolean withSpecialTokens, @Nullable Integer maxSequenceLength, @Nullable Tokenization.Truncate truncate, @Nullable Integer span) -
BertJapaneseTokenization
- Throws:
IOException
-
-
Method Details
-
createJpParser
public static org.elasticsearch.xcontent.ConstructingObjectParser<BertJapaneseTokenization,Void> createJpParser(boolean ignoreUnknownFields) -
fromXContent
public static BertJapaneseTokenization fromXContent(org.elasticsearch.xcontent.XContentParser parser, boolean lenient) -
getMaskToken
- Specified by:
getMaskTokenin classTokenization
-
getWriteableName
-
getName
- Returns:
- The name of the XContentObject that is to be serialized
-