Module org.elasticsearch.xcore

Package org.elasticsearch.xpack.core.ml.inference.trainedmodel

Class BertJapaneseTokenization

java.lang.Object

org.elasticsearch.xpack.core.ml.inference.trainedmodel.Tokenization

org.elasticsearch.xpack.core.ml.inference.trainedmodel.BertJapaneseTokenization

All Implemented Interfaces:

NamedWriteable, Writeable, ToXContent, ToXContentObject, NamedXContentObject

public class BertJapaneseTokenization
extends Tokenization

Nested Class Summary

Nested classes/interfaces inherited from class org.elasticsearch.xpack.core.ml.inference.trainedmodel.Tokenization

Tokenization.SpanSettings, Tokenization.Truncate

Nested classes/interfaces inherited from interface org.elasticsearch.xcontent.ToXContent

ToXContent.DelegatingMapParams, ToXContent.MapParams, ToXContent.Params

Nested classes/interfaces inherited from interface org.elasticsearch.common.io.stream.Writeable

Writeable.Reader<V>, Writeable.Writer<V>

Field Summary

Fields

Modifier and Type	Field	Description
static final String	MASK_TOKEN
static final ParseField	NAME

Fields inherited from class org.elasticsearch.xpack.core.ml.inference.trainedmodel.Tokenization

DEFAULT_MAX_SEQUENCE_LENGTH, DO_LOWER_CASE, doLowerCase, MAX_SEQUENCE_LENGTH, maxSequenceLength, span, SPAN, truncate, TRUNCATE, UNSET_SPAN_VALUE, WITH_SPECIAL_TOKENS, withSpecialTokens

Fields inherited from interface org.elasticsearch.xcontent.ToXContent

EMPTY, EMPTY_PARAMS

Constructor Summary

Constructors

Constructor	Description
BertJapaneseTokenization(Boolean doLowerCase, Boolean withSpecialTokens, Integer maxSequenceLength, Tokenization.Truncate truncate, Integer span)
BertJapaneseTokenization(StreamInput in)

Method Summary

Modifier and Type	Method	Description
static ConstructingObjectParser<BertJapaneseTokenization,Void>	createJpParser(boolean ignoreUnknownFields)
static BertJapaneseTokenization	fromXContent(XContentParser parser, boolean lenient)
String	getMaskToken()
String	getName()
String	getWriteableName()

Methods inherited from class org.elasticsearch.xpack.core.ml.inference.trainedmodel.Tokenization

createDefault, doLowerCase, equals, getMaxSequenceLength, getSpan, getTruncate, hashCode, maxSequenceLength, toXContent, updateWindowSettings, validateSpanAndMaxSequenceLength, validateSpanAndTruncate, validateVocabulary, withSpecialTokens, writeTo

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.elasticsearch.xcontent.ToXContentObject

isFragment

Field Details

NAME

public static final ParseField NAME

MASK_TOKEN

public static final String MASK_TOKEN

See Also:

• Constant Field Values

Constructor Details

BertJapaneseTokenization

public BertJapaneseTokenization(@Nullable
 Boolean doLowerCase,
 @Nullable
 Boolean withSpecialTokens,
 @Nullable
 Integer maxSequenceLength,
 @Nullable
 Tokenization.Truncate truncate,
 @Nullable
 Integer span)

BertJapaneseTokenization

public BertJapaneseTokenization(StreamInput in)
                         throws IOException

Throws:

IOException

Method Details

createJpParser

public static ConstructingObjectParser<BertJapaneseTokenization,Void> createJpParser(boolean ignoreUnknownFields)

fromXContent

public static BertJapaneseTokenization fromXContent(XContentParser parser,
 boolean lenient)

getMaskToken

public String getMaskToken()

Specified by:

getMaskToken in class Tokenization

getWriteableName

public String getWriteableName()

getName

public String getName()

Returns:

The name of the XContentObject that is to be serialized