templates.SequenceTokenTypeTemplateSchema

class templates.SequenceTokenTypeTemplateSchema(*args, **kwargs)

Sequence Token Type template

Parameters

Parameters

Name	Type	Default	Info
field	str		Field.
token_type	{"lowercase", "capitalized", "uppercase"}		Token type.
word_boundary_tokenizer	bool	True	Tokenize or not.
merge_token	bool	False to maintain backward compatability	Merge nearby tokens or not.