Uses of Class
org.apache.lucene.analysis.Tokenizer

Packages that use Tokenizer

Package	Description
org.apache.lucene.analysis
org.apache.lucene.analysis.core
org.apache.lucene.analysis.standard
org.apache.lucene.analysis.util
org.elasticsearch.index.analysis
org.elasticsearch.indices.analysis

Uses of Tokenizer in org.apache.lucene.analysis

Constructors in org.apache.lucene.analysis with parameters of type Tokenizer

Constructor	Description
TokenStreamComponents(Tokenizer tokenizer)	Creates a new Analyzer.TokenStreamComponents from a Tokenizer
TokenStreamComponents(Tokenizer tokenizer, TokenStream result)	Creates a new Analyzer.TokenStreamComponents instance

Uses of Tokenizer in org.apache.lucene.analysis.core

Subclasses of Tokenizer in org.apache.lucene.analysis.core

Modifier and Type	Class	Description
class	KeywordTokenizer	Emits the entire input as a single token.
class	LetterTokenizer	A LetterTokenizer is a tokenizer that divides text at non-letters.
class	WhitespaceTokenizer	A tokenizer that divides text at whitespace characters as defined by Character.isWhitespace(int).

Uses of Tokenizer in org.apache.lucene.analysis.standard

Subclasses of Tokenizer in org.apache.lucene.analysis.standard

Modifier and Type	Class	Description
class	ClassicTokenizer	A grammar-based tokenizer constructed with JFlex
class	StandardTokenizer	A grammar-based tokenizer constructed with JFlex.

Uses of Tokenizer in org.apache.lucene.analysis.util

Subclasses of Tokenizer in org.apache.lucene.analysis.util

Modifier and Type	Class	Description
class	CharTokenizer	An abstract base class for simple, character-oriented tokenizers.

Methods in org.apache.lucene.analysis.util that return Tokenizer

Modifier and Type	Method	Description
Tokenizer	TokenizerFactory.create()	Creates a TokenStream of the specified input using the default attribute factory.
abstract Tokenizer	TokenizerFactory.create(AttributeFactory factory)	Creates a TokenStream of the specified input using the given AttributeFactory

Uses of Tokenizer in org.elasticsearch.index.analysis

Methods in org.elasticsearch.index.analysis that return Tokenizer

Modifier and Type	Method	Description
Tokenizer	StandardTokenizerFactory.create()
Tokenizer	TokenizerFactory.create()

Method parameters in org.elasticsearch.index.analysis with type arguments of type Tokenizer

Modifier and Type	Method	Description
static PreConfiguredTokenizer	PreConfiguredTokenizer.elasticsearchVersion(String name, Function<Version,Tokenizer> create)	Create a pre-configured tokenizer that may vary based on the Elasticsearch version.
static PreConfiguredTokenizer	PreConfiguredTokenizer.luceneVersion(String name, Function<Version,Tokenizer> create)	Create a pre-configured tokenizer that may vary based on the Lucene version.
static TokenizerFactory	TokenizerFactory.newFactory(String name, Supplier<Tokenizer> supplier)
static PreConfiguredTokenizer	PreConfiguredTokenizer.singleton(String name, Supplier<Tokenizer> create)	Create a pre-configured tokenizer that may not vary at all.

Uses of Tokenizer in org.elasticsearch.indices.analysis

Methods in org.elasticsearch.indices.analysis that return Tokenizer

Modifier and Type	Method	Description
protected abstract Tokenizer	PreBuiltTokenizers.create(Version version)