Uses of Class
org.apache.lucene.analysis.TokenStream

Packages that use TokenStream

Package	Description
org.apache.lucene.analysis
org.apache.lucene.analysis.core
org.apache.lucene.analysis.en
org.apache.lucene.analysis.hunspell
org.apache.lucene.analysis.miscellaneous
org.apache.lucene.analysis.ngram
org.apache.lucene.analysis.shingle
org.apache.lucene.analysis.standard
org.apache.lucene.analysis.synonym
org.apache.lucene.analysis.util
org.apache.lucene.codecs
org.apache.lucene.document
org.apache.lucene.index
org.apache.lucene.index.memory
org.apache.lucene.search.highlight
org.apache.lucene.search.suggest.analyzing
org.apache.lucene.search.suggest.document
org.apache.lucene.search.uhighlight
org.apache.lucene.spatial.prefix
org.apache.lucene.util
org.apache.lucene.util.graph
org.elasticsearch.index.analysis
org.elasticsearch.index.mapper
org.elasticsearch.index.query
org.elasticsearch.search.suggest.phrase

Uses of TokenStream in org.apache.lucene.analysis

Subclasses of TokenStream in org.apache.lucene.analysis

Modifier and Type	Class	Description
class	CachingTokenFilter	This class can be used if the token attributes of a TokenStream are intended to be consumed more than once.
class	FilteringTokenFilter	Abstract base class for TokenFilters that may remove tokens.
class	GraphTokenFilter	An abstract TokenFilter that exposes its input stream as a graph Call GraphTokenFilter.incrementBaseToken() to move the root of the graph to the next position in the TokenStream, GraphTokenFilter.incrementGraphToken() to move along the current graph, and GraphTokenFilter.incrementGraph() to reset to the next graph based at the current root.
class	LowerCaseFilter	Normalizes token text to lower case.
class	StopFilter	Removes stop words from a token stream.
class	TokenFilter	A TokenFilter is a TokenStream whose input is another TokenStream.
class	Tokenizer	A Tokenizer is a TokenStream whose input is a Reader.

Fields in org.apache.lucene.analysis declared as TokenStream

Modifier and Type	Field	Description
protected TokenStream	TokenFilter.input	The source of tokens for this filter.
protected TokenStream	Analyzer.TokenStreamComponents.sink	Sink tokenstream, such as the outer tokenfilter decorating the chain.

Methods in org.apache.lucene.analysis that return TokenStream

Modifier and Type	Method	Description
TokenStream	Analyzer.TokenStreamComponents.getTokenStream()	Returns the sink TokenStream
protected TokenStream	Analyzer.normalize(String fieldName, TokenStream in)	Wrap the given TokenStream in order to apply normalization filters.
protected TokenStream	AnalyzerWrapper.normalize(String fieldName, TokenStream in)
TokenStream	Analyzer.tokenStream(String fieldName, Reader reader)	Returns a TokenStream suitable for fieldName, tokenizing the contents of reader.
TokenStream	Analyzer.tokenStream(String fieldName, String text)	Returns a TokenStream suitable for fieldName, tokenizing the contents of text.
protected TokenStream	AnalyzerWrapper.wrapTokenStreamForNormalization(String fieldName, TokenStream in)	Wraps / alters the given TokenStream for normalization purposes, taken from the wrapped Analyzer, to form new components.
protected TokenStream	DelegatingAnalyzerWrapper.wrapTokenStreamForNormalization(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	Analyzer.normalize(String fieldName, TokenStream in)	Wrap the given TokenStream in order to apply normalization filters.
protected TokenStream	AnalyzerWrapper.normalize(String fieldName, TokenStream in)
Automaton	TokenStreamToAutomaton.toAutomaton(TokenStream in)	Pulls the graph (including PositionLengthAttribute) from the provided TokenStream, and creates the corresponding automaton where arcs are bytes (or Unicode code points if unicodeArcs = true) from each term.
protected TokenStream	AnalyzerWrapper.wrapTokenStreamForNormalization(String fieldName, TokenStream in)	Wraps / alters the given TokenStream for normalization purposes, taken from the wrapped Analyzer, to form new components.
protected TokenStream	DelegatingAnalyzerWrapper.wrapTokenStreamForNormalization(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis with parameters of type TokenStream

Constructor	Description
CachingTokenFilter(TokenStream input)	Create a new CachingTokenFilter around input.
FilteringTokenFilter(TokenStream in)	Create a new FilteringTokenFilter.
GraphTokenFilter(TokenStream input)	Create a new GraphTokenFilter
LowerCaseFilter(TokenStream in)	Create a new LowerCaseFilter, that normalizes token text to lower case.
StopFilter(TokenStream in, CharArraySet stopWords)	Constructs a filter which removes words from the input TokenStream that are named in the Set.
TokenFilter(TokenStream input)	Construct a token stream filtering the given input.
TokenStreamComponents(Consumer<Reader> source, TokenStream result)	Creates a new Analyzer.TokenStreamComponents instance.
TokenStreamComponents(Tokenizer tokenizer, TokenStream result)	Creates a new Analyzer.TokenStreamComponents instance

Uses of TokenStream in org.apache.lucene.analysis.core

Subclasses of TokenStream in org.apache.lucene.analysis.core

Modifier and Type	Class	Description
class	FlattenGraphFilter	Converts an incoming graph token stream, such as one from SynonymGraphFilter, into a flat form so that all nodes form a single linear chain with no side paths.
class	KeywordTokenizer	Emits the entire input as a single token.
class	LetterTokenizer	A LetterTokenizer is a tokenizer that divides text at non-letters.
class	WhitespaceTokenizer	A tokenizer that divides text at whitespace characters as defined by Character.isWhitespace(int).

Methods in org.apache.lucene.analysis.core that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	SimpleAnalyzer.normalize(String fieldName, TokenStream in)
protected TokenStream	StopAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.core with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	SimpleAnalyzer.normalize(String fieldName, TokenStream in)
protected TokenStream	StopAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.core with parameters of type TokenStream

Constructor	Description
FlattenGraphFilter(TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.en

Subclasses of TokenStream in org.apache.lucene.analysis.en

Modifier and Type	Class	Description
class	EnglishPossessiveFilter	TokenFilter that removes possessives (trailing 's) from words.
class	PorterStemFilter	Transforms the token stream as per the Porter stemming algorithm.

Methods in org.apache.lucene.analysis.en that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	EnglishAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.en with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	EnglishAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.en with parameters of type TokenStream

Constructor	Description
EnglishPossessiveFilter(TokenStream input)
PorterStemFilter(TokenStream in)

Uses of TokenStream in org.apache.lucene.analysis.hunspell

Subclasses of TokenStream in org.apache.lucene.analysis.hunspell

Modifier and Type	Class	Description
class	HunspellStemFilter	TokenFilter that uses hunspell affix rules and words to stem tokens.

Constructors in org.apache.lucene.analysis.hunspell with parameters of type TokenStream

Constructor	Description
HunspellStemFilter(TokenStream input, Dictionary dictionary)	Create a HunspellStemFilter outputting all possible stems.
HunspellStemFilter(TokenStream input, Dictionary dictionary, boolean dedup)	Create a HunspellStemFilter outputting all possible stems.
HunspellStemFilter(TokenStream input, Dictionary dictionary, boolean dedup, boolean longestOnly)	Creates a new HunspellStemFilter that will stem tokens from the given TokenStream using affix rules in the provided Dictionary

Uses of TokenStream in org.apache.lucene.analysis.miscellaneous

Subclasses of TokenStream in org.apache.lucene.analysis.miscellaneous

Modifier and Type	Class	Description
class	ConcatenateGraphFilter	Concatenates/Joins every incoming token with a separator into one output token for every path through the token stream (which is a graph).
class	DeDuplicatingTokenFilter	Inspects token streams for duplicate sequences of tokens.
class	KeywordMarkerFilter	Marks terms as keywords via the KeywordAttribute.
class	SetKeywordMarkerFilter	Marks terms as keywords via the KeywordAttribute.

Constructors in org.apache.lucene.analysis.miscellaneous with parameters of type TokenStream

Constructor	Description
ConcatenateGraphFilter(TokenStream inputTokenStream)	Creates a token stream to convert input to a token stream of accepted strings by its token stream graph.
ConcatenateGraphFilter(TokenStream inputTokenStream, boolean preserveSep, boolean preservePositionIncrements, int maxGraphExpansions)	Calls ConcatenateGraphFilter(org.apache.lucene.analysis.TokenStream, java.lang.Character, boolean, int)
ConcatenateGraphFilter(TokenStream inputTokenStream, Character tokenSeparator, boolean preservePositionIncrements, int maxGraphExpansions)	Creates a token stream to convert input to a token stream of accepted strings by its token stream graph.
DeDuplicatingTokenFilter(TokenStream in, DuplicateByteSequenceSpotter byteStreamDuplicateSpotter)
DeDuplicatingTokenFilter(TokenStream in, DuplicateByteSequenceSpotter byteStreamDuplicateSpotter, boolean emitDuplicates)
KeywordMarkerFilter(TokenStream in)	Creates a new KeywordMarkerFilter
SetKeywordMarkerFilter(TokenStream in, CharArraySet keywordSet)	Create a new KeywordSetMarkerFilter, that marks the current token as a keyword if the tokens term buffer is contained in the given set via the KeywordAttribute.

Uses of TokenStream in org.apache.lucene.analysis.ngram

Subclasses of TokenStream in org.apache.lucene.analysis.ngram

Modifier and Type	Class	Description
class	EdgeNGramTokenFilter	Tokenizes the given token into n-grams of given size(s).

Constructors in org.apache.lucene.analysis.ngram with parameters of type TokenStream

Constructor	Description
EdgeNGramTokenFilter(TokenStream input, int gramSize)	Creates an EdgeNGramTokenFilter that produces edge n-grams of the given size.
EdgeNGramTokenFilter(TokenStream input, int minGram, int maxGram, boolean preserveOriginal)	Creates an EdgeNGramTokenFilter that, for a given input term, produces all edge n-grams with lengths >= minGram and <= maxGram.

Uses of TokenStream in org.apache.lucene.analysis.shingle

Subclasses of TokenStream in org.apache.lucene.analysis.shingle

Modifier and Type	Class	Description
class	FixedShingleFilter	A FixedShingleFilter constructs shingles (token n-grams) from a token stream.
class	ShingleFilter	A ShingleFilter constructs shingles (token n-grams) from a token stream.

Constructors in org.apache.lucene.analysis.shingle with parameters of type TokenStream

Constructor	Description
FixedShingleFilter(TokenStream input, int shingleSize)	Creates a FixedShingleFilter over an input token stream
FixedShingleFilter(TokenStream input, int shingleSize, String tokenSeparator, String fillerToken)	Creates a FixedShingleFilter over an input token stream
ShingleFilter(TokenStream input)	Construct a ShingleFilter with default shingle size: 2.
ShingleFilter(TokenStream input, int maxShingleSize)	Constructs a ShingleFilter with the specified shingle size from the TokenStream input
ShingleFilter(TokenStream input, int minShingleSize, int maxShingleSize)	Constructs a ShingleFilter with the specified shingle size from the TokenStream input
ShingleFilter(TokenStream input, String tokenType)	Construct a ShingleFilter with the specified token type for shingle tokens and the default shingle size: 2

Uses of TokenStream in org.apache.lucene.analysis.standard

Subclasses of TokenStream in org.apache.lucene.analysis.standard

Modifier and Type	Class	Description
class	ClassicFilter	Normalizes tokens extracted with ClassicTokenizer.
class	ClassicTokenizer	A grammar-based tokenizer constructed with JFlex
class	StandardTokenizer	A grammar-based tokenizer constructed with JFlex.

Methods in org.apache.lucene.analysis.standard that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	ClassicAnalyzer.normalize(String fieldName, TokenStream in)
protected TokenStream	StandardAnalyzer.normalize(String fieldName, TokenStream in)

Methods in org.apache.lucene.analysis.standard with parameters of type TokenStream

Modifier and Type	Method	Description
protected TokenStream	ClassicAnalyzer.normalize(String fieldName, TokenStream in)
protected TokenStream	StandardAnalyzer.normalize(String fieldName, TokenStream in)

Constructors in org.apache.lucene.analysis.standard with parameters of type TokenStream

Constructor	Description
ClassicFilter(TokenStream in)	Construct filtering in.

Uses of TokenStream in org.apache.lucene.analysis.synonym

Subclasses of TokenStream in org.apache.lucene.analysis.synonym

Modifier and Type	Class	Description
class	SynonymFilter	Deprecated. Use SynonymGraphFilter instead, but be sure to also use FlattenGraphFilter at index time (not at search time) as well.
class	SynonymGraphFilter	Applies single- or multi-token synonyms from a SynonymMap to an incoming TokenStream, producing a fully correct graph output.

Constructors in org.apache.lucene.analysis.synonym with parameters of type TokenStream

Constructor	Description
SynonymFilter(TokenStream input, SynonymMap synonyms, boolean ignoreCase)	Deprecated.
SynonymGraphFilter(TokenStream input, SynonymMap synonyms, boolean ignoreCase)	Apply previously built synonyms to incoming tokens.

Uses of TokenStream in org.apache.lucene.analysis.util

Subclasses of TokenStream in org.apache.lucene.analysis.util

Modifier and Type	Class	Description
class	CharTokenizer	An abstract base class for simple, character-oriented tokenizers.

Methods in org.apache.lucene.analysis.util that return TokenStream

Modifier and Type	Method	Description
abstract TokenStream	TokenFilterFactory.create(TokenStream input)	Transform the specified input TokenStream
TokenStream	TokenFilterFactory.normalize(TokenStream input)	Normalize the specified input TokenStream While the default implementation returns input unchanged, filters that should be applied at normalization time can delegate to create method.

Methods in org.apache.lucene.analysis.util with parameters of type TokenStream

Modifier and Type	Method	Description
abstract TokenStream	TokenFilterFactory.create(TokenStream input)	Transform the specified input TokenStream
TokenStream	TokenFilterFactory.normalize(TokenStream input)	Normalize the specified input TokenStream While the default implementation returns input unchanged, filters that should be applied at normalization time can delegate to create method.

Uses of TokenStream in org.apache.lucene.codecs

Methods in org.apache.lucene.codecs that return TokenStream

Modifier and Type	Method	Description
TokenStream	StoredFieldsWriter.MergeVisitor.tokenStream(Analyzer analyzer, TokenStream reuse)

Methods in org.apache.lucene.codecs with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	StoredFieldsWriter.MergeVisitor.tokenStream(Analyzer analyzer, TokenStream reuse)

Uses of TokenStream in org.apache.lucene.document

Fields in org.apache.lucene.document declared as TokenStream

Modifier and Type	Field	Description
protected TokenStream	Field.tokenStream	Pre-analyzed tokenStream for indexed fields; this is separate from fieldsData because you are allowed to have both; eg maybe field has a String value but you customize how it's tokenized

Methods in org.apache.lucene.document that return TokenStream

Modifier and Type	Method	Description
TokenStream	Field.tokenStream(Analyzer analyzer, TokenStream reuse)
TokenStream	Field.tokenStreamValue()	The TokenStream for this field to be used when indexing, or null.

Methods in org.apache.lucene.document with parameters of type TokenStream

Modifier and Type	Method	Description
void	Field.setTokenStream(TokenStream tokenStream)	Expert: sets the token stream to be used for indexing and causes isIndexed() and isTokenized() to return true.
TokenStream	Field.tokenStream(Analyzer analyzer, TokenStream reuse)

Constructors in org.apache.lucene.document with parameters of type TokenStream

Constructor	Description
Field(String name, TokenStream tokenStream, IndexableFieldType type)	Create field with TokenStream value.
TextField(String name, TokenStream stream)	Creates a new un-stored TextField with TokenStream value.

Uses of TokenStream in org.apache.lucene.index

Methods in org.apache.lucene.index that return TokenStream

Modifier and Type	Method	Description
TokenStream	IndexableField.tokenStream(Analyzer analyzer, TokenStream reuse)	Creates the TokenStream used for indexing this field.

Methods in org.apache.lucene.index with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	IndexableField.tokenStream(Analyzer analyzer, TokenStream reuse)	Creates the TokenStream used for indexing this field.

Uses of TokenStream in org.apache.lucene.index.memory

Methods in org.apache.lucene.index.memory that return TokenStream

Modifier and Type	Method	Description
<T> TokenStream	MemoryIndex.keywordTokenStream(Collection<T> keywords)	Convenience method; Creates and returns a token stream that generates a token for each keyword in the given collection, "as is", without any transforming text analysis.

Methods in org.apache.lucene.index.memory with parameters of type TokenStream

Modifier and Type	Method	Description
void	MemoryIndex.addField(String fieldName, TokenStream stream)	Iterates over the given token stream and adds the resulting terms to the index; Equivalent to adding a tokenized, indexed, termVectorStored, unstored, Lucene Field.
void	MemoryIndex.addField(String fieldName, TokenStream stream, int positionIncrementGap)	Iterates over the given token stream and adds the resulting terms to the index; Equivalent to adding a tokenized, indexed, termVectorStored, unstored, Lucene Field.
void	MemoryIndex.addField(String fieldName, TokenStream tokenStream, int positionIncrementGap, int offsetGap)	Iterates over the given token stream and adds the resulting terms to the index; Equivalent to adding a tokenized, indexed, termVectorStored, unstored, Lucene Field.

Uses of TokenStream in org.apache.lucene.search.highlight

Subclasses of TokenStream in org.apache.lucene.search.highlight

Modifier and Type	Class	Description
class	OffsetLimitTokenFilter	This TokenFilter limits the number of tokens while indexing by adding up the current offset.
class	TokenStreamFromTermVector	TokenStream created from a term vector field.

Methods in org.apache.lucene.search.highlight that return TokenStream

Modifier and Type	Method	Description
TokenStream	WeightedSpanTermExtractor.getTokenStream()	Returns the tokenStream which may have been wrapped in a CachingTokenFilter.
TokenStream	QueryScorer.init(TokenStream tokenStream)
TokenStream	Scorer.init(TokenStream tokenStream)	Called to init the Scorer with a TokenStream.

Methods in org.apache.lucene.search.highlight with parameters of type TokenStream

Modifier and Type	Method	Description
String	Highlighter.getBestFragment(TokenStream tokenStream, String text)	Highlights chosen terms in a text, extracting the most relevant section.
String[]	Highlighter.getBestFragments(TokenStream tokenStream, String text, int maxNumFragments)	Highlights chosen terms in a text, extracting the most relevant sections.
String	Highlighter.getBestFragments(TokenStream tokenStream, String text, int maxNumFragments, String separator)	Highlights terms in the text , extracting the most relevant sections and concatenating the chosen fragments with a separator (typically "...").
TextFragment[]	Highlighter.getBestTextFragments(TokenStream tokenStream, String text, boolean mergeContiguousFragments, int maxNumFragments)	Low level api to get the most relevant (formatted) sections of the document.
Map<String,WeightedSpanTerm>	WeightedSpanTermExtractor.getWeightedSpanTerms(Query query, float boost, TokenStream tokenStream)	Creates a Map of WeightedSpanTerms from the given Query and TokenStream.
Map<String,WeightedSpanTerm>	WeightedSpanTermExtractor.getWeightedSpanTerms(Query query, float boost, TokenStream tokenStream, String fieldName)	Creates a Map of WeightedSpanTerms from the given Query and TokenStream.
Map<String,WeightedSpanTerm>	WeightedSpanTermExtractor.getWeightedSpanTermsWithScores(Query query, float boost, TokenStream tokenStream, String fieldName, IndexReader reader)	Creates a Map of WeightedSpanTerms from the given Query and TokenStream.
TokenStream	QueryScorer.init(TokenStream tokenStream)
TokenStream	Scorer.init(TokenStream tokenStream)	Called to init the Scorer with a TokenStream.
void	Fragmenter.start(String originalText, TokenStream tokenStream)	Initializes the Fragmenter.
void	NullFragmenter.start(String s, TokenStream tokenStream)
void	SimpleFragmenter.start(String originalText, TokenStream stream)
void	SimpleSpanFragmenter.start(String originalText, TokenStream tokenStream)

Constructors in org.apache.lucene.search.highlight with parameters of type TokenStream

Constructor	Description
OffsetLimitTokenFilter(TokenStream input, int offsetLimit)
TokenGroup(TokenStream tokenStream)

Uses of TokenStream in org.apache.lucene.search.suggest.analyzing

Subclasses of TokenStream in org.apache.lucene.search.suggest.analyzing

Modifier and Type	Class	Description
class	SuggestStopFilter	Like StopFilter except it will not remove the last token if that token was not followed by some token separator.

Constructors in org.apache.lucene.search.suggest.analyzing with parameters of type TokenStream

Constructor	Description
SuggestStopFilter(TokenStream input, CharArraySet stopWords)	Sole constructor.

Uses of TokenStream in org.apache.lucene.search.suggest.document

Subclasses of TokenStream in org.apache.lucene.search.suggest.document

Modifier and Type	Class	Description
class	CompletionTokenStream	A ConcatenateGraphFilter but we can set the payload and provide access to config options.

Methods in org.apache.lucene.search.suggest.document that return TokenStream

Modifier and Type	Method	Description
TokenStream	SuggestField.tokenStream(Analyzer analyzer, TokenStream reuse)

Methods in org.apache.lucene.search.suggest.document with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	SuggestField.tokenStream(Analyzer analyzer, TokenStream reuse)
protected CompletionTokenStream	ContextSuggestField.wrapTokenStream(TokenStream stream)
protected CompletionTokenStream	SuggestField.wrapTokenStream(TokenStream stream)	Wraps a stream with a CompletionTokenStream.

Uses of TokenStream in org.apache.lucene.search.uhighlight

Methods in org.apache.lucene.search.uhighlight that return TokenStream

Modifier and Type	Method	Description
protected TokenStream	AnalysisOffsetStrategy.tokenStream(String content)

Uses of TokenStream in org.apache.lucene.spatial.prefix

Subclasses of TokenStream in org.apache.lucene.spatial.prefix

Modifier and Type	Class	Description
class	BytesRefIteratorTokenStream	A TokenStream used internally by PrefixTreeStrategy.
class	PrefixTreeStrategy.ShapeTokenStream

Uses of TokenStream in org.apache.lucene.util

Methods in org.apache.lucene.util with parameters of type TokenStream

Modifier and Type	Method	Description
protected Query	QueryBuilder.analyzeBoolean(String field, TokenStream stream)	Creates simple boolean query from the cached tokenstream contents
protected Query	QueryBuilder.analyzeGraphBoolean(String field, TokenStream source, BooleanClause.Occur operator)	Creates a boolean query from a graph token stream.
protected Query	QueryBuilder.analyzeGraphPhrase(TokenStream source, String field, int phraseSlop)	Creates graph phrase query from the tokenstream contents
protected Query	QueryBuilder.analyzeMultiBoolean(String field, TokenStream stream, BooleanClause.Occur operator)	Creates complex boolean query from the cached tokenstream contents
protected Query	QueryBuilder.analyzeMultiPhrase(String field, TokenStream stream, int slop)	Creates complex phrase query from the cached tokenstream contents
protected Query	QueryBuilder.analyzePhrase(String field, TokenStream stream, int slop)	Creates simple phrase query from the cached tokenstream contents
protected Query	QueryBuilder.analyzeTerm(String field, TokenStream stream)	Creates simple term query from the cached tokenstream contents
protected Query	QueryBuilder.createFieldQuery(TokenStream source, BooleanClause.Occur operator, String field, boolean quoted, int phraseSlop)	Creates a query from a token stream.
protected SpanQuery	QueryBuilder.createSpanQuery(TokenStream in, String field)	Creates a span query from the tokenstream.

Uses of TokenStream in org.apache.lucene.util.graph

Methods in org.apache.lucene.util.graph that return types with arguments of type TokenStream

Modifier and Type	Method	Description
Iterator<TokenStream>	GraphTokenStreamFiniteStrings.getFiniteStrings()	Get all finite strings from the automaton.
Iterator<TokenStream>	GraphTokenStreamFiniteStrings.getFiniteStrings(int startState, int endState)	Get all finite strings that start at startState and end at endState.

Constructors in org.apache.lucene.util.graph with parameters of type TokenStream

Constructor	Description
GraphTokenStreamFiniteStrings(TokenStream in)

Uses of TokenStream in org.elasticsearch.index.analysis

Methods in org.elasticsearch.index.analysis that return TokenStream

Modifier and Type	Method	Description
TokenStream	HunspellTokenFilterFactory.create(TokenStream tokenStream)
TokenStream	ShingleTokenFilterFactory.create(TokenStream tokenStream)
TokenStream	ShingleTokenFilterFactory.Factory.create(TokenStream tokenStream)
TokenStream	StopTokenFilterFactory.create(TokenStream tokenStream)
TokenStream	TokenFilterFactory.create(TokenStream tokenStream)
protected TokenStream	CustomAnalyzer.normalize(String fieldName, TokenStream in)
default TokenStream	NormalizingTokenFilterFactory.normalize(TokenStream tokenStream)
protected TokenStream	ReloadableCustomAnalyzer.normalize(String fieldName, TokenStream in)
default TokenStream	TokenFilterFactory.normalize(TokenStream tokenStream)	Normalize a tokenStream for use in multi-term queries The default implementation is a no-op

Methods in org.elasticsearch.index.analysis with parameters of type TokenStream

Modifier and Type	Method	Description
TokenStream	HunspellTokenFilterFactory.create(TokenStream tokenStream)
TokenStream	ShingleTokenFilterFactory.create(TokenStream tokenStream)
TokenStream	ShingleTokenFilterFactory.Factory.create(TokenStream tokenStream)
TokenStream	StopTokenFilterFactory.create(TokenStream tokenStream)
TokenStream	TokenFilterFactory.create(TokenStream tokenStream)
protected TokenStream	CustomAnalyzer.normalize(String fieldName, TokenStream in)
default TokenStream	NormalizingTokenFilterFactory.normalize(TokenStream tokenStream)
protected TokenStream	ReloadableCustomAnalyzer.normalize(String fieldName, TokenStream in)
default TokenStream	TokenFilterFactory.normalize(TokenStream tokenStream)	Normalize a tokenStream for use in multi-term queries The default implementation is a no-op

Method parameters in org.elasticsearch.index.analysis with type arguments of type TokenStream

Modifier and Type	Method	Description
static PreConfiguredTokenFilter	PreConfiguredTokenFilter.elasticsearchVersion(String name, boolean useFilterForMultitermQueries, BiFunction<TokenStream,Version,TokenStream> create)	Create a pre-configured token filter that may vary based on the Elasticsearch version.
static PreConfiguredTokenFilter	PreConfiguredTokenFilter.elasticsearchVersion(String name, boolean useFilterForMultitermQueries, BiFunction<TokenStream,Version,TokenStream> create)	Create a pre-configured token filter that may vary based on the Elasticsearch version.
static PreConfiguredTokenFilter	PreConfiguredTokenFilter.luceneVersion(String name, boolean useFilterForMultitermQueries, BiFunction<TokenStream,Version,TokenStream> create)	Create a pre-configured token filter that may vary based on the Lucene version.
static PreConfiguredTokenFilter	PreConfiguredTokenFilter.luceneVersion(String name, boolean useFilterForMultitermQueries, BiFunction<TokenStream,Version,TokenStream> create)	Create a pre-configured token filter that may vary based on the Lucene version.
static PreConfiguredTokenFilter	PreConfiguredTokenFilter.singleton(String name, boolean useFilterForMultitermQueries, boolean allowForSynonymParsing, Function<TokenStream,TokenStream> create)	Create a pre-configured token filter that may not vary at all.
static PreConfiguredTokenFilter	PreConfiguredTokenFilter.singleton(String name, boolean useFilterForMultitermQueries, boolean allowForSynonymParsing, Function<TokenStream,TokenStream> create)	Create a pre-configured token filter that may not vary at all.
static PreConfiguredTokenFilter	PreConfiguredTokenFilter.singleton(String name, boolean useFilterForMultitermQueries, Function<TokenStream,TokenStream> create)	Create a pre-configured token filter that may not vary at all.
static PreConfiguredTokenFilter	PreConfiguredTokenFilter.singleton(String name, boolean useFilterForMultitermQueries, Function<TokenStream,TokenStream> create)	Create a pre-configured token filter that may not vary at all.
static PreConfiguredTokenFilter	PreConfiguredTokenFilter.singletonWithVersion(String name, boolean useFilterForMultitermQueries, boolean useFilterForParsingSynonyms, BiFunction<TokenStream,Version,TokenStream> create)	Create a pre-configured token filter that may vary based on the Elasticsearch version.
static PreConfiguredTokenFilter	PreConfiguredTokenFilter.singletonWithVersion(String name, boolean useFilterForMultitermQueries, boolean useFilterForParsingSynonyms, BiFunction<TokenStream,Version,TokenStream> create)	Create a pre-configured token filter that may vary based on the Elasticsearch version.
static PreConfiguredTokenFilter	PreConfiguredTokenFilter.singletonWithVersion(String name, boolean useFilterForMultitermQueries, BiFunction<TokenStream,Version,TokenStream> create)	Create a pre-configured token filter that may vary based on the Elasticsearch version.
static PreConfiguredTokenFilter	PreConfiguredTokenFilter.singletonWithVersion(String name, boolean useFilterForMultitermQueries, BiFunction<TokenStream,Version,TokenStream> create)	Create a pre-configured token filter that may vary based on the Elasticsearch version.

Uses of TokenStream in org.elasticsearch.index.mapper

Methods in org.elasticsearch.index.mapper that return TokenStream

Modifier and Type	Method	Description
TokenStream	CustomDocValuesField.tokenStream(Analyzer analyzer, TokenStream reuse)

Methods in org.elasticsearch.index.mapper with parameters of type TokenStream

Modifier and Type	Method	Description
static Query	TextFieldMapper.createPhrasePrefixQuery(TokenStream stream, String field, int slop, int maxExpansions, String prefixField, IntPredicate usePrefixField)
static Query	TextFieldMapper.createPhraseQuery(TokenStream stream, String field, int slop, boolean enablePositionIncrements)
static boolean	TextFieldMapper.TextFieldType.hasGaps(TokenStream stream)
Query	MappedFieldType.multiPhraseQuery(TokenStream stream, int slop, boolean enablePositionIncrements)
Query	TextFieldMapper.TextFieldType.multiPhraseQuery(TokenStream stream, int slop, boolean enablePositionIncrements)
Query	MappedFieldType.phrasePrefixQuery(TokenStream stream, int slop, int maxExpansions)
Query	TextFieldMapper.TextFieldType.phrasePrefixQuery(TokenStream stream, int slop, int maxExpansions)
Query	MappedFieldType.phraseQuery(TokenStream stream, int slop, boolean enablePositionIncrements)
Query	TextFieldMapper.TextFieldType.phraseQuery(TokenStream stream, int slop, boolean enablePosIncrements)
TokenStream	CustomDocValuesField.tokenStream(Analyzer analyzer, TokenStream reuse)

Uses of TokenStream in org.elasticsearch.index.query

Methods in org.elasticsearch.index.query with parameters of type TokenStream

Modifier and Type	Method	Description
protected List<IntervalsSource>	IntervalBuilder.analyzeGraph(TokenStream source)
protected IntervalsSource	IntervalBuilder.analyzeSynonyms(TokenStream ts, int maxGaps, boolean ordered)
protected IntervalsSource	IntervalBuilder.analyzeTerm(TokenStream ts)
protected List<IntervalsSource>	IntervalBuilder.analyzeTerms(TokenStream ts)

Uses of TokenStream in org.elasticsearch.search.suggest.phrase

Methods in org.elasticsearch.search.suggest.phrase with parameters of type TokenStream

Modifier and Type	Method	Description
static int	DirectCandidateGenerator.analyze(TokenStream stream, DirectCandidateGenerator.TokenConsumer consumer)	NOTE: this method closes the TokenStream, even on exception, which is awkward because really the caller who called Analyzer.tokenStream(java.lang.String, java.io.Reader) should close it, but when trying that there are recursion issues when we try to use the same TokenStream twice in the same recursion...
void	DirectCandidateGenerator.TokenConsumer.reset(TokenStream stream)