| Class | Description |
|---|---|
| BasicSourcedToken |
An interned version of a string, with provinance information
|
| BasicToken |
An interned version of a string.
|
| CharacterTokenizer |
Character tokenizer implementation.
|
| NGramTokenizer |
Wraps another tokenizer, and adds all computes all ngrams of
characters from a single token produced by the inner tokenizer.
|
| SimpleSourcedTokenizer |
Simple implementation of a Tokenizer.
|
| SimpleTokenizer |
Simple implementation of a Tokenizer.
|
| TagLinkToken |
Copyright © 2016. All rights reserved.