Uses of Class
com.mpobjects.bdparsii.tokenizer.Token

Packages that use Token

Package	Description
com.mpobjects.bdparsii.tokenizer

Uses of Token in com.mpobjects.bdparsii.tokenizer

Methods in com.mpobjects.bdparsii.tokenizer that return Token

Modifier and Type	Method and Description
Token	Token.addToContent(char ch) Adds the given character to the content (and the source) but not to the trigger
Token	Token.addToContent(Char ch) Adds the given Char to the content (and the source) but not to the trigger
Token	Token.addToSource(Char ch) Adds the given Char to the source of this token, but neither to the trigger nor to the content.
Token	Token.addToTrigger(Char ch) Adds the given Char to the trigger (and the source) but not to the content
static Token	Token.create(Token.TokenType type, Position pos) Creates a new token with the given type, using the given position as location info.
static Token	Token.createAndFill(Token.TokenType type, Char ch) Creates a new token with the given type, using the Char a initial trigger and content.
protected Token	Tokenizer.endOfInput()
protected Token	Tokenizer.fetch()
protected Token	Tokenizer.fetchId() Reads and returns an identifier
protected Token	Tokenizer.fetchNumber() Reads and returns a number.
protected Token	Tokenizer.fetchSpecialId() Reads and returns a special id.
protected Token	Tokenizer.fetchString() Reads and returns a string constant.
protected Token	Tokenizer.fetchSymbol() Reads and returns a symbol.
protected Token	Tokenizer.handleKeywords(Token idToken) Checks if the given identifier is a keyword and returns an appropriate Token
Token	Token.silentAddToContent(char ch) Adds a character to the content without adding it to the source.

Methods in com.mpobjects.bdparsii.tokenizer with parameters of type Token

Modifier and Type	Method and Description
protected Token	Tokenizer.handleKeywords(Token idToken) Checks if the given identifier is a keyword and returns an appropriate Token
protected boolean	Tokenizer.handleStringEscape(char separator, char escapeChar, Token stringToken) Evaluates an string escape like \n