Class ListTokenSource

java.lang.Object

org.graalvm.shadowed.org.antlr.v4.runtime.ListTokenSource

All Implemented Interfaces:

TokenSource

public class ListTokenSource
extends Object
implements TokenSource

Provides an implementation of TokenSource as a wrapper around a list of Token objects.

If the final token in the list is an Token.EOF token, it will be used as the EOF token for every call to nextToken() after the end of the list is reached. Otherwise, an EOF token will be created.

Field Summary

Fields

Modifier and Type	Field	Description
protected Token	eofToken	This field caches the EOF token for the token source.
protected int	i	The index into tokens of token to return by the next call to nextToken().
protected final List<? extends Token>	tokens	The wrapped collection of Token objects to return.

Constructor Summary

Constructors

Constructor	Description
ListTokenSource(List<? extends Token> tokens)	Constructs a new ListTokenSource instance from the specified collection of Token objects.
ListTokenSource(List<? extends Token> tokens, String sourceName)	Constructs a new ListTokenSource instance from the specified collection of Token objects and source name.

Method Summary

Modifier and Type	Method	Description
int	getCharPositionInLine()	Get the index into the current line for the current position in the input stream.
CharStream	getInputStream()	Get the CharStream from which this token source is currently providing tokens.
int	getLine()	Get the line number for the current position in the input stream.
String	getSourceName()	Gets the name of the underlying input source.
TokenFactory<?>	getTokenFactory()	Gets the TokenFactory this token source is currently using for creating Token objects from the input.
Token	nextToken()	Return a Token object from your input stream (usually a CharStream).
void	setTokenFactory(TokenFactory<?> factory)	Set the TokenFactory this token source should use for creating Token objects from the input.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

tokens

protected final List<? extends Token> tokens

The wrapped collection of Token objects to return.

i

protected int i

The index into tokens of token to return by the next call to nextToken(). The end of the input is indicated by this value being greater than or equal to the number of items in tokens.

eofToken

protected Token eofToken

This field caches the EOF token for the token source.

Constructor Details

ListTokenSource

public ListTokenSource(List<? extends Token> tokens)

Constructs a new ListTokenSource instance from the specified collection of Token objects.

Parameters:

tokens - The collection of Token objects to provide as a TokenSource.

Throws:

NullPointerException - if tokens is null

ListTokenSource

public ListTokenSource(List<? extends Token> tokens,
 String sourceName)

Constructs a new ListTokenSource instance from the specified collection of Token objects and source name.

Parameters:

tokens - The collection of Token objects to provide as a TokenSource.

sourceName - The name of the TokenSource. If this value is null, getSourceName() will attempt to infer the name from the next Token (or the previous token if the end of the input has been reached).

Throws:

NullPointerException - if tokens is null

Method Details

getCharPositionInLine

public int getCharPositionInLine()

Get the index into the current line for the current position in the input stream. The first character on a line has position 0.

Specified by:

getCharPositionInLine in interface TokenSource

Returns:

The line number for the current position in the input stream, or -1 if the current token source does not track character positions.

nextToken

public Token nextToken()

Return a Token object from your input stream (usually a CharStream). Do not fail/return upon lexing error; keep chewing on the characters until you get a good one; errors are not passed through to the parser.

Specified by:

nextToken in interface TokenSource

getLine

public int getLine()

Get the line number for the current position in the input stream. The first line in the input is line 1.

Specified by:

getLine in interface TokenSource

Returns:

The line number for the current position in the input stream, or 0 if the current token source does not track line numbers.

getInputStream

public CharStream getInputStream()

Get the CharStream from which this token source is currently providing tokens.

Specified by:

getInputStream in interface TokenSource

Returns:

The CharStream associated with the current position in the input, or null if no input stream is available for the token source.

getSourceName

public String getSourceName()

Gets the name of the underlying input source. This method returns a non-null, non-empty string. If such a name is not known, this method returns IntStream.UNKNOWN_SOURCE_NAME.

Specified by:

getSourceName in interface TokenSource

setTokenFactory

public void setTokenFactory(TokenFactory<?> factory)

Set the TokenFactory this token source should use for creating Token objects from the input.

Specified by:

setTokenFactory in interface TokenSource

Parameters:

factory - The TokenFactory to use for creating tokens.

getTokenFactory

public TokenFactory<?> getTokenFactory()

Gets the TokenFactory this token source is currently using for creating Token objects from the input.

Specified by:

getTokenFactory in interface TokenSource

Returns:

The TokenFactory currently used by this token source.