org.eclipse.xtext.parser.antlr

Class XtextTokenStream

java.lang.Object

org.antlr.runtime.CommonTokenStream

org.eclipse.xtext.parser.antlr.XtextTokenStream

All Implemented Interfaces:

org.antlr.runtime.IntStream, org.antlr.runtime.TokenStream

public class XtextTokenStream
extends org.antlr.runtime.CommonTokenStream

A token stream that is aware of the current lookahead.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static interface	XtextTokenStream.HiddenTokens

Field Summary

Fields inherited from class org.antlr.runtime.CommonTokenStream

channel, channelOverrideMap, discardOffChannelTokens, discardSet, lastMarker, p, tokens, tokenSource

Constructor Summary

Constructors

Constructor and Description
XtextTokenStream()
XtextTokenStream(org.antlr.runtime.TokenSource tokenSource, int channel)
XtextTokenStream(org.antlr.runtime.TokenSource tokenSource, ITokenDefProvider tokenDefProvider)

Method Summary

Modifier and Type	Method and Description
protected void	basicConsume() Make super-impl of consume() accessible to subtypes.
protected int	basicMark() Make super-impl of mark() accessible to subtypes.
protected void	basicRewind(int marker) Make super-impl of CommonTokenStream.rewind() accessible to subtypes.
void	consume()
protected void	doSetHiddenTokens(java.lang.String... lexerRules)
int	getCurrentLookAhead()
protected int	getFirstMarker()
java.lang.String	getLexerErrorMessage(org.antlr.runtime.Token invalidToken)
protected int	getTokenIndex(org.antlr.runtime.Token tok)
void	initCurrentLookAhead(int currentLookAhead)
int	LA(int i)
org.antlr.runtime.Token	LT(int k) Same as CommonTokenStream.LT(int) except that we skip hidden tokens even for k == 1.
int	mark()
void	rewind(int marker)
XtextTokenStream.HiddenTokens	setHiddenTokens(java.lang.String... lexerRules)
void	setInitialHiddenTokens(java.lang.String... lexerRules)
protected void	skipHiddenTokens()
java.lang.String	toString(int start, int stop)

Methods inherited from class org.antlr.runtime.CommonTokenStream

discardOffChannelTokens, discardTokenType, fillBuffer, get, getSourceName, getTokens, getTokens, getTokens, getTokens, getTokens, getTokenSource, index, LB, release, reset, rewind, seek, setTokenSource, setTokenTypeChannel, size, skipOffTokenChannels, skipOffTokenChannelsReverse, toString, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

XtextTokenStream

public XtextTokenStream()

XtextTokenStream

public XtextTokenStream(org.antlr.runtime.TokenSource tokenSource,
                        int channel)

XtextTokenStream

public XtextTokenStream(org.antlr.runtime.TokenSource tokenSource,
                        ITokenDefProvider tokenDefProvider)

Method Detail

toString

public java.lang.String toString(int start,
                                 int stop)

Specified by:

toString in interface org.antlr.runtime.TokenStream

Overrides:

toString in class org.antlr.runtime.CommonTokenStream

setHiddenTokens

public XtextTokenStream.HiddenTokens setHiddenTokens(java.lang.String... lexerRules)

setInitialHiddenTokens

public void setInitialHiddenTokens(java.lang.String... lexerRules)

doSetHiddenTokens

protected void doSetHiddenTokens(java.lang.String... lexerRules)

skipHiddenTokens

protected void skipHiddenTokens()

LA

public int LA(int i)

Specified by:

LA in interface org.antlr.runtime.IntStream

Overrides:

LA in class org.antlr.runtime.CommonTokenStream

getTokenIndex

protected int getTokenIndex(org.antlr.runtime.Token tok)

Since:

2.22

consume

public void consume()

Specified by:

consume in interface org.antlr.runtime.IntStream

Overrides:

consume in class org.antlr.runtime.CommonTokenStream

basicConsume

protected void basicConsume()

Make super-impl of consume() accessible to subtypes.

Since:

2.22

getFirstMarker

protected int getFirstMarker()

Since:

2.1

mark

public int mark()

Specified by:

mark in interface org.antlr.runtime.IntStream

Overrides:

mark in class org.antlr.runtime.CommonTokenStream

basicMark

protected int basicMark()

Make super-impl of mark() accessible to subtypes.

Since:

2.22

rewind

public void rewind(int marker)

Specified by:

rewind in interface org.antlr.runtime.IntStream

Overrides:

rewind in class org.antlr.runtime.CommonTokenStream

basicRewind

protected void basicRewind(int marker)

Make super-impl of CommonTokenStream.rewind() accessible to subtypes.

Since:

2.22

LT

public org.antlr.runtime.Token LT(int k)

Same as CommonTokenStream.LT(int) except that we skip hidden tokens even for k == 1.

Specified by:

LT in interface org.antlr.runtime.TokenStream

Overrides:

LT in class org.antlr.runtime.CommonTokenStream

getCurrentLookAhead

public int getCurrentLookAhead()

initCurrentLookAhead

public void initCurrentLookAhead(int currentLookAhead)

getLexerErrorMessage

public java.lang.String getLexerErrorMessage(org.antlr.runtime.Token invalidToken)