Package org.eclipse.xtext.parser.antlr

Class AbstractSplittingTokenSource

java.lang.Object

org.eclipse.xtext.parser.antlr.AbstractSplittingTokenSource

All Implemented Interfaces:

org.antlr.runtime.TokenSource

Direct Known Subclasses:

AbstractIndentationTokenSource

public abstract class AbstractSplittingTokenSource
extends java.lang.Object
implements org.antlr.runtime.TokenSource

Delegating token source that is capable of splitting a given token into other tokens. This allows clients to implement sophisticated semantics when dealing with certain token types in different use cases, e.g. a token source that will provide tokens for a python-like whitespace aware language.

Field Summary

Fields

Modifier and Type	Field	Description
static java.lang.String	LEXER_RULE_PREFIX	Deprecated. use TokenTool.LEXER_RULE_PREFIX if strictly necessary.

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	AbstractSplittingTokenSource()

Method Summary

Modifier and Type	Method	Description
protected org.antlr.runtime.CommonToken	createToken(org.antlr.runtime.CommonToken prototype, java.lang.String text, java.lang.Integer charPosInLine, java.lang.Integer channel, java.lang.Integer start, java.lang.Integer stop, java.lang.Integer type)	Create a new token from the given prototype.
protected abstract void	doSplitToken(org.antlr.runtime.Token splitMe, ITokenAcceptor acceptor)
org.antlr.runtime.TokenSource	getDelegate()
java.lang.String	getLexerRuleName(java.lang.String antlrTokenDef)
protected java.lang.String	getRuleName(org.antlr.runtime.Token token)
java.lang.String	getSourceName()
protected int	getTokenType(java.lang.String ruleNameWithRulePrefix)
void	initializeTokenDefsFrom(ITokenDefProvider provider)
org.antlr.runtime.Token	nextToken()
void	setDelegate(org.antlr.runtime.TokenSource delegate)
protected abstract boolean	shouldSplitToken(org.antlr.runtime.Token candidate)
protected void	splitToken(org.antlr.runtime.Token splitMe, ITokenAcceptor acceptor)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LEXER_RULE_PREFIX

@Deprecated
public static final java.lang.String LEXER_RULE_PREFIX

Deprecated.

use TokenTool.LEXER_RULE_PREFIX if strictly necessary.

See Also:

Constant Field Values

Constructor Detail

AbstractSplittingTokenSource

protected AbstractSplittingTokenSource()

Method Detail

setDelegate

public void setDelegate(org.antlr.runtime.TokenSource delegate)

getDelegate

public org.antlr.runtime.TokenSource getDelegate()

nextToken

public org.antlr.runtime.Token nextToken()

Specified by:

nextToken in interface org.antlr.runtime.TokenSource

getSourceName

public java.lang.String getSourceName()

Specified by:

getSourceName in interface org.antlr.runtime.TokenSource

getRuleName

protected java.lang.String getRuleName(org.antlr.runtime.Token token)

getTokenType

protected int getTokenType(java.lang.String ruleNameWithRulePrefix)

getLexerRuleName

public java.lang.String getLexerRuleName(java.lang.String antlrTokenDef)

initializeTokenDefsFrom

@Inject
public void initializeTokenDefsFrom(ITokenDefProvider provider)

splitToken

protected void splitToken(org.antlr.runtime.Token splitMe,
                          ITokenAcceptor acceptor)

shouldSplitToken

protected abstract boolean shouldSplitToken(org.antlr.runtime.Token candidate)

doSplitToken

protected abstract void doSplitToken(org.antlr.runtime.Token splitMe,
                                     ITokenAcceptor acceptor)

createToken

protected org.antlr.runtime.CommonToken createToken(org.antlr.runtime.CommonToken prototype,
                                                    java.lang.String text,
                                                    java.lang.Integer charPosInLine,
                                                    java.lang.Integer channel,
                                                    java.lang.Integer start,
                                                    java.lang.Integer stop,
                                                    java.lang.Integer type)

Create a new token from the given prototype. Any argument besides the prototype is optional and will be ignored if its value is null.