org.eclipse.xtext.parser.antlr

Class AbstractSplittingTokenSource

java.lang.Object

org.eclipse.xtext.parser.antlr.AbstractSplittingTokenSource

All Implemented Interfaces:

org.antlr.runtime.TokenSource

Direct Known Subclasses:

AbstractIndentationTokenSource

public abstract class AbstractSplittingTokenSource
extends Object
implements org.antlr.runtime.TokenSource

Delegating token source that is capable of splitting a given token into other tokens. This allows clients to implement sophisticated semantics when dealing with certain token types in different use cases, e.g. a token source that will provide tokens for a python-like whitespace aware language.

Author:

Sebastian Zarnekow - Initial contribution and API

Field Summary

Fields

Modifier and Type	Field and Description
static String	LEXER_RULE_PREFIX Deprecated. use TokenTool.LEXER_RULE_PREFIX if strictly necessary.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	AbstractSplittingTokenSource()

Method Summary

Methods

Modifier and Type	Method and Description
protected org.antlr.runtime.CommonToken	createToken(org.antlr.runtime.CommonToken prototype, String text, Integer charPosInLine, Integer channel, Integer start, Integer stop, Integer type) Create a new token from the given prototype.
protected abstract void	doSplitToken(org.antlr.runtime.Token splitMe, ITokenAcceptor acceptor)
org.antlr.runtime.TokenSource	getDelegate()
String	getLexerRuleName(String antlrTokenDef)
protected String	getRuleName(org.antlr.runtime.Token token)
String	getSourceName()
protected int	getTokenType(String ruleNameWithRulePrefix)
void	initializeTokenDefsFrom(ITokenDefProvider provider)
org.antlr.runtime.Token	nextToken()
void	setDelegate(org.antlr.runtime.TokenSource delegate)
protected abstract boolean	shouldSplitToken(org.antlr.runtime.Token candidate)
protected void	splitToken(org.antlr.runtime.Token splitMe, ITokenAcceptor acceptor)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LEXER_RULE_PREFIX

@Deprecated
public static final String LEXER_RULE_PREFIX

Deprecated. use TokenTool.LEXER_RULE_PREFIX if strictly necessary.

See Also:

Constant Field Values

Constructor Detail

AbstractSplittingTokenSource

protected AbstractSplittingTokenSource()

Method Detail

setDelegate

public void setDelegate(org.antlr.runtime.TokenSource delegate)

getDelegate

public org.antlr.runtime.TokenSource getDelegate()

nextToken

public org.antlr.runtime.Token nextToken()

Specified by:

nextToken in interface org.antlr.runtime.TokenSource

getSourceName

public String getSourceName()

Specified by:

getSourceName in interface org.antlr.runtime.TokenSource

getRuleName

protected String getRuleName(org.antlr.runtime.Token token)

getTokenType

protected int getTokenType(String ruleNameWithRulePrefix)

getLexerRuleName

public String getLexerRuleName(String antlrTokenDef)

initializeTokenDefsFrom

@Inject
public void initializeTokenDefsFrom(ITokenDefProvider provider)

splitToken

protected void splitToken(org.antlr.runtime.Token splitMe,
              ITokenAcceptor acceptor)

shouldSplitToken

protected abstract boolean shouldSplitToken(org.antlr.runtime.Token candidate)

doSplitToken

protected abstract void doSplitToken(org.antlr.runtime.Token splitMe,
                ITokenAcceptor acceptor)

createToken

protected org.antlr.runtime.CommonToken createToken(org.antlr.runtime.CommonToken prototype,
                                        String text,
                                        Integer charPosInLine,
                                        Integer channel,
                                        Integer start,
                                        Integer stop,
                                        Integer type)

Create a new token from the given prototype. Any argument besides the prototype is optional and will be ignored if its value is null.