Package org.eclipse.xtext.parser.antlr

Class AbstractIndentationTokenSource

  • All Implemented Interfaces:
    org.antlr.runtime.TokenSource
    public abstract class AbstractIndentationTokenSource
extends AbstractSplittingTokenSource
    Abstract implementation of a token source, that splits according to indentation levels after newlines. Blank lines are ignored. At the end of file token, all pending indentations will be handled and dedentation tokens will be issued accordingly if shouldEmitPendingEndTokens() returns true.
    Since:
    2.8

    • Field Detail

      • indentationStack

        protected java.util.Stack<java.lang.Integer> indentationStack

      • currentOffset

        protected int currentOffset

      • currentIndentation

        protected int currentIndentation

      • currentEnd

        protected int currentEnd

      • nextOffset

        protected int nextOffset

    • Constructor Detail

      • AbstractIndentationTokenSource

        protected AbstractIndentationTokenSource​(org.antlr.runtime.TokenSource delegate)

    • Method Detail

      • shouldSplitTokenImpl

        protected abstract boolean shouldSplitTokenImpl​(org.antlr.runtime.Token token)
        Return true if the given token is a whitespace token that should be splitted. Usually the token type will be compared to one of the int constants from the parser.

      • doSplitEofToken

        protected void doSplitEofToken​(org.antlr.runtime.Token token,
                               ITokenAcceptor result)
        Since:
        2.13

      • shouldEmitPendingEndTokens

        protected boolean shouldEmitPendingEndTokens()

      • nextToken

        public org.antlr.runtime.Token nextToken()
        Specified by:
        nextToken in interface org.antlr.runtime.TokenSource
        Overrides:
        nextToken in class AbstractSplittingTokenSource

      • doSplitTokenImpl

        protected void doSplitTokenImpl​(org.antlr.runtime.Token token,
                                ITokenAcceptor result)
        The token was previously determined as potentially to-be-splitted thus we emit additional indentation or dedenting tokens.

      • handleRemainingText

        protected void handleRemainingText​(org.antlr.runtime.Token token,
                                   java.lang.String text,
                                   int indentation,
                                   ITokenAcceptor result)
        Since:
        2.13

      • decreaseIndentation

        protected void decreaseIndentation​(int indentation,
                                   ITokenAcceptor result)

      • splitWithText

        protected void splitWithText​(org.antlr.runtime.Token token,
                             java.lang.String text,
                             ITokenAcceptor result)

      • createEndToken

        protected org.antlr.runtime.Token createEndToken​(int offset)

      • getEndTokenType

        protected abstract int getEndTokenType()

      • getBeginTokenType

        protected abstract int getBeginTokenType()

      • createBeginToken

        protected org.antlr.runtime.Token createBeginToken​(int offset)

      • computeIndentationRelevantCharCount

        protected int computeIndentationRelevantCharCount​(java.lang.String text)

      • computeIndentation

        protected int computeIndentation​(java.lang.String text)

      • getTabWidth

        protected char getTabWidth()