Package io.trino.cli.lexer

Class DelimiterLexer

java.lang.Object
org.antlr.v4.runtime.Recognizer<Integer,org.antlr.v4.runtime.atn.LexerATNSimulator>
org.antlr.v4.runtime.Lexer
io.trino.grammar.sql.SqlBaseLexer
io.trino.cli.lexer.DelimiterLexer
All Implemented Interfaces:
org.antlr.v4.runtime.TokenSource
public class DelimiterLexer extends io.trino.grammar.sql.SqlBaseLexer
This is a special-purpose lexer that can identify custom delimiters in addition to every other token in the SQL grammar.

The code in nextToken() is a copy of the implementation in org.antlr.v4.runtime.Lexer, with a bit added to match the token before the default behavior is invoked.

  • Field Summary

    Fields inherited from class io.trino.grammar.sql.SqlBaseLexer

    _ATN, _decisionToDFA, _serializedATN, _sharedContextCache, ABSENT, ADD, ADMIN, AFTER, ALL, ALTER, ANALYZE, AND, ANY, ARRAY, AS, ASC, ASTERISK, AT, AUTHORIZATION, BACKQUOTED_IDENTIFIER, BEGIN, BERNOULLI, BETWEEN, BINARY_LITERAL, BOTH, BRACKETED_COMMENT, BY, CALL, CALLED, CASCADE, CASE, CAST, CATALOG, CATALOGS, channelNames, COLUMN, COLUMNS, COMMENT, COMMIT, COMMITTED, CONCAT, CONDITIONAL, CONSTRAINT, COPARTITION, COUNT, CREATE, CROSS, CUBE, CURRENT, CURRENT_CATALOG, CURRENT_DATE, CURRENT_PATH, CURRENT_ROLE, CURRENT_SCHEMA, CURRENT_TIME, CURRENT_TIMESTAMP, CURRENT_USER, DATA, DATE, DAY, DEALLOCATE, DECIMAL_VALUE, DECLARE, DEFAULT, DEFINE, DEFINER, DELETE, DENY, DESC, DESCRIBE, DESCRIPTOR, DETERMINISTIC, DIGIT_IDENTIFIER, DISTINCT, DISTRIBUTED, DO, DOUBLE, DOUBLE_VALUE, DROP, ELSE, ELSEIF, EMPTY, ENCODING, END, EQ, ERROR, ESCAPE, EXCEPT, EXCLUDING, EXECUTE, EXISTS, EXPLAIN, EXTRACT, FALSE, FETCH, FILTER, FINAL, FIRST, FOLLOWING, FOR, FORMAT, FROM, FULL, FUNCTION, FUNCTIONS, GRACE, GRANT, GRANTED, GRANTS, GRAPHVIZ, GROUP, GROUPING, GROUPS, GT, GTE, HAVING, HOUR, IDENTIFIER, IF, IGNORE, IMMEDIATE, IN, INCLUDING, INITIAL, INNER, INPUT, INSERT, INTEGER_VALUE, INTERSECT, INTERVAL, INTO, INVOKER, IO, IS, ISOLATION, ITERATE, JOIN, JSON, JSON_ARRAY, JSON_EXISTS, JSON_OBJECT, JSON_QUERY, JSON_TABLE, JSON_VALUE, KEEP, KEY, KEYS, LANGUAGE, LAST, LATERAL, LEADING, LEAVE, LEFT, LEVEL, LIKE, LIMIT, LISTAGG, LOCAL, LOCALTIME, LOCALTIMESTAMP, LOGICAL, LOOP, LT, LTE, MAP, MATCH, MATCH_RECOGNIZE, MATCHED, MATCHES, MATERIALIZED, MEASURES, MERGE, MINUS, MINUTE, modeNames, MONTH, NATURAL, NEQ, NESTED, NEXT, NFC, NFD, NFKC, NFKD, NO, NONE, NORMALIZE, NOT, NULL, NULLIF, NULLS, OBJECT, OF, OFFSET, OMIT, ON, ONE, ONLY, OPTION, OR, ORDER, ORDINALITY, OUTER, OUTPUT, OVER, OVERFLOW, PARTITION, PARTITIONS, PASSING, PAST, PATH, PATTERN, PER, PERCENT, PERIOD, PERMUTE, PLAN, PLUS, POSITION, PRECEDING, PRECISION, PREPARE, PRIVILEGES, PROPERTIES, PRUNE, QUESTION_MARK, QUOTED_IDENTIFIER, QUOTES, RANGE, READ, RECURSIVE, REFRESH, RENAME, REPEAT, REPEATABLE, REPLACE, RESET, RESPECT, RESTRICT, RETURN, RETURNING, RETURNS, REVOKE, RIGHT, ROLE, ROLES, ROLLBACK, ROLLUP, ROW, ROWS, ruleNames, RUNNING, SCALAR, SCHEMA, SCHEMAS, SECOND, SECURITY, SEEK, SELECT, SEMICOLON, SERIALIZABLE, SESSION, SET, SETS, SHOW, SIMPLE_COMMENT, SLASH, SOME, START, STATS, STRING, SUBSET, SUBSTRING, SYSTEM, T__0, T__1, T__10, T__11, T__12, T__13, T__14, T__15, T__16, T__2, T__3, T__4, T__5, T__6, T__7, T__8, T__9, TABLE, TABLES, TABLESAMPLE, TEXT, TEXT_STRING, THEN, TIES, TIME, TIMESTAMP, TO, tokenNames, TRAILING, TRANSACTION, TRIM, TRUE, TRUNCATE, TRY_CAST, TYPE, UESCAPE, UNBOUNDED, UNCOMMITTED, UNCONDITIONAL, UNICODE_STRING, UNION, UNIQUE, UNKNOWN, UNMATCHED, UNNEST, UNRECOGNIZED, UNTIL, UPDATE, USE, USER, USING, UTF16, UTF32, UTF8, VALIDATE, VALUE, VALUES, VERBOSE, VERSION, VIEW, VOCABULARY, WHEN, WHERE, WHILE, WINDOW, WITH, WITHIN, WITHOUT, WORK, WRAPPER, WRITE, WS, YEAR, ZONE

    Fields inherited from class org.antlr.v4.runtime.Lexer

    _channel, _factory, _hitEOF, _input, _mode, _modeStack, _text, _token, _tokenFactorySourcePair, _tokenStartCharIndex, _tokenStartCharPositionInLine, _tokenStartLine, _type, DEFAULT_MODE, DEFAULT_TOKEN_CHANNEL, HIDDEN, MAX_CHAR_VALUE, MIN_CHAR_VALUE, MORE, SKIP

    Fields inherited from class org.antlr.v4.runtime.Recognizer

    _interp, EOF

  • Constructor Summary

    Constructors
    Constructor
    Description
    DelimiterLexer(org.antlr.v4.runtime.CharStream input, Set<String> delimiters)
     

  • Method Summary

    Modifier and Type
    Method
    Description
    boolean
    isDelimiter(org.antlr.v4.runtime.Token token)
     
    org.antlr.v4.runtime.Token
    nextToken()
     

    Methods inherited from class io.trino.grammar.sql.SqlBaseLexer

    getATN, getChannelNames, getGrammarFileName, getModeNames, getRuleNames, getSerializedATN, getTokenNames, getVocabulary

    Methods inherited from class org.antlr.v4.runtime.Lexer

    emit, emit, emitEOF, getAllTokens, getChannel, getCharErrorDisplay, getCharIndex, getCharPositionInLine, getErrorDisplay, getErrorDisplay, getInputStream, getLine, getSourceName, getText, getToken, getTokenFactory, getType, mode, more, notifyListeners, popMode, pushMode, recover, recover, reset, setChannel, setCharPositionInLine, setInputStream, setLine, setText, setToken, setTokenFactory, setType, skip

    Methods inherited from class org.antlr.v4.runtime.Recognizer

    action, addErrorListener, getErrorHeader, getErrorListenerDispatch, getErrorListeners, getInterpreter, getParseInfo, getRuleIndexMap, getState, getTokenErrorDisplay, getTokenType, getTokenTypeMap, precpred, removeErrorListener, removeErrorListeners, sempred, setInterpreter, setState

    Methods inherited from class java.lang.Object

    clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

  • Constructor Details

    • DelimiterLexer

      public DelimiterLexer(org.antlr.v4.runtime.CharStream input, Set<String> delimiters)

  • Method Details

    • isDelimiter

      public boolean isDelimiter(org.antlr.v4.runtime.Token token)

    • nextToken

      public org.antlr.v4.runtime.Token nextToken()
      Specified by:
      nextToken in interface org.antlr.v4.runtime.TokenSource
      Overrides:
      nextToken in class org.antlr.v4.runtime.Lexer