Package uk.co.real_logic.sbe.ir

Class Token

java.lang.Object
uk.co.real_logic.sbe.ir.Token
public class Token extends Object
Class to encapsulate a token of information for the message schema stream. This Intermediate Representation (IR) is intended to be language, schema, platform independent.

Processing and optimization could be run over a list of Tokens to perform various functions

  • re-ordering of fields based on encodedLength
  • padding of fields in order to provide expansion room
  • computing offsets of individual fields
  • etc.

IR could be used to generate code or other specifications. It should be possible to do the following:

  • generate a FIX/SBE schema from IR
  • generate an ASN.1 spec from IR
  • generate a GPB spec from IR
  • etc.

IR could be serialized to storage or network via code generated by SBE. Then read back in to a List of Tokens.

The entire IR of an entity is a List of Token objects. The order of this list is very important. Encoding of fields is done by nodes pointing to specific encoding PrimitiveType objects. Each encoding node contains encodedLength, offset, byte order, and Encoding. Entities relevant to the encoding such as fields, messages, repeating groups, etc. are encapsulated in the list as nodes themselves. Although, they will in most cases never be serialized. The boundaries of these entities are delimited by BEGIN and END Signal values in the node Encoding. A list structure like this allows for each concatenation of encodings as well as easy traversal.

An example encoding of a message headerStructure might be like this.

  • Token 0 - Signal = BEGIN_MESSAGE, schemaId = 100
  • Token 1 - Signal = BEGIN_FIELD, schemaId = 25
  • Token 2 - Signal = ENCODING, PrimitiveType = uint32, encodedLength = 4, offset = 0
  • Token 3 - Signal = END_FIELD
  • Token 4 - Signal = END_MESSAGE

  • Field Details

    • INVALID_ID

      public static final int INVALID_ID
      Invalid ID value.
      See Also:

    • VARIABLE_LENGTH

      public static final int VARIABLE_LENGTH
      Length not determined
      See Also:

  • Constructor Details

    • Token

      public Token(Signal signal, String name, String referencedName, String description, String packageName, int id, int version, int deprecated, int encodedLength, int offset, int componentTokenCount, Encoding encoding)
      Construct an Token by providing values for all fields.
      Parameters:
      signal - for the token role.
      name - of the token in the message.
      referencedName - of the type when created from a ref in a composite.
      description - of what the token is for.
      packageName - of the token in the message. Use null, except for BEGIN_MESSAGE tokens for types that require an explicit package.
      id - as the identifier in the message declaration.
      version - application within the template.
      deprecated - as of this version.
      encodedLength - of the component part.
      offset - in the underlying message as octets.
      componentTokenCount - number of tokens in this component.
      encoding - of the primitive field.

  • Method Details

    • signal

      public Signal signal()
      Signal the role of this token.
      Returns:
      the Signal for the token.

    • name

      public String name()
      Return the name of the token
      Returns:
      name of the token

    • packageName

      public String packageName()
      Return the packageName of the token
      Returns:
      packageName of the token or null, if it was not set explicitly.

    • referencedName

      public String referencedName()
      Get the name of the type when this is from a reference.
      Returns:
      the name of the type when this is from a reference.

    • description

      public String description()
      Description for what the token is to be used for.
      Returns:
      description for what the token is to be used for.

    • id

      public int id()
      Return the ID of the token assigned by the specification
      Returns:
      ID of the token assigned by the specification

    • version

      public int version()
      The version context for this token. This is the schema version in which the type was introduced.
      Returns:
      version for this type.

    • deprecated

      public int deprecated()
      The version in which this context was deprecated.
      Returns:
      the version in which this context was deprecated.

    • applicableTypeName

      public String applicableTypeName()
      Get the name of the type that should be applied in context.
      Returns:
      the name of the type that should be applied in context.

    • encodedLength

      public int encodedLength()
      The encodedLength of this token in bytes.
      Returns:
      the encodedLength of this node. A value of 0 means the node has no encodedLength when encoded. A value of VARIABLE_LENGTH means this node represents a variable length field.

    • encodedLength

      public void encodedLength(int encodedLength)
      Set the encoded length for this node. See encodedLength().
      Parameters:
      encodedLength - that is overriding existing value.

    • arrayLength

      public int arrayLength()
      The number of encoded primitives in this type.
      Returns:
      number of encoded primitives in this type.

    • matchOnLength

      public CharSequence matchOnLength(Supplier<CharSequence> one, Supplier<CharSequence> many)
      Match which approach to take based on the length of the token. If length is zero then an empty String is returned.
      Parameters:
      one - to be used when length is one.
      many - to be used when length is greater than one.
      Returns:
      the CharSequence representing the token depending on the length.

    • offset

      public int offset()
      The offset for this token in the message.
      Returns:
      the offset of this Token. A value of 0 means the node has no relevant offset. A value of VARIABLE_LENGTH means this node's true offset is dependent on variable length fields ahead of it in the encoding.

    • componentTokenCount

      public int componentTokenCount()
      The number of tokens that make up this component.
      Returns:
      the number of tokens that make up this component.

    • componentTokenCount

      public void componentTokenCount(int componentTokenCount)
      Set the number of tokens this component has.
      Parameters:
      componentTokenCount - the number of tokens this component has.

    • encoding

      public Encoding encoding()
      Return the Encoding of the Token.
      Returns:
      encoding of the Token

    • isConstantEncoding

      public boolean isConstantEncoding()
      Is the encoding presence is a constant or not?
      Returns:
      true if the encoding presence is a constant or false if not.

    • isOptionalEncoding

      public boolean isOptionalEncoding()
      Is the encoding presence is optional or not?
      Returns:
      true if the encoding presence is optional or false if not.

    • toString

      public String toString()
      Overrides:
      toString in class Object