Package org.apache.pinot.spi.data

Class Schema

  • All Implemented Interfaces:
    Serializable
    public final class Schema
extends Object
implements Serializable
    The Schema class is defined for each table to describe the details of the table's fields (columns).

    Four field types are supported: DIMENSION, METRIC, TIME, DATE_TIME. (DimensionFieldSpec, MetricFieldSpec, TimeFieldSpec, DateTimeFieldSpec)

    For each field, a FieldSpec is defined to provide the details of the field.

    There could be multiple DIMENSION or METRIC or DATE_TIME fields, but at most 1 TIME field.

    In pinot, we store data using 5 DataTypes: INT, LONG, FLOAT, DOUBLE, STRING. All other DataTypes will be converted to one of them.

    See Also:
    Serialized Form

    • Constructor Detail

      • Schema

        public Schema()

    • Method Detail

      • getSchemaName

        public String getSchemaName()
        NOTE: schema name could be null in tests

      • setSchemaName

        public void setSchemaName​(String schemaName)

      • getPrimaryKeyColumns

        public List<String> getPrimaryKeyColumns()

      • setPrimaryKeyColumns

        public void setPrimaryKeyColumns​(List<String> primaryKeyColumns)

      • setDimensionFieldSpecs

        @Deprecated
public void setDimensionFieldSpecs​(List<DimensionFieldSpec> dimensionFieldSpecs)
        Deprecated.
        Required by JSON deserializer. DO NOT USE. DO NOT REMOVE. Adding @Deprecated to prevent usage

      • setMetricFieldSpecs

        @Deprecated
public void setMetricFieldSpecs​(List<MetricFieldSpec> metricFieldSpecs)
        Deprecated.
        Required by JSON deserializer. DO NOT USE. DO NOT REMOVE. Adding @Deprecated to prevent usage

      • setDateTimeFieldSpecs

        @Deprecated
public void setDateTimeFieldSpecs​(List<DateTimeFieldSpec> dateTimeFieldSpecs)
        Deprecated.
        Required by JSON deserializer. DO NOT USE. DO NOT REMOVE. Adding @Deprecated to prevent usage

      • setTimeFieldSpec

        @Deprecated
public void setTimeFieldSpec​(TimeFieldSpec timeFieldSpec)
        Deprecated.
        Required by JSON deserializer. DO NOT USE. DO NOT REMOVE. Adding @Deprecated to prevent usage

      • addField

        public void addField​(FieldSpec fieldSpec)

      • removeField

        public boolean removeField​(String columnName)

      • hasColumn

        public boolean hasColumn​(String columnName)

      • hasJSONColumn

        public boolean hasJSONColumn()

      • getColumnNames

        public Set<String> getColumnNames()

      • getPhysicalColumnNames

        public Set<String> getPhysicalColumnNames()

      • size

        public int size()

      • getFieldSpecFor

        public FieldSpec getFieldSpecFor​(String columnName)

      • getSpecForTimeColumn

        @Nullable
public DateTimeFieldSpec getSpecForTimeColumn​(String timeColumnName)
        Fetches the DateTimeFieldSpec for the given time column name. If the columnName is a DATE_TIME column, returns the DateTimeFieldSpec If the columnName is a TIME column, converts to DateTimeFieldSpec before returning

      • getDimensionNames

        public List<String> getDimensionNames()

      • getMetricNames

        public List<String> getMetricNames()

      • getDateTimeNames

        public List<String> getDateTimeNames()

      • toJsonObject

        public com.fasterxml.jackson.databind.node.ObjectNode toJsonObject()
        Returns a json representation of the schema.

      • toPrettyJsonString

        public String toPrettyJsonString()
        Returns a pretty json string representation of the schema.

      • toSingleLineJsonString

        public String toSingleLineJsonString()
        Returns a single-line json string representation of the schema.

      • validate

        public void validate()
        Validates a pinot schema.

        The following validations are performed:

        • For dimension, time, date time fields, support FieldSpec.DataType: INT, LONG, FLOAT, DOUBLE, BOOLEAN, TIMESTAMP, STRING, BYTES
        • For metric fields, support FieldSpec.DataType: INT, LONG, FLOAT, DOUBLE, BYTES

      • updateBooleanFieldsIfNeeded

        public void updateBooleanFieldsIfNeeded​(Schema oldSchema)
        Updates fields with BOOLEAN data type to STRING if the data type in the old schema is STRING. BOOLEAN data type was stored as STRING within the schema before release 0.8.0. In release 0.8.0, we introduced native BOOLEAN support and BOOLEAN data type is no longer replaced with STRING. To keep the existing schema backward compatible, when the new field spec has BOOLEAN data type and the old field spec has STRING data type, set the new field spec's data type to STRING.

      • isBackwardCompatibleWith

        public boolean isBackwardCompatibleWith​(Schema oldSchema)
        Check whether the current schema is backward compatible with oldSchema. Backward compatibility requires (1) all columns in oldSchema should be retained. (2) all column fieldSpecs should be backward compatible with the old ones.
        Parameters:
        oldSchema - old schema

      • hashCode

        public int hashCode()
        Overrides:
        hashCode in class Object

      • convertToDateTimeFieldSpec

        public static DateTimeFieldSpec convertToDateTimeFieldSpec​(TimeFieldSpec timeFieldSpec)
        Helper method that converts a TimeFieldSpec to DateTimeFieldSpec 1) If timeFieldSpec contains only incoming granularity spec, directly convert it to a dateTimeFieldSpec 2) If timeFieldSpec contains incoming aas well as outgoing granularity spec, use the outgoing spec to construct the dateTimeFieldSpec, and configure a transform function for the conversion from incoming