Package org.apache.pinot.spi.data.readers

Class BaseRecordExtractor<T>

  • Type Parameters:
    T - the format of the input record
    All Implemented Interfaces:
    Serializable, RecordExtractor<T>
    public abstract class BaseRecordExtractor<T>
extends Object
implements RecordExtractor<T>
    Base abstract class for extracting and converting the fields of various data formats into supported Pinot data types.
    See Also:
    Serialized Form

    • Constructor Detail

      • BaseRecordExtractor

        public BaseRecordExtractor()

    • Method Detail

      • convert

        @Nullable
public Object convert​(Object value)
        Converts the field value to either a single value (string, number, byte[]), multi value (Object[]) or a Map. Returns null if the value is an empty array/collection/map. Natively Pinot only understands single values and multi values. Map is useful only if some ingestion transform functions operates on it in the transformation layer.
        Specified by:
        convert in interface RecordExtractor<T>
        Parameters:
        value - the field value to be converted
        Returns:
        The converted field value. Returns null for empty array/collection/map.

      • isRecord

        protected boolean isRecord​(Object value)
        Returns whether the object is of the data format's base type. Override this method if the extractor can handle the conversion of nested record types.

      • isMultiValue

        protected boolean isMultiValue​(Object value)
        Returns whether the object is of a multi-value type. Override this method if the data format represents multi-value objects differently.

      • isMap

        protected boolean isMap​(Object value)
        Returns whether the object is of a map type. Override this method if the data format represents map objects differently.

      • convertRecord

        @Nullable
protected Object convertRecord​(Object value)
        Handles the conversion of every field of the object for the particular data format. Override this method if the extractor can convert nested record types.
        Parameters:
        value - should be verified to be a record type prior to calling this method as it will be handled with this assumption

      • convertMultiValue

        @Nullable
protected Object convertMultiValue​(Object value)
        Handles the conversion of each element of a multi-value object. Returns null if the field value is null. This implementation converts the Collection to an Object array. Any elements of the Collection that are null will be excluded from the returned multi-value object. Override this method if the data format requires a different conversion for its multi-value objects.
        Parameters:
        value - should be verified to be a Collection type prior to calling this method as it will be casted to a Collection without checking

      • convertMap

        @Nullable
protected Object convertMap​(Object value)
        Handles the conversion of every value of the map. Note that map keys will be handled as a single-value type. Returns null if the field value is null. This should be overridden if the data format requires a different conversion for map values.
        Parameters:
        value - should be verified to be a Map type prior to calling this method as it will be casted to a Map without checking

      • convertSingleValue

        protected Object convertSingleValue​(Object value)
        Converts single value types. This should be overridden if the data format requires a different conversion for its single values.