Interface RecordExtractor<T>

  • Type Parameters:
    T - The format of the input record
    All Superinterfaces:
    Serializable
    All Known Implementing Classes:
    BaseRecordExtractor

    public interface RecordExtractor<T>
    extends Serializable
    Extracts fields from input records 1) Number/String/ByteBuffer become single-value column 2) Collections become Object[] i.e. multi-value column 3) Nested/Complex fields (e.g. json maps, avro maps, avro records) become Map
    • Method Summary

      Modifier and Type Method Description
      Object convert​(Object value)
      Converts a field of the given input record.
      GenericRow extract​(T from, GenericRow to)
      Extracts fields as listed in the sourceFieldNames from the given input record and sets them into the GenericRow
      void init​(Set<String> fields, RecordExtractorConfig recordExtractorConfig)
      Initialize the record extractor with its config
    • Method Detail

      • init

        void init​(@Nullable
                  Set<String> fields,
                  RecordExtractorConfig recordExtractorConfig)
        Initialize the record extractor with its config
        Parameters:
        fields - List of field names to extract from the provided input record. If null or empty, extracts all fields.
        recordExtractorConfig - The record extractor config
      • extract

        GenericRow extract​(T from,
                           GenericRow to)
        Extracts fields as listed in the sourceFieldNames from the given input record and sets them into the GenericRow
        Parameters:
        from - The input record
        to - The output GenericRow
        Returns:
        The output GenericRow
      • convert

        @Nullable
        Object convert​(Object value)
        Converts a field of the given input record. The field value will be converted to either a single value (string, number, byte[]), multi value (Object[]) or a Map. Natively Pinot only understands single values and multi values. Map is useful only if some ingestion transform functions operates on it in the transformation layer.
        Parameters:
        value - the field value to be converted
        Returns:
        The converted field value. Returns null for empty array/collection/map.