Package org.apache.pinot.segment.local.recordtransformer

Class ComplexTypeTransformer

  • All Implemented Interfaces:
    Serializable, RecordTransformer
    public class ComplexTypeTransformer
extends Object
implements RecordTransformer
    A transformer to handle the complex types such as Map and Collection, with flattening and unnesting.

    The map flattening rule will recursively flatten all the map types, except for those under the collection that is not marked as to unnest. For example: 

     {
    "t1":{
       "array":[
          {
             "t2":{
                "a":"v1"
             }
          }
       ]
    }
 }
    flattens to 
     {
    "t1.array":[
       {
          "t2.a":"v1"
       }
    ]
 }

    The unnesting rule will flatten all the collections provided, which are the paths navigating to the collections. For the same example above. If the the collectionToUnnest is provided as "t1.array", then the rule will unnest the previous output to: 
     [
    {
       "t1.arrayt2.a": "v1",
    }
 ]
    Note the unnest rule will output a collection of generic rows under the field GenericRow.MULTIPLE_RECORDS_KEY. TODO: support multi-dimensional array handling
    See Also:
    Serialized Form

    • Constructor Detail

      • ComplexTypeTransformer

        public ComplexTypeTransformer​(TableConfig tableConfig)

    • Method Detail

      • getComplexTypeTransformer

        @Nullable
public static ComplexTypeTransformer getComplexTypeTransformer​(TableConfig tableConfig)
        Returns:
        the complex type transformer defined table config, null if the table config does not have the config

      • transform

        public GenericRow transform​(GenericRow record)
        Description copied from interface: RecordTransformer
        Transforms a record based on some custom rules.
        Specified by:
        transform in interface RecordTransformer
        Parameters:
        record - Record to transform
        Returns:
        Transformed record, or null if the record does not follow certain rules.

      • flattenMap

        protected void flattenMap​(GenericRow record,
                          List<String> columns)
        Recursively flatten all the Maps in the record. It will also navigate into the collections marked as "unnest" and flatten the nested maps.

      • renamePrefixes

        protected void renamePrefixes​(GenericRow record)
        Loops through all columns and renames the column's prefix with the corresponding replacement if the prefix matches.

      • isNonPrimitiveArray

        protected static boolean isNonPrimitiveArray​(Object obj)