Class TransformPipeline


  • public class TransformPipeline
    extends Object
    The class for transforming validating GenericRow data against table schema and table config. It is used mainly but not limited by RealTimeDataManager for each row that is going to be indexed into Pinot.
    • Constructor Detail

      • TransformPipeline

        public TransformPipeline​(RecordTransformer recordTransformer,
                                 @Nullable
                                 ComplexTypeTransformer complexTypeTransformer)
        Constructs a transform pipeline with customized RecordTransformer and customized ComplexTypeTransformer
        Parameters:
        recordTransformer - the customized record transformer
        complexTypeTransformer - the customized complexType transformer
      • TransformPipeline

        public TransformPipeline​(TableConfig tableConfig,
                                 Schema schema)
        Constructs a transform pipeline based on TableConfig and table schema.
        Parameters:
        tableConfig - the config for the table
        schema - the table schema
    • Method Detail

      • getPassThroughPipeline

        public static TransformPipeline getPassThroughPipeline()
        Returns a pass through pipeline that does not transform the record.
      • processRow

        public void processRow​(GenericRow decodedRow,
                               TransformPipeline.Result reusedResult)
                        throws Exception
        Process and validate the decoded row against schema.
        Parameters:
        decodedRow - the row data to pass in
        reusedResult - the reused result so we can reduce objects created for each row
        Throws:
        Exception - when data has issues like schema validation. Fetch the partialResult from Exception