org.apache.hudi.table.format.cow

Class ParquetSplitReaderUtil

java.lang.Object

org.apache.hudi.table.format.cow.ParquetSplitReaderUtil

public class ParquetSplitReaderUtil
extends Object

Util for generating ParquetColumnarRowSplitReader.

NOTE: reference from Flink release 1.11.2 ParquetSplitReaderUtil, modify to support INT64 based TIMESTAMP_MILLIS as ConvertedType, should remove when Flink supports that.

Constructor Summary

Constructors

Constructor and Description
ParquetSplitReaderUtil()

Method Summary

Modifier and Type	Method and Description
static org.apache.flink.formats.parquet.vector.reader.ColumnReader	createColumnReader(boolean utcTimestamp, org.apache.flink.table.types.logical.LogicalType fieldType, org.apache.parquet.schema.Type physicalType, List<org.apache.parquet.column.ColumnDescriptor> descriptors, org.apache.parquet.column.page.PageReadStore pages)
static org.apache.flink.table.data.columnar.vector.writable.WritableColumnVector	createWritableColumnVector(int batchSize, org.apache.flink.table.types.logical.LogicalType fieldType, org.apache.parquet.schema.Type physicalType, List<org.apache.parquet.column.ColumnDescriptor> descriptors)
static ParquetColumnarRowSplitReader	genPartColumnarRowReader(boolean utcTimestamp, boolean caseSensitive, org.apache.hadoop.conf.Configuration conf, String[] fullFieldNames, org.apache.flink.table.types.DataType[] fullFieldTypes, Map<String,Object> partitionSpec, int[] selectedFields, int batchSize, org.apache.flink.core.fs.Path path, long splitStart, long splitLength, org.apache.parquet.filter2.predicate.FilterPredicate filterPredicate, org.apache.parquet.filter.UnboundRecordFilter recordFilter) Util for generating partitioned ParquetColumnarRowSplitReader.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ParquetSplitReaderUtil

public ParquetSplitReaderUtil()

Method Detail

genPartColumnarRowReader

public static ParquetColumnarRowSplitReader genPartColumnarRowReader(boolean utcTimestamp,
                                                                     boolean caseSensitive,
                                                                     org.apache.hadoop.conf.Configuration conf,
                                                                     String[] fullFieldNames,
                                                                     org.apache.flink.table.types.DataType[] fullFieldTypes,
                                                                     Map<String,Object> partitionSpec,
                                                                     int[] selectedFields,
                                                                     int batchSize,
                                                                     org.apache.flink.core.fs.Path path,
                                                                     long splitStart,
                                                                     long splitLength,
                                                                     org.apache.parquet.filter2.predicate.FilterPredicate filterPredicate,
                                                                     org.apache.parquet.filter.UnboundRecordFilter recordFilter)
                                                              throws IOException

Util for generating partitioned ParquetColumnarRowSplitReader.

Throws:

IOException

createColumnReader

public static org.apache.flink.formats.parquet.vector.reader.ColumnReader createColumnReader(boolean utcTimestamp,
                                                                                             org.apache.flink.table.types.logical.LogicalType fieldType,
                                                                                             org.apache.parquet.schema.Type physicalType,
                                                                                             List<org.apache.parquet.column.ColumnDescriptor> descriptors,
                                                                                             org.apache.parquet.column.page.PageReadStore pages)
                                                                                      throws IOException

Throws:

IOException

createWritableColumnVector

public static org.apache.flink.table.data.columnar.vector.writable.WritableColumnVector createWritableColumnVector(int batchSize,
                                                                                                                   org.apache.flink.table.types.logical.LogicalType fieldType,
                                                                                                                   org.apache.parquet.schema.Type physicalType,
                                                                                                                   List<org.apache.parquet.column.ColumnDescriptor> descriptors)