org.apache.paimon.format.parquet.reader

类 AbstractColumnReader<VECTOR extends org.apache.paimon.data.columnar.writable.WritableColumnVector>

java.lang.Object

org.apache.paimon.format.parquet.reader.AbstractColumnReader<VECTOR>

所有已实现的接口:

ColumnReader<VECTOR>

直接已知子类:

BooleanColumnReader, ByteColumnReader, BytesColumnReader, DoubleColumnReader, FixedLenBytesColumnReader, FloatColumnReader, IntColumnReader, LongColumnReader, ShortColumnReader, TimestampColumnReader

public abstract class AbstractColumnReader<VECTOR extends org.apache.paimon.data.columnar.writable.WritableColumnVector>
extends Object
implements ColumnReader<VECTOR>

Abstract ColumnReader. See ColumnReaderImpl, part of the code is referred from Apache Spark and Apache Parquet.

字段概要

字段

限定符和类型	字段和说明
protected org.apache.parquet.column.ColumnDescriptor	descriptor
protected org.apache.parquet.column.Dictionary	dictionary The dictionary, if this column has dictionary encoding.
protected int	maxDefLevel Maximum definition level for this column.
protected org.apache.paimon.format.parquet.reader.RunLengthDecoder	runLenDecoder Run length decoder for data and dictionary.

构造器概要

构造器

构造器和说明
AbstractColumnReader(org.apache.parquet.column.ColumnDescriptor descriptor, org.apache.parquet.column.page.PageReader pageReader)

方法概要

限定符和类型	方法和说明
protected void	afterReadPage() After read a page, we may need some initialization.
protected void	checkTypeName(org.apache.parquet.schema.PrimitiveType.PrimitiveTypeName expectedName)
protected abstract void	readBatch(int rowId, int num, VECTOR column) Read batch from runLenDecoder and dataInputStream.
protected abstract void	readBatchFromDictionaryIds(int rowId, int num, VECTOR column, org.apache.paimon.data.columnar.writable.WritableIntVector dictionaryIds) Decode dictionary ids to data.
void	readToVector(int readNumber, VECTOR vector) Reads `total` values from this columnReader into column.
protected boolean	supportLazyDecode() Support lazy dictionary ids decode.

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

字段详细资料

dictionary

protected final org.apache.parquet.column.Dictionary dictionary

The dictionary, if this column has dictionary encoding.

maxDefLevel

protected final int maxDefLevel

Maximum definition level for this column.

descriptor

protected final org.apache.parquet.column.ColumnDescriptor descriptor

runLenDecoder

protected org.apache.paimon.format.parquet.reader.RunLengthDecoder runLenDecoder

Run length decoder for data and dictionary.

构造器详细资料

AbstractColumnReader

public AbstractColumnReader(org.apache.parquet.column.ColumnDescriptor descriptor,
                            org.apache.parquet.column.page.PageReader pageReader)
                     throws IOException

抛出:

IOException

方法详细资料

checkTypeName

protected void checkTypeName(org.apache.parquet.schema.PrimitiveType.PrimitiveTypeName expectedName)

readToVector

public final void readToVector(int readNumber,
                               VECTOR vector)
                        throws IOException

Reads `total` values from this columnReader into column.

指定者:

readToVector 在接口中 ColumnReader<VECTOR extends org.apache.paimon.data.columnar.writable.WritableColumnVector>

参数:

readNumber - number to read.

vector - vector to write.

抛出:

IOException

afterReadPage

protected void afterReadPage()

After read a page, we may need some initialization.

supportLazyDecode

protected boolean supportLazyDecode()

Support lazy dictionary ids decode. See more in ParquetDictionary. If return false, we will decode all the data first.

readBatch

protected abstract void readBatch(int rowId,
                                  int num,
                                  VECTOR column)

Read batch from runLenDecoder and dataInputStream.

readBatchFromDictionaryIds

protected abstract void readBatchFromDictionaryIds(int rowId,
                                                   int num,
                                                   VECTOR column,
                                                   org.apache.paimon.data.columnar.writable.WritableIntVector dictionaryIds)

Decode dictionary ids to data. From runLenDecoder and dictionaryIdsDecoder.