org.apache.paimon.format.parquet.reader

类 BaseVectorizedColumnReader

java.lang.Object

org.apache.paimon.format.parquet.reader.BaseVectorizedColumnReader

所有已实现的接口:

ColumnReader<org.apache.paimon.data.columnar.writable.WritableColumnVector>

直接已知子类:

ArrayColumnReader

public abstract class BaseVectorizedColumnReader
extends Object
implements ColumnReader<org.apache.paimon.data.columnar.writable.WritableColumnVector>

It's column level Parquet reader which is used to read a batch of records for a column, part of the code is referred from Apache Hive and Apache Parquet.

嵌套类概要

嵌套类

限定符和类型	类和说明
protected static class	BaseVectorizedColumnReader.NullIntIterator Reading zero always.
protected static class	BaseVectorizedColumnReader.RLEIntIterator Reading int from RunLengthBitPackingHybridDecoder.
protected static class	BaseVectorizedColumnReader.ValuesReaderIntIterator Reading int from ValuesReader.

字段概要

字段

限定符和类型	字段和说明
protected ParquetDataColumnReader	dataColumn
protected org.apache.paimon.types.DataType	dataType
protected int	definitionLevel
protected org.apache.paimon.format.parquet.reader.BaseVectorizedColumnReader.IntIterator	definitionLevelColumn
protected org.apache.parquet.column.ColumnDescriptor	descriptor
protected ParquetDataColumnReader	dictionary The dictionary, if this column has dictionary encoding.
protected long	endOfPageValueCount value that indicates the end of the current page.
protected boolean	isCurrentPageDictionaryEncoded If true, the current page is dictionary encoded.
protected boolean	isUtcTimestamp
protected int	maxDefLevel Maximum definition level for this column.
protected org.apache.parquet.column.page.PageReader	pageReader
protected int	pageValueCount Total values in the current page.
protected int	repetitionLevel
protected org.apache.paimon.format.parquet.reader.BaseVectorizedColumnReader.IntIterator	repetitionLevelColumn Repetition/Definition/Value readers.
protected org.apache.parquet.schema.Type	type
protected long	valuesRead Total number of values read.

构造器概要

构造器

构造器和说明
BaseVectorizedColumnReader(org.apache.parquet.column.ColumnDescriptor descriptor, org.apache.parquet.column.page.PageReader pageReader, boolean isUtcTimestamp, org.apache.parquet.schema.Type parquetType, org.apache.paimon.types.DataType dataType)

方法概要

限定符和类型	方法和说明
protected void	readPage()
protected void	readRepetitionAndDefinitionLevels()

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

从接口继承的方法 org.apache.paimon.format.parquet.reader.ColumnReader

readToVector

字段详细资料

isUtcTimestamp

protected boolean isUtcTimestamp

valuesRead

protected long valuesRead

Total number of values read.

endOfPageValueCount

protected long endOfPageValueCount

value that indicates the end of the current page. That is, if valuesRead == endOfPageValueCount, we are at the end of the page.

dictionary

protected final ParquetDataColumnReader dictionary

The dictionary, if this column has dictionary encoding.

isCurrentPageDictionaryEncoded

protected boolean isCurrentPageDictionaryEncoded

If true, the current page is dictionary encoded.

maxDefLevel

protected final int maxDefLevel

Maximum definition level for this column.

definitionLevel

protected int definitionLevel

repetitionLevel

protected int repetitionLevel

repetitionLevelColumn

protected org.apache.paimon.format.parquet.reader.BaseVectorizedColumnReader.IntIterator repetitionLevelColumn

Repetition/Definition/Value readers.

definitionLevelColumn

protected org.apache.paimon.format.parquet.reader.BaseVectorizedColumnReader.IntIterator definitionLevelColumn

dataColumn

protected ParquetDataColumnReader dataColumn

pageValueCount

protected int pageValueCount

Total values in the current page.

pageReader

protected final org.apache.parquet.column.page.PageReader pageReader

descriptor

protected final org.apache.parquet.column.ColumnDescriptor descriptor

type

protected final org.apache.parquet.schema.Type type

dataType

protected final org.apache.paimon.types.DataType dataType

构造器详细资料

BaseVectorizedColumnReader

public BaseVectorizedColumnReader(org.apache.parquet.column.ColumnDescriptor descriptor,
                                  org.apache.parquet.column.page.PageReader pageReader,
                                  boolean isUtcTimestamp,
                                  org.apache.parquet.schema.Type parquetType,
                                  org.apache.paimon.types.DataType dataType)
                           throws IOException

抛出:

IOException

方法详细资料

readRepetitionAndDefinitionLevels

protected void readRepetitionAndDefinitionLevels()

readPage

protected void readPage()