org.apache.hadoop.hive.ql.io.parquet.vector

Class VectorizedPrimitiveColumnReader

java.lang.Object

org.apache.hadoop.hive.ql.io.parquet.vector.BaseVectorizedColumnReader

org.apache.hadoop.hive.ql.io.parquet.vector.VectorizedPrimitiveColumnReader

All Implemented Interfaces:

VectorizedColumnReader

public class VectorizedPrimitiveColumnReader
extends BaseVectorizedColumnReader

It's column level Parquet reader which is used to read a batch of records for a column, part of the code is referred from Apache Spark and Apache Parquet. The read calls correspond to the various hivetypes. When the data was read, it would have been checked for validity with respect to the hivetype. The isValid call will return the result of that check. The readers will keep the value as returned by the reader when valid, and set the value to null when it is invalid.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.hive.ql.io.parquet.vector.BaseVectorizedColumnReader

BaseVectorizedColumnReader.NullIntIterator, BaseVectorizedColumnReader.RLEIntIterator, BaseVectorizedColumnReader.ValuesReaderIntIterator

Field Summary

Fields inherited from class org.apache.hadoop.hive.ql.io.parquet.vector.BaseVectorizedColumnReader

dataColumn, definitionLevel, definitionLevelColumn, descriptor, dictionary, endOfPageValueCount, hiveType, isCurrentPageDictionaryEncoded, legacyConversionEnabled, maxDefLevel, pageReader, pageValueCount, repetitionLevel, repetitionLevelColumn, skipProlepticConversion, skipTimestampConversion, type, valuesRead, writerTimezone

Constructor Summary

Constructors

Constructor and Description
VectorizedPrimitiveColumnReader(org.apache.parquet.column.ColumnDescriptor descriptor, org.apache.parquet.column.page.PageReader pageReader, boolean skipTimestampConversion, ZoneId writerTimezone, boolean skipProlepticConversion, boolean legacyConversionEnabled, org.apache.parquet.schema.Type type, TypeInfo hiveType)

Method Summary

Modifier and Type	Method and Description
void	readBatch(int total, ColumnVector column, TypeInfo columnType) read records with specified size and type into the columnVector

Methods inherited from class org.apache.hadoop.hive.ql.io.parquet.vector.BaseVectorizedColumnReader

decimalTypeCheck, readPage, readRepetitionAndDefinitionLevels

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

VectorizedPrimitiveColumnReader

public VectorizedPrimitiveColumnReader(org.apache.parquet.column.ColumnDescriptor descriptor,
                                       org.apache.parquet.column.page.PageReader pageReader,
                                       boolean skipTimestampConversion,
                                       ZoneId writerTimezone,
                                       boolean skipProlepticConversion,
                                       boolean legacyConversionEnabled,
                                       org.apache.parquet.schema.Type type,
                                       TypeInfo hiveType)
                                throws IOException

Throws:

IOException

Method Detail

readBatch

public void readBatch(int total,
                      ColumnVector column,
                      TypeInfo columnType)
               throws IOException

Description copied from interface: VectorizedColumnReader

read records with specified size and type into the columnVector

Parameters:

total - number of records to read into the column vector

column - column vector where the reader will read data into

columnType - the type of column vector

Throws:

IOException