Package io.trino.plugin.hive.parquet

Class ParquetPageSourceFactory

java.lang.Object

io.trino.plugin.hive.parquet.ParquetPageSourceFactory

All Implemented Interfaces:

HivePageSourceFactory

public class ParquetPageSourceFactory
extends Object
implements HivePageSourceFactory

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static interface	ParquetPageSourceFactory.ParquetReaderProvider

Field Summary

Fields

Modifier and Type	Field	Description
static final HiveColumnHandle	PARQUET_ROW_INDEX_COLUMN	If this object is passed as one of the columns for createPageSource, it will be populated as an additional column containing the index of each row read.

Constructor Summary

Constructors

Constructor	Description
ParquetPageSourceFactory(TrinoFileSystemFactory fileSystemFactory, FileFormatDataSourceStats stats, ParquetReaderConfig config, HiveConfig hiveConfig)

Method Summary

Modifier and Type	Method	Description
static ReaderPageSource	createPageSource(TrinoInputFile inputFile, long start, long length, List<HiveColumnHandle> columns, TupleDomain<HiveColumnHandle> effectivePredicate, boolean useColumnNames, org.joda.time.DateTimeZone timeZone, FileFormatDataSourceStats stats, ParquetReaderOptions options, Optional<ParquetWriteValidation> parquetWriteValidation, int domainCompactionThreshold)	This method is available for other callers to use directly.
Optional<ReaderPageSource>	createPageSource(ConnectorSession session, Location path, long start, long length, long estimatedFileSize, Properties schema, List<HiveColumnHandle> columns, TupleDomain<HiveColumnHandle> effectivePredicate, Optional<AcidInfo> acidInfo, OptionalInt bucketNumber, boolean originalFile, AcidTransaction transaction)
static ConnectorPageSource	createParquetPageSource(List<HiveColumnHandle> baseColumns, org.apache.parquet.schema.MessageType fileSchema, org.apache.parquet.io.MessageColumnIO messageColumn, boolean useColumnNames, ParquetPageSourceFactory.ParquetReaderProvider parquetReaderProvider)
static Optional<org.apache.parquet.internal.filter2.columnindex.ColumnIndexStore>	getColumnIndexStore(ParquetDataSource dataSource, org.apache.parquet.hadoop.metadata.BlockMetaData blockMetadata, Map<List<String>,org.apache.parquet.column.ColumnDescriptor> descriptorsByPath, TupleDomain<org.apache.parquet.column.ColumnDescriptor> parquetTupleDomain, ParquetReaderOptions options)
static Optional<org.apache.parquet.schema.Type>	getColumnType(HiveColumnHandle column, org.apache.parquet.schema.MessageType messageType, boolean useParquetColumnNames)
static Optional<org.apache.parquet.schema.MessageType>	getParquetMessageType(List<HiveColumnHandle> columns, boolean useColumnNames, org.apache.parquet.schema.MessageType fileSchema)
static TupleDomain<org.apache.parquet.column.ColumnDescriptor>	getParquetTupleDomain(Map<List<String>,org.apache.parquet.column.ColumnDescriptor> descriptorsByPath, TupleDomain<HiveColumnHandle> effectivePredicate, org.apache.parquet.schema.MessageType fileSchema, boolean useColumnNames)
static Properties	stripUnnecessaryProperties(Properties schema)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Details

PARQUET_ROW_INDEX_COLUMN

public static final HiveColumnHandle PARQUET_ROW_INDEX_COLUMN

If this object is passed as one of the columns for createPageSource, it will be populated as an additional column containing the index of each row read.

Constructor Details

ParquetPageSourceFactory

@Inject
public ParquetPageSourceFactory(TrinoFileSystemFactory fileSystemFactory,
 FileFormatDataSourceStats stats,
 ParquetReaderConfig config,
 HiveConfig hiveConfig)

Method Details

stripUnnecessaryProperties

public static Properties stripUnnecessaryProperties(Properties schema)

createPageSource

public Optional<ReaderPageSource> createPageSource(ConnectorSession session,
 Location path,
 long start,
 long length,
 long estimatedFileSize,
 Properties schema,
 List<HiveColumnHandle> columns,
 TupleDomain<HiveColumnHandle> effectivePredicate,
 Optional<AcidInfo> acidInfo,
 OptionalInt bucketNumber,
 boolean originalFile,
 AcidTransaction transaction)

Specified by:

createPageSource in interface HivePageSourceFactory

createPageSource

public static ReaderPageSource createPageSource(TrinoInputFile inputFile,
 long start,
 long length,
 List<HiveColumnHandle> columns,
 TupleDomain<HiveColumnHandle> effectivePredicate,
 boolean useColumnNames,
 org.joda.time.DateTimeZone timeZone,
 FileFormatDataSourceStats stats,
 ParquetReaderOptions options,
 Optional<ParquetWriteValidation> parquetWriteValidation,
 int domainCompactionThreshold)

This method is available for other callers to use directly.

getParquetMessageType

public static Optional<org.apache.parquet.schema.MessageType> getParquetMessageType(List<HiveColumnHandle> columns,
 boolean useColumnNames,
 org.apache.parquet.schema.MessageType fileSchema)

getColumnType

public static Optional<org.apache.parquet.schema.Type> getColumnType(HiveColumnHandle column,
 org.apache.parquet.schema.MessageType messageType,
 boolean useParquetColumnNames)

getColumnIndexStore

public static Optional<org.apache.parquet.internal.filter2.columnindex.ColumnIndexStore> getColumnIndexStore(ParquetDataSource dataSource,
 org.apache.parquet.hadoop.metadata.BlockMetaData blockMetadata,
 Map<List<String>,org.apache.parquet.column.ColumnDescriptor> descriptorsByPath,
 TupleDomain<org.apache.parquet.column.ColumnDescriptor> parquetTupleDomain,
 ParquetReaderOptions options)

getParquetTupleDomain

public static TupleDomain<org.apache.parquet.column.ColumnDescriptor> getParquetTupleDomain(Map<List<String>,org.apache.parquet.column.ColumnDescriptor> descriptorsByPath,
 TupleDomain<HiveColumnHandle> effectivePredicate,
 org.apache.parquet.schema.MessageType fileSchema,
 boolean useColumnNames)

createParquetPageSource

public static ConnectorPageSource createParquetPageSource(List<HiveColumnHandle> baseColumns,
 org.apache.parquet.schema.MessageType fileSchema,
 org.apache.parquet.io.MessageColumnIO messageColumn,
 boolean useColumnNames,
 ParquetPageSourceFactory.ParquetReaderProvider parquetReaderProvider)
                                                   throws IOException

Throws:

IOException