org.apache.hudi.hadoop

Class HoodieParquetInputFormat

java.lang.Object

org.apache.hadoop.mapred.FileInputFormat<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.ArrayWritable>

org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat

org.apache.hudi.hadoop.HoodieParquetInputFormatBase

org.apache.hudi.hadoop.HoodieParquetInputFormat

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.hive.ql.exec.vector.VectorizedInputFormatInterface, org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.ArrayWritable>

Direct Known Subclasses:

HoodieInputFormat, HoodieParquetRealtimeInputFormat

@UseRecordReaderFromInputFormat
 @UseFileSplitsFromInputFormat
public class HoodieParquetInputFormat
extends HoodieParquetInputFormatBase

HoodieInputFormat which understands the Hoodie File Structure and filters files based on the Hoodie Mode. If paths that does not correspond to a hoodie table then they are passed in as is (as what FileInputFormat.listStatus() would do). The JobConf could have paths from multiple Hoodie/Non-Hoodie tables

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapred.FileInputFormat

org.apache.hadoop.mapred.FileInputFormat.Counter

Field Summary

Fields inherited from class org.apache.hadoop.mapred.FileInputFormat

INPUT_DIR_RECURSIVE, NUM_INPUT_FILES

Constructor Summary

Constructors

Modifier	Constructor and Description
	HoodieParquetInputFormat()
protected	HoodieParquetInputFormat(HoodieCopyOnWriteTableInputFormat delegate)

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.ArrayWritable>	getRecordReader(org.apache.hadoop.mapred.InputSplit split, org.apache.hadoop.mapred.JobConf job, org.apache.hadoop.mapred.Reporter reporter)

Methods inherited from class org.apache.hudi.hadoop.HoodieParquetInputFormatBase

getConf, getSplits, isSplitable, listStatus, makeSplit, makeSplit, setConf

Methods inherited from class org.apache.hadoop.mapred.FileInputFormat

addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getInputPathFilter, getInputPaths, getSplitHosts, setInputPathFilter, setInputPaths, setInputPaths, setMinSplitSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

HoodieParquetInputFormat

public HoodieParquetInputFormat()

HoodieParquetInputFormat

protected HoodieParquetInputFormat(HoodieCopyOnWriteTableInputFormat delegate)

Method Detail

getRecordReader

public org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.ArrayWritable> getRecordReader(org.apache.hadoop.mapred.InputSplit split,
                                                                                                                                   org.apache.hadoop.mapred.JobConf job,
                                                                                                                                   org.apache.hadoop.mapred.Reporter reporter)
                                                                                                                            throws IOException

Specified by:

getRecordReader in interface org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.ArrayWritable>

Overrides:

getRecordReader in class org.apache.hadoop.hive.ql.io.parquet.MapredParquetInputFormat

Throws:

IOException