org.apache.hudi.hadoop

Class HoodieTableInputFormat

java.lang.Object

org.apache.hadoop.mapred.FileInputFormat<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.ArrayWritable>

org.apache.hudi.hadoop.HoodieTableInputFormat

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable, org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.ArrayWritable>

Direct Known Subclasses:

HoodieCopyOnWriteTableInputFormat

public abstract class HoodieTableInputFormat
extends org.apache.hadoop.mapred.FileInputFormat<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.ArrayWritable>
implements org.apache.hadoop.conf.Configurable

Abstract base class of the Hive's FileInputFormat implementations allowing for reading of Hudi's Copy-on-Write (COW) and Merge-on-Read (MOR) tables

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hadoop.mapred.FileInputFormat

org.apache.hadoop.mapred.FileInputFormat.Counter

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.conf.Configuration	conf

Fields inherited from class org.apache.hadoop.mapred.FileInputFormat

INPUT_DIR_RECURSIVE, LOG, NUM_INPUT_FILES

Constructor Summary

Constructors

Constructor and Description
HoodieTableInputFormat()

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.conf.Configuration	getConf()
protected boolean	isSplitable(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path filename)
protected org.apache.hadoop.fs.FileStatus[]	listStatus(org.apache.hadoop.mapred.JobConf job)
protected org.apache.hadoop.mapred.FileSplit	makeSplit(org.apache.hadoop.fs.Path file, long start, long length, String[] hosts)
protected org.apache.hadoop.mapred.FileSplit	makeSplit(org.apache.hadoop.fs.Path file, long start, long length, String[] hosts, String[] inMemoryHosts)
void	setConf(org.apache.hadoop.conf.Configuration conf)

Methods inherited from class org.apache.hadoop.mapred.FileInputFormat

addInputPath, addInputPathRecursively, addInputPaths, computeSplitSize, getBlockIndex, getInputPathFilter, getInputPaths, getRecordReader, getSplitHosts, getSplits, setInputPathFilter, setInputPaths, setInputPaths, setMinSplitSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

conf

protected org.apache.hadoop.conf.Configuration conf

Constructor Detail

HoodieTableInputFormat

public HoodieTableInputFormat()

Method Detail

getConf

public final org.apache.hadoop.conf.Configuration getConf()

Specified by:

getConf in interface org.apache.hadoop.conf.Configurable

setConf

public final void setConf(org.apache.hadoop.conf.Configuration conf)

Specified by:

setConf in interface org.apache.hadoop.conf.Configurable

isSplitable

protected boolean isSplitable(org.apache.hadoop.fs.FileSystem fs,
                              org.apache.hadoop.fs.Path filename)

Overrides:

isSplitable in class org.apache.hadoop.mapred.FileInputFormat<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.ArrayWritable>

makeSplit

protected org.apache.hadoop.mapred.FileSplit makeSplit(org.apache.hadoop.fs.Path file,
                                                       long start,
                                                       long length,
                                                       String[] hosts)

Overrides:

makeSplit in class org.apache.hadoop.mapred.FileInputFormat<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.ArrayWritable>

makeSplit

protected org.apache.hadoop.mapred.FileSplit makeSplit(org.apache.hadoop.fs.Path file,
                                                       long start,
                                                       long length,
                                                       String[] hosts,
                                                       String[] inMemoryHosts)

Overrides:

makeSplit in class org.apache.hadoop.mapred.FileInputFormat<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.ArrayWritable>

listStatus

protected org.apache.hadoop.fs.FileStatus[] listStatus(org.apache.hadoop.mapred.JobConf job)
                                                throws IOException

Overrides:

listStatus in class org.apache.hadoop.mapred.FileInputFormat<org.apache.hadoop.io.NullWritable,org.apache.hadoop.io.ArrayWritable>

Throws:

IOException