org.apache.hudi.hadoop.realtime

Class HoodieRealtimeBootstrapBaseFileSplit

java.lang.Object

org.apache.hadoop.mapreduce.InputSplit

org.apache.hadoop.mapred.FileSplit

org.apache.hudi.hadoop.BootstrapBaseFileSplit

org.apache.hudi.hadoop.realtime.HoodieRealtimeBootstrapBaseFileSplit

All Implemented Interfaces:

org.apache.hadoop.io.Writable, org.apache.hadoop.mapred.InputSplit, org.apache.hadoop.mapred.InputSplitWithLocationInfo, RealtimeSplit

public class HoodieRealtimeBootstrapBaseFileSplit
extends BootstrapBaseFileSplit
implements RealtimeSplit

Realtime FileSplit with external base file NOTE: If you're adding fields here you need to make sure that you appropriately de-/serialize them in RealtimeSplit.readFromInput(DataInput) and RealtimeSplit.writeToOutput(DataOutput)

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hudi.hadoop.BootstrapBaseFileSplit

BootstrapBaseFileSplit.WrapperFileSplit

Constructor Summary

Constructors

Constructor and Description
HoodieRealtimeBootstrapBaseFileSplit() NOTE: This ctor is necessary for Hive to be able to serialize and then instantiate it when deserializing back
HoodieRealtimeBootstrapBaseFileSplit(org.apache.hadoop.mapred.FileSplit baseSplit, String basePath, List<HoodieLogFile> deltaLogFiles, String maxInstantTime, org.apache.hadoop.mapred.FileSplit externalFileSplit, boolean belongsToIncrementalQuery, Option<HoodieVirtualKeyInfo> virtualKeyInfoOpt)

Method Summary

Modifier and Type	Method and Description
String	getBasePath() Return Base Path of the dataset.
boolean	getBelongsToIncrementalQuery() Returns the flag whether this split belongs to an Incremental Query
List<HoodieLogFile>	getDeltaLogFiles()
String	getMaxCommitTime() Return Max Instant Time.
Option<HoodieVirtualKeyInfo>	getVirtualKeyInfo() Returns Virtual key info if meta fields are disabled.
void	readFields(DataInput in)
void	setBasePath(String basePath) Set Base Path.
void	setBelongsToIncrementalQuery(boolean belongsToIncrementalPath) Sets the flag whether this split belongs to an Incremental Query
void	setDeltaLogFiles(List<HoodieLogFile> deltaLogFiles)
void	setMaxCommitTime(String maxInstantTime) Update Maximum valid instant time.
void	setVirtualKeyInfo(Option<HoodieVirtualKeyInfo> virtualKeyInfo)
void	write(DataOutput out)

Methods inherited from class org.apache.hudi.hadoop.BootstrapBaseFileSplit

getBootstrapFileSplit

Methods inherited from class org.apache.hadoop.mapred.FileSplit

getLength, getLocationInfo, getLocations, getPath, getStart, toString

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.hudi.hadoop.realtime.RealtimeSplit

getDeltaLogPaths, getLength, getPath, getStart, readFromInput, writeToOutput

Methods inherited from interface org.apache.hadoop.mapred.InputSplitWithLocationInfo

getLocationInfo

Methods inherited from interface org.apache.hadoop.mapred.InputSplit

getLocations

Constructor Detail

HoodieRealtimeBootstrapBaseFileSplit

public HoodieRealtimeBootstrapBaseFileSplit()

NOTE: This ctor is necessary for Hive to be able to serialize and then instantiate it when deserializing back

HoodieRealtimeBootstrapBaseFileSplit

public HoodieRealtimeBootstrapBaseFileSplit(org.apache.hadoop.mapred.FileSplit baseSplit,
                                            String basePath,
                                            List<HoodieLogFile> deltaLogFiles,
                                            String maxInstantTime,
                                            org.apache.hadoop.mapred.FileSplit externalFileSplit,
                                            boolean belongsToIncrementalQuery,
                                            Option<HoodieVirtualKeyInfo> virtualKeyInfoOpt)
                                     throws IOException

Throws:

IOException

Method Detail

write

public void write(DataOutput out)
           throws IOException

Specified by:

write in interface org.apache.hadoop.io.Writable

Overrides:

write in class BootstrapBaseFileSplit

Throws:

IOException

readFields

public void readFields(DataInput in)
                throws IOException

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Overrides:

readFields in class BootstrapBaseFileSplit

Throws:

IOException

getDeltaLogFiles

public List<HoodieLogFile> getDeltaLogFiles()

Specified by:

getDeltaLogFiles in interface RealtimeSplit

setDeltaLogFiles

public void setDeltaLogFiles(List<HoodieLogFile> deltaLogFiles)

Specified by:

setDeltaLogFiles in interface RealtimeSplit

getMaxCommitTime

public String getMaxCommitTime()

Description copied from interface: RealtimeSplit

Return Max Instant Time.

Specified by:

getMaxCommitTime in interface RealtimeSplit

Returns:

getBasePath

public String getBasePath()

Description copied from interface: RealtimeSplit

Return Base Path of the dataset.

Specified by:

getBasePath in interface RealtimeSplit

Returns:

getVirtualKeyInfo

public Option<HoodieVirtualKeyInfo> getVirtualKeyInfo()

Description copied from interface: RealtimeSplit

Returns Virtual key info if meta fields are disabled.

Specified by:

getVirtualKeyInfo in interface RealtimeSplit

Returns:

getBelongsToIncrementalQuery

public boolean getBelongsToIncrementalQuery()

Description copied from interface: RealtimeSplit

Returns the flag whether this split belongs to an Incremental Query

Specified by:

getBelongsToIncrementalQuery in interface RealtimeSplit

setBelongsToIncrementalQuery

public void setBelongsToIncrementalQuery(boolean belongsToIncrementalPath)

Description copied from interface: RealtimeSplit

Sets the flag whether this split belongs to an Incremental Query

Specified by:

setBelongsToIncrementalQuery in interface RealtimeSplit

setMaxCommitTime

public void setMaxCommitTime(String maxInstantTime)

Description copied from interface: RealtimeSplit

Update Maximum valid instant time.

Specified by:

setMaxCommitTime in interface RealtimeSplit

setBasePath

public void setBasePath(String basePath)

Description copied from interface: RealtimeSplit

Set Base Path.

Specified by:

setBasePath in interface RealtimeSplit

setVirtualKeyInfo

public void setVirtualKeyInfo(Option<HoodieVirtualKeyInfo> virtualKeyInfo)

Specified by:

setVirtualKeyInfo in interface RealtimeSplit