org.apache.hudi.bootstrap

Class SparkFullBootstrapDataProviderBase

java.lang.Object

org.apache.hudi.client.bootstrap.FullRecordBootstrapDataProvider<org.apache.spark.api.java.JavaRDD<HoodieRecord>>

org.apache.hudi.bootstrap.SparkFullBootstrapDataProviderBase

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

SparkOrcBootstrapDataProvider, SparkParquetBootstrapDataProvider

public abstract class SparkFullBootstrapDataProviderBase
extends FullRecordBootstrapDataProvider<org.apache.spark.api.java.JavaRDD<HoodieRecord>>

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.hudi.client.bootstrap.FullRecordBootstrapDataProvider

context, LOG, props

Constructor Summary

Constructors

Constructor and Description
SparkFullBootstrapDataProviderBase(TypedProperties props, HoodieSparkEngineContext context)

Method Summary

Modifier and Type	Method and Description
org.apache.spark.api.java.JavaRDD<HoodieRecord>	generateInputRecords(String tableName, String sourceBasePath, List<Pair<String,List<HoodieFileStatus>>> partitionPathsWithFiles, HoodieWriteConfig config)
protected abstract String	getFormat()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SparkFullBootstrapDataProviderBase

public SparkFullBootstrapDataProviderBase(TypedProperties props,
                                          HoodieSparkEngineContext context)

Method Detail

generateInputRecords

public org.apache.spark.api.java.JavaRDD<HoodieRecord> generateInputRecords(String tableName,
                                                                            String sourceBasePath,
                                                                            List<Pair<String,List<HoodieFileStatus>>> partitionPathsWithFiles,
                                                                            HoodieWriteConfig config)

Specified by:

generateInputRecords in class FullRecordBootstrapDataProvider<org.apache.spark.api.java.JavaRDD<HoodieRecord>>

getFormat

protected abstract String getFormat()