org.apache.hudi.integ.testsuite

类 HoodieDeltaStreamerWrapper

java.lang.Object

org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer

org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper

所有已实现的接口:

Serializable

public class HoodieDeltaStreamerWrapper
extends HoodieDeltaStreamer

Extends the HoodieDeltaStreamer to expose certain operations helpful in running the Test Suite. This is done to achieve 2 things 1) Leverage some components of HoodieDeltaStreamer 2) Piggyback on the suite to test HoodieDeltaStreamer

另请参阅:

序列化表格

嵌套类概要

从类继承的嵌套类/接口 org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer

HoodieDeltaStreamer.Config, HoodieDeltaStreamer.DeltaSyncService

字段概要

从类继承的字段 org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer

cfg, CHECKPOINT_KEY, CHECKPOINT_RESET_KEY, DELTASYNC_POOL_NAME, deltaSyncService

构造器概要

构造器

构造器和说明
HoodieDeltaStreamerWrapper(HoodieDeltaStreamer.Config cfg, org.apache.spark.api.java.JavaSparkContext jssc)

方法概要

限定符和类型	方法和说明
org.apache.spark.api.java.JavaRDD<org.apache.hudi.client.WriteStatus>	bulkInsert()
org.apache.spark.api.java.JavaRDD<org.apache.hudi.client.WriteStatus>	compact()
org.apache.hudi.common.util.collection.Pair<SchemaProvider,org.apache.hudi.common.util.collection.Pair<String,org.apache.spark.api.java.JavaRDD<org.apache.hudi.common.model.HoodieRecord>>>	fetchSource()
org.apache.spark.api.java.JavaRDD<org.apache.hudi.client.WriteStatus>	insert()
org.apache.spark.api.java.JavaRDD<org.apache.hudi.client.WriteStatus>	insertOverwrite()
org.apache.spark.api.java.JavaRDD<org.apache.hudi.client.WriteStatus>	insertOverwriteTable()
void	scheduleCompact()
org.apache.spark.api.java.JavaRDD<org.apache.hudi.client.WriteStatus>	upsert(org.apache.hudi.common.model.WriteOperationType operation)

从类继承的方法 org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer

getConfig, getConfig, getDeltaSyncService, main, shutdownGracefully, sync

从类继承的方法 java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

构造器详细资料

HoodieDeltaStreamerWrapper

public HoodieDeltaStreamerWrapper(HoodieDeltaStreamer.Config cfg,
                                  org.apache.spark.api.java.JavaSparkContext jssc)
                           throws Exception

抛出:

Exception

方法详细资料

upsert

public org.apache.spark.api.java.JavaRDD<org.apache.hudi.client.WriteStatus> upsert(org.apache.hudi.common.model.WriteOperationType operation)
                                                                             throws Exception

抛出:

Exception

insert

public org.apache.spark.api.java.JavaRDD<org.apache.hudi.client.WriteStatus> insert()
                                                                             throws Exception

抛出:

Exception

bulkInsert

public org.apache.spark.api.java.JavaRDD<org.apache.hudi.client.WriteStatus> bulkInsert()
                                                                                 throws Exception

抛出:

Exception

insertOverwrite

public org.apache.spark.api.java.JavaRDD<org.apache.hudi.client.WriteStatus> insertOverwrite()
                                                                                      throws Exception

抛出:

Exception

insertOverwriteTable

public org.apache.spark.api.java.JavaRDD<org.apache.hudi.client.WriteStatus> insertOverwriteTable()
                                                                                           throws Exception

抛出:

Exception

scheduleCompact

public void scheduleCompact()
                     throws Exception

抛出:

Exception

compact

public org.apache.spark.api.java.JavaRDD<org.apache.hudi.client.WriteStatus> compact()
                                                                              throws Exception

抛出:

Exception

fetchSource

public org.apache.hudi.common.util.collection.Pair<SchemaProvider,org.apache.hudi.common.util.collection.Pair<String,org.apache.spark.api.java.JavaRDD<org.apache.hudi.common.model.HoodieRecord>>> fetchSource()
                                                                                                                                                                                                         throws Exception

抛出:

Exception