org.apache.hudi.utilities.sources

Class JsonKafkaSource

java.lang.Object

org.apache.hudi.utilities.sources.Source<org.apache.spark.api.java.JavaRDD<String>>

org.apache.hudi.utilities.sources.JsonSource

org.apache.hudi.utilities.sources.JsonKafkaSource

All Implemented Interfaces:

Serializable

public class JsonKafkaSource
extends JsonSource

Read json kafka data

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.hudi.utilities.sources.Source

Source.SourceType

Field Summary

Fields inherited from class org.apache.hudi.utilities.sources.Source

props, sparkContext, sparkSession

Constructor Summary

Constructors

Constructor and Description
JsonKafkaSource(TypedProperties properties, org.apache.spark.api.java.JavaSparkContext sparkContext, org.apache.spark.sql.SparkSession sparkSession, SchemaProvider schemaProvider)

Method Summary

Modifier and Type	Method and Description
protected InputBatch<org.apache.spark.api.java.JavaRDD<String>>	fetchNewData(Option<String> lastCheckpointStr, long sourceLimit)

Methods inherited from class org.apache.hudi.utilities.sources.Source

fetchNext, getSourceType, getSparkSession

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

JsonKafkaSource

public JsonKafkaSource(TypedProperties properties,
                       org.apache.spark.api.java.JavaSparkContext sparkContext,
                       org.apache.spark.sql.SparkSession sparkSession,
                       SchemaProvider schemaProvider)

Method Detail

fetchNewData

protected InputBatch<org.apache.spark.api.java.JavaRDD<String>> fetchNewData(Option<String> lastCheckpointStr,
                                                                             long sourceLimit)

Specified by:

fetchNewData in class Source<org.apache.spark.api.java.JavaRDD<String>>