| Package | Description |
|---|---|
| org.apache.hudi.utilities.deltastreamer | |
| org.apache.hudi.utilities.sources |
| Modifier and Type | Method and Description |
|---|---|
InputBatch<org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord>> |
SourceFormatAdapter.fetchNewDataInAvroFormat(Option<String> lastCkptStr,
long sourceLimit)
Fetch new data in avro format.
|
InputBatch<org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>> |
SourceFormatAdapter.fetchNewDataInRowFormat(Option<String> lastCkptStr,
long sourceLimit)
Fetch new data in row format.
|
| Modifier and Type | Method and Description |
|---|---|
protected InputBatch<org.apache.spark.api.java.JavaRDD<String>> |
JsonDFSSource.fetchNewData(Option<String> lastCkptStr,
long sourceLimit) |
protected InputBatch<org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>> |
RowSource.fetchNewData(Option<String> lastCkptStr,
long sourceLimit) |
protected InputBatch<org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord>> |
AvroKafkaSource.fetchNewData(Option<String> lastCheckpointStr,
long sourceLimit) |
protected InputBatch<org.apache.spark.api.java.JavaRDD<String>> |
JsonKafkaSource.fetchNewData(Option<String> lastCheckpointStr,
long sourceLimit) |
protected InputBatch<org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord>> |
AvroDFSSource.fetchNewData(Option<String> lastCkptStr,
long sourceLimit) |
protected InputBatch<org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord>> |
HiveIncrPullSource.fetchNewData(Option<String> lastCheckpointStr,
long sourceLimit) |
protected abstract InputBatch<T> |
Source.fetchNewData(Option<String> lastCkptStr,
long sourceLimit) |
InputBatch<T> |
Source.fetchNext(Option<String> lastCkptStr,
long sourceLimit)
Main API called by Hoodie Delta Streamer to fetch records
|
Copyright © 2019 The Apache Software Foundation. All rights reserved.