S - output schema typeD - output record typepublic interface Extractor<S,D> extends Closeable
All source specific logic for a data source should be encapsulated in an
implementation of this interface and Source.
| Modifier and Type | Method and Description |
|---|---|
long |
getExpectedRecordCount()
Get the expected source record count.
|
long |
getHighWatermark()
Deprecated.
there is no longer support for reporting the high watermark via this method, please see
Watermarks for more information.
|
S |
getSchema()
Get the schema (metadata) of the extracted data records.
|
default D |
readRecord(D reuse)
Read the next data record from the data source.
|
default RecordEnvelope<D> |
readRecordEnvelope()
Read an
RecordEnvelope. |
default StreamEntity<D> |
readStreamEntity()
Read an
StreamEntity. |
default RecordStreamWithMetadata<D,S> |
recordStream(AtomicBoolean shutdownRequest) |
default void |
shutdown()
Called to notify the Extractor it should shut down as soon as possible.
|
S getSchema() throws IOException
IOException - if there is problem getting the schema@Nullable default D readRecord(@Deprecated D reuse) throws DataRecordException, IOException
Reuse of data records has been deprecated and is not executed internally.
reuse - the data record object to be reusedDataRecordException - if there is problem with the extracted data recordIOException - if there is problem extracting the next data record from the sourcelong getExpectedRecordCount()
@Deprecated long getHighWatermark()
default void shutdown()
throws JobShutdownException
JobShutdownException - if the extractor does not support early termination. This will cause the task to fail.default RecordEnvelope<D> readRecordEnvelope() throws DataRecordException, IOException
DataRecordExceptionIOExceptiondefault StreamEntity<D> readStreamEntity() throws DataRecordException, IOException
StreamEntity. By default, just return result of readRecordEnvelope().DataRecordExceptionIOExceptiondefault RecordStreamWithMetadata<D,S> recordStream(AtomicBoolean shutdownRequest) throws IOException
shutdownRequest - an AtomicBoolean that becomes true when a shutdown has been requested.Flowable with the records from this source. Note the flowable should honor downstream backpressure.IOException