public interface WorkUnitStreamSource<S,D> extends Source<S,D>
Source that produces a WorkUnitStream.
Streamed sources may offer certain advantages over non-streamed sources if the job launcher supports them:
* Low memory usage: If the WorkUnitStream is backed by a generator, the job launcher may optimize memory
usage by never materializing all work units in memory.
* Eager processing of slow sources: If the source is slow at producing work units, the job launcher may start
processing available work units while future work units are still being computed.
* Infinite work unit streams: in the future, some job launchers will support infinite streams of work units.| Modifier and Type | Method and Description |
|---|---|
WorkUnitStream |
getWorkunitStream(SourceState state)
Get the
WorkUnitStream to process. |
getExtractor, getWorkunits, isEarlyStopped, shutdownWorkUnitStream getWorkunitStream(SourceState state)
WorkUnitStream to process.