D - data record typepublic interface DataWriter<D> extends Closeable, Flushable
Generally, one work unit has a dedicated DataWriter instance, which processes only one dataset
| Modifier and Type | Method and Description |
|---|---|
long |
bytesWritten()
Get the number of bytes written.
|
void |
cleanup()
Cleanup context/resources.
|
void |
commit()
Commit the data written.
|
default void |
flush()
Flush data written by the writer.
|
default Descriptor |
getDataDescriptor()
The method should return a
Descriptor that represents what the writer is writing |
default ControlMessageHandler |
getMessageHandler()
Default handler calls flush on this object when a
FlushControlMessage is received |
long |
recordsWritten()
Get the number of records written.
|
default void |
write(D record)
Write a data record.
|
default void |
writeEnvelope(RecordEnvelope<D> recordEnvelope)
Write the input
RecordEnvelope. |
default void write(D record) throws IOException
record - data record to writeIOException - if there is anything wrong writing the recordvoid commit()
throws IOException
IOException - if there is anything wrong committing the outputvoid cleanup()
throws IOException
IOException - if there is anything wrong doing cleanup.long recordsWritten()
long bytesWritten()
throws IOException
IOExceptiondefault Descriptor getDataDescriptor()
Descriptor that represents what the writer is writing
Note that, this information might be useless and discarded by a
DataPublisher, which determines the final form of dataset or partition
DatasetDescriptor if it writes files of a dataset or
a PartitionDescriptor if it writes files of a dataset partition or
null if it is uselessdefault void writeEnvelope(RecordEnvelope<D> recordEnvelope) throws IOException
RecordEnvelope. By default, just call write(Object).
DataWriters that implement this method must acknowledge the recordEnvelope once the write has been acknowledged
by the destination system.IOExceptiondefault ControlMessageHandler getMessageHandler()
FlushControlMessage is receivedControlMessageHandler.default void flush()
throws IOException
flush in interface FlushableIOException