org.apache.hudi.io

Class HoodieCreateHandle<T,I,K,O>

java.lang.Object

org.apache.hudi.io.HoodieIOHandle<T,I,K,O>

org.apache.hudi.io.HoodieWriteHandle<T,I,K,O>

org.apache.hudi.io.HoodieCreateHandle<T,I,K,O>

Direct Known Subclasses:

HoodieBootstrapHandle, HoodieUnboundedCreateHandle

@NotThreadSafe
public class HoodieCreateHandle<T,I,K,O>
extends HoodieWriteHandle<T,I,K,O>

Field Summary

Fields

Modifier and Type	Field and Description
protected HoodieFileWriter	fileWriter
protected long	insertRecordsWritten
protected org.apache.hadoop.fs.Path	path
protected long	recordsDeleted
protected long	recordsWritten

Fields inherited from class org.apache.hudi.io.HoodieWriteHandle

fileId, newRecordLocation, partitionPath, recordMerger, schemaOnReadEnabled, taskContextSupplier, timer, writeSchema, writeSchemaWithMetaFields, writeStatus, writeToken

Fields inherited from class org.apache.hudi.io.HoodieIOHandle

config, fs, hoodieTable, instantTime

Constructor Summary

Constructors

Constructor and Description
HoodieCreateHandle(HoodieWriteConfig config, String instantTime, HoodieTable<T,I,K,O> hoodieTable, String partitionPath, String fileId, Map<String,HoodieRecord<T>> recordMap, TaskContextSupplier taskContextSupplier) Called by the compactor code path.
HoodieCreateHandle(HoodieWriteConfig config, String instantTime, HoodieTable<T,I,K,O> hoodieTable, String partitionPath, String fileId, Option<org.apache.avro.Schema> overriddenSchema, TaskContextSupplier taskContextSupplier)
HoodieCreateHandle(HoodieWriteConfig config, String instantTime, HoodieTable<T,I,K,O> hoodieTable, String partitionPath, String fileId, Option<org.apache.avro.Schema> overriddenSchema, TaskContextSupplier taskContextSupplier, boolean preserveMetadata)
HoodieCreateHandle(HoodieWriteConfig config, String instantTime, HoodieTable<T,I,K,O> hoodieTable, String partitionPath, String fileId, TaskContextSupplier taskContextSupplier)
HoodieCreateHandle(HoodieWriteConfig config, String instantTime, HoodieTable<T,I,K,O> hoodieTable, String partitionPath, String fileId, TaskContextSupplier taskContextSupplier, boolean preserveMetadata)

Method Summary

Modifier and Type	Method and Description
boolean	canWrite(HoodieRecord record) Determines whether we can accept the incoming records, into the current file.
List<WriteStatus>	close() Performs actions to durably, persist the current changes and returns a WriteStatus object.
protected void	doWrite(HoodieRecord record, org.apache.avro.Schema schema, TypedProperties props) Perform the actual writing of the given record into the backing file.
IOType	getIOType()
protected void	setupWriteStatus() Set up the write status.
void	write() Writes all records passed.

Methods inherited from class org.apache.hudi.io.HoodieWriteHandle

createLogWriter, createLogWriter, createLogWriter, createMarkerFile, getAttemptId, getConfig, getFileId, getFileSystem, getHoodieTableMetaClient, getPartitionId, getPartitionPath, getStageId, getWriterSchema, getWriterSchemaWithMetaFields, isClosed, makeNewFilePath, makeNewPath, markClosed, toAvroRecord, write, writeStatuses

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

fileWriter

protected HoodieFileWriter fileWriter

path

protected final org.apache.hadoop.fs.Path path

recordsWritten

protected long recordsWritten

insertRecordsWritten

protected long insertRecordsWritten

recordsDeleted

protected long recordsDeleted

Constructor Detail

HoodieCreateHandle

public HoodieCreateHandle(HoodieWriteConfig config,
                          String instantTime,
                          HoodieTable<T,I,K,O> hoodieTable,
                          String partitionPath,
                          String fileId,
                          TaskContextSupplier taskContextSupplier)

HoodieCreateHandle

public HoodieCreateHandle(HoodieWriteConfig config,
                          String instantTime,
                          HoodieTable<T,I,K,O> hoodieTable,
                          String partitionPath,
                          String fileId,
                          TaskContextSupplier taskContextSupplier,
                          boolean preserveMetadata)

HoodieCreateHandle

public HoodieCreateHandle(HoodieWriteConfig config,
                          String instantTime,
                          HoodieTable<T,I,K,O> hoodieTable,
                          String partitionPath,
                          String fileId,
                          Option<org.apache.avro.Schema> overriddenSchema,
                          TaskContextSupplier taskContextSupplier)

HoodieCreateHandle

public HoodieCreateHandle(HoodieWriteConfig config,
                          String instantTime,
                          HoodieTable<T,I,K,O> hoodieTable,
                          String partitionPath,
                          String fileId,
                          Option<org.apache.avro.Schema> overriddenSchema,
                          TaskContextSupplier taskContextSupplier,
                          boolean preserveMetadata)

HoodieCreateHandle

public HoodieCreateHandle(HoodieWriteConfig config,
                          String instantTime,
                          HoodieTable<T,I,K,O> hoodieTable,
                          String partitionPath,
                          String fileId,
                          Map<String,HoodieRecord<T>> recordMap,
                          TaskContextSupplier taskContextSupplier)

Called by the compactor code path.

Method Detail

canWrite

public boolean canWrite(HoodieRecord record)

Description copied from class: HoodieWriteHandle

Determines whether we can accept the incoming records, into the current file. Depending on

- Whether it belongs to the same partitionPath as existing records - Whether the current file written bytes lt max file size

Overrides:

canWrite in class HoodieWriteHandle<T,I,K,O>

doWrite

protected void doWrite(HoodieRecord record,
                       org.apache.avro.Schema schema,
                       TypedProperties props)

Perform the actual writing of the given record into the backing file.

Overrides:

doWrite in class HoodieWriteHandle<T,I,K,O>

write

public void write()

Writes all records passed.

getIOType

public IOType getIOType()

Specified by:

getIOType in class HoodieWriteHandle<T,I,K,O>

close

public List<WriteStatus> close()

Performs actions to durably, persist the current changes and returns a WriteStatus object.

Specified by:

close in class HoodieWriteHandle<T,I,K,O>

setupWriteStatus

protected void setupWriteStatus()
                         throws IOException

Set up the write status.

Throws:

IOException - if error occurs