org.apache.hudi.io

Class HoodieMergeHandleWithChangeLog<T,I,K,O>

java.lang.Object

org.apache.hudi.io.HoodieIOHandle<T,I,K,O>

org.apache.hudi.io.HoodieWriteHandle<T,I,K,O>

org.apache.hudi.io.HoodieMergeHandle<T,I,K,O>

org.apache.hudi.io.HoodieMergeHandleWithChangeLog<T,I,K,O>

Direct Known Subclasses:

HoodieSortedMergeHandleWithChangeLog

public class HoodieMergeHandleWithChangeLog<T,I,K,O>
extends HoodieMergeHandle<T,I,K,O>

A merge handle that supports logging change logs.

Field Summary

Fields

Modifier and Type	Field and Description
protected HoodieCDCLogger	cdcLogger

Fields inherited from class org.apache.hudi.io.HoodieMergeHandle

fileWriter, insertRecordsWritten, keyGeneratorOpt, keyToNewRecords, newFilePath, oldFilePath, partitionFields, partitionValues, recordsDeleted, recordsWritten, updatedRecordsWritten, useWriterSchemaForCompaction, writtenRecordKeys

Fields inherited from class org.apache.hudi.io.HoodieWriteHandle

fileId, newRecordLocation, partitionPath, recordMerger, schemaOnReadEnabled, taskContextSupplier, timer, writeSchema, writeSchemaWithMetaFields, writeStatus, writeToken

Fields inherited from class org.apache.hudi.io.HoodieIOHandle

config, fs, hoodieTable, instantTime

Constructor Summary

Constructors

Constructor and Description
HoodieMergeHandleWithChangeLog(HoodieWriteConfig config, String instantTime, HoodieTable<T,I,K,O> hoodieTable, Iterator<HoodieRecord<T>> recordItr, String partitionPath, String fileId, TaskContextSupplier taskContextSupplier, Option<BaseKeyGenerator> keyGeneratorOpt)
HoodieMergeHandleWithChangeLog(HoodieWriteConfig config, String instantTime, HoodieTable<T,I,K,O> hoodieTable, Map<String,HoodieRecord<T>> keyToNewRecords, String partitionPath, String fileId, HoodieBaseFile dataFileToBeMerged, TaskContextSupplier taskContextSupplier, Option<BaseKeyGenerator> keyGeneratorOpt) Called by compactor code path.

Method Summary

Modifier and Type	Method and Description
List<WriteStatus>	close()
protected void	writeInsertRecord(HoodieRecord<T> newRecord)
protected boolean	writeUpdateRecord(HoodieRecord<T> newRecord, HoodieRecord<T> oldRecord, Option<HoodieRecord> combinedRecordOpt, org.apache.avro.Schema writerSchema)

Methods inherited from class org.apache.hudi.io.HoodieMergeHandle

baseFileForMerge, getIOType, getLatestBaseFile, getOldFilePath, getPartitionFields, getPartitionValues, init, initializeIncomingRecordsMap, makeOldAndNewFilePaths, performMergeDataValidationCheck, setPartitionFields, setPartitionValues, setWriteStatusPath, write, writeIncomingRecords, writeInsertRecord, writeRecord, writeToFile

Methods inherited from class org.apache.hudi.io.HoodieWriteHandle

canWrite, createLogWriter, createLogWriter, createLogWriter, createMarkerFile, doWrite, getAttemptId, getConfig, getFileId, getFileSystem, getHoodieTableMetaClient, getPartitionId, getPartitionPath, getStageId, getWriterSchema, getWriterSchemaWithMetaFields, isClosed, makeNewFilePath, makeNewPath, markClosed, toAvroRecord, write, writeStatuses

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

cdcLogger

protected final HoodieCDCLogger cdcLogger

Constructor Detail

HoodieMergeHandleWithChangeLog

public HoodieMergeHandleWithChangeLog(HoodieWriteConfig config,
                                      String instantTime,
                                      HoodieTable<T,I,K,O> hoodieTable,
                                      Iterator<HoodieRecord<T>> recordItr,
                                      String partitionPath,
                                      String fileId,
                                      TaskContextSupplier taskContextSupplier,
                                      Option<BaseKeyGenerator> keyGeneratorOpt)

HoodieMergeHandleWithChangeLog

public HoodieMergeHandleWithChangeLog(HoodieWriteConfig config,
                                      String instantTime,
                                      HoodieTable<T,I,K,O> hoodieTable,
                                      Map<String,HoodieRecord<T>> keyToNewRecords,
                                      String partitionPath,
                                      String fileId,
                                      HoodieBaseFile dataFileToBeMerged,
                                      TaskContextSupplier taskContextSupplier,
                                      Option<BaseKeyGenerator> keyGeneratorOpt)

Called by compactor code path.

Method Detail

writeUpdateRecord

protected boolean writeUpdateRecord(HoodieRecord<T> newRecord,
                                    HoodieRecord<T> oldRecord,
                                    Option<HoodieRecord> combinedRecordOpt,
                                    org.apache.avro.Schema writerSchema)
                             throws IOException

Overrides:

writeUpdateRecord in class HoodieMergeHandle<T,I,K,O>

Throws:

IOException

writeInsertRecord

protected void writeInsertRecord(HoodieRecord<T> newRecord)
                          throws IOException

Overrides:

writeInsertRecord in class HoodieMergeHandle<T,I,K,O>

Throws:

IOException

close

public List<WriteStatus> close()

Overrides:

close in class HoodieMergeHandle<T,I,K,O>