org.apache.hudi.common.model

Class DefaultHoodieRecordPayload

java.lang.Object

org.apache.hudi.common.model.BaseAvroPayload

org.apache.hudi.common.model.OverwriteWithLatestAvroPayload

org.apache.hudi.common.model.DefaultHoodieRecordPayload

All Implemented Interfaces:

Serializable, HoodieRecordPayload<OverwriteWithLatestAvroPayload>

Direct Known Subclasses:

EventTimeAvroPayload

public class DefaultHoodieRecordPayload
extends OverwriteWithLatestAvroPayload

HoodieRecordPayload impl that honors ordering field in both preCombine and combineAndGetUpdateValue.

1. preCombine - Picks the latest delta record for a key, based on an ordering field 2. combineAndGetUpdateValue/getInsertValue - Chooses the latest record based on ordering field value.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static String	METADATA_EVENT_TIME_KEY

Fields inherited from class org.apache.hudi.common.model.BaseAvroPayload

orderingVal, recordBytes

Constructor Summary

Constructors

Constructor and Description
DefaultHoodieRecordPayload(org.apache.avro.generic.GenericRecord record, Comparable orderingVal)
DefaultHoodieRecordPayload(Option<org.apache.avro.generic.GenericRecord> record)

Method Summary

Modifier and Type	Method and Description
Option<org.apache.avro.generic.IndexedRecord>	combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema, Properties properties) This methods lets you write custom merging/combining logic to produce new values as a function of current value on storage and whats contained in this object.
Option<org.apache.avro.generic.IndexedRecord>	getInsertValue(org.apache.avro.Schema schema, Properties properties) Generates an avro record out of the given HoodieRecordPayload, to be written out to storage.
Option<Map<String,String>>	getMetadata() This method can be used to extract some metadata from HoodieRecordPayload.
protected boolean	needUpdatingPersistedRecord(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.generic.IndexedRecord incomingRecord, Properties properties)

Methods inherited from class org.apache.hudi.common.model.OverwriteWithLatestAvroPayload

combineAndGetUpdateValue, getInsertValue, getOrderingValue, isDeleteRecord, overwriteField, preCombine

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface org.apache.hudi.common.model.HoodieRecordPayload

preCombine

Field Detail

METADATA_EVENT_TIME_KEY

public static final String METADATA_EVENT_TIME_KEY

See Also:

Constant Field Values

Constructor Detail

DefaultHoodieRecordPayload

public DefaultHoodieRecordPayload(org.apache.avro.generic.GenericRecord record,
                                  Comparable orderingVal)

DefaultHoodieRecordPayload

public DefaultHoodieRecordPayload(Option<org.apache.avro.generic.GenericRecord> record)

Method Detail

combineAndGetUpdateValue

public Option<org.apache.avro.generic.IndexedRecord> combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue,
                                                                              org.apache.avro.Schema schema,
                                                                              Properties properties)
                                                                       throws IOException

Description copied from interface: HoodieRecordPayload

This methods lets you write custom merging/combining logic to produce new values as a function of current value on storage and whats contained in this object. Implementations can leverage properties if required.

eg: 1) You are updating counters, you may want to add counts to currentValue and write back updated counts 2) You may be reading DB redo logs, and merge them with current image for a database row on storage

Parameters:

currentValue - Current value in storage, to merge/combine this payload with

schema - Schema used for record

properties - Payload related properties. For example pass the ordering field(s) name to extract from value in storage.

Returns:

new combined/merged value to be written back to storage. EMPTY to skip writing this record.

Throws:

IOException

getInsertValue

public Option<org.apache.avro.generic.IndexedRecord> getInsertValue(org.apache.avro.Schema schema,
                                                                    Properties properties)
                                                             throws IOException

Description copied from interface: HoodieRecordPayload

Generates an avro record out of the given HoodieRecordPayload, to be written out to storage. Called when writing a new value for the given HoodieKey, wherein there is no existing record in storage to be combined against. (i.e insert) Return EMPTY to skip writing this record. Implementations can leverage properties if required.

Parameters:

schema - Schema used for record

properties - Payload related properties. For example pass the ordering field(s) name to extract from value in storage.

Returns:

the IndexedRecord to be inserted.

Throws:

IOException

getMetadata

public Option<Map<String,String>> getMetadata()

Description copied from interface: HoodieRecordPayload

This method can be used to extract some metadata from HoodieRecordPayload. The metadata is passed to WriteStatus.markSuccess() and WriteStatus.markFailure() in order to compute some aggregate metrics using the metadata in the context of a write success or failure.

Returns:

the metadata in the form of Map if any.

needUpdatingPersistedRecord

protected boolean needUpdatingPersistedRecord(org.apache.avro.generic.IndexedRecord currentValue,
                                              org.apache.avro.generic.IndexedRecord incomingRecord,
                                              Properties properties)