Uses of Interface
org.apache.hudi.common.data.HoodieData

Packages that use HoodieData

Package	Description
org.apache.hudi.common.data
org.apache.hudi.common.engine
org.apache.hudi.metadata

Uses of HoodieData in org.apache.hudi.common.data

Classes in org.apache.hudi.common.data that implement HoodieData

Modifier and Type	Class and Description
class	HoodieListData<T> In-memory implementation of HoodieData holding internally a Stream of objects.

Methods in org.apache.hudi.common.data that return HoodieData

Modifier and Type	Method and Description
HoodieData<T>	HoodieData.distinct() Returns new HoodieData collection holding only distinct objects of the original one This is a stateful intermediate operation
HoodieData<T>	HoodieListData.distinct()
HoodieData<T>	HoodieData.distinct(int parallelism) Returns new HoodieData collection holding only distinct objects of the original one This is a stateful intermediate operation
HoodieData<T>	HoodieListData.distinct(int parallelism)
default <O> HoodieData<T>	HoodieData.distinctWithKey(SerializableFunction<T,O> keyGetter, int parallelism)
<O> HoodieData<T>	HoodieListData.distinctWithKey(SerializableFunction<T,O> keyGetter, int parallelism)
HoodieData<T>	HoodieData.filter(SerializableFunction<T,Boolean> filterFunc) Returns new instance of HoodieData collection only containing elements matching provided filterFunc (ie ones it returns true on)
HoodieData<T>	HoodieListData.filter(SerializableFunction<T,Boolean> filterFunc)
<O> HoodieData<O>	HoodieData.flatMap(SerializableFunction<T,Iterator<O>> func) Maps every element in the collection into a collection of the new elements (provided by Iterator) using provided mapping func, subsequently flattening the result (by concatenating) into a single collection This is an intermediate operation
<O> HoodieData<O>	HoodieListData.flatMap(SerializableFunction<T,Iterator<O>> func)
HoodieData<K>	HoodieListPairData.keys()
HoodieData<K>	HoodiePairData.keys() Returns a HoodieData holding the key from every corresponding pair
<O> HoodieData<O>	HoodieListPairData.map(SerializableFunction<Pair<K,V>,O> func)
<O> HoodieData<O>	HoodiePairData.map(SerializableFunction<Pair<K,V>,O> func)
<O> HoodieData<O>	HoodieData.map(SerializableFunction<T,O> func) Maps every element in the collection using provided mapping func.
<O> HoodieData<O>	HoodieListData.map(SerializableFunction<T,O> func)
<O> HoodieData<O>	HoodieData.mapPartitions(SerializableFunction<Iterator<T>,Iterator<O>> func, boolean preservesPartitioning) Maps every element in the collection's partition (if applicable) by applying provided mapping func to every collection's partition This is an intermediate operation
<O> HoodieData<O>	HoodieListData.mapPartitions(SerializableFunction<Iterator<T>,Iterator<O>> func, boolean preservesPartitioning)
HoodieData<T>	HoodieData.repartition(int parallelism) Re-partitions underlying collection (if applicable) making sure new HoodieData has exactly parallelism partitions
HoodieData<T>	HoodieListData.repartition(int parallelism)
HoodieData<T>	HoodieData.union(HoodieData<T> other) Unions HoodieData with another instance of HoodieData.
HoodieData<T>	HoodieListData.union(HoodieData<T> other)
HoodieData<V>	HoodieListPairData.values()
HoodieData<V>	HoodiePairData.values() Returns a HoodieData holding the value from every corresponding pair

Methods in org.apache.hudi.common.data with parameters of type HoodieData

Modifier and Type	Method and Description
HoodieData<T>	HoodieData.union(HoodieData<T> other) Unions HoodieData with another instance of HoodieData.
HoodieData<T>	HoodieListData.union(HoodieData<T> other)

Uses of HoodieData in org.apache.hudi.common.engine

Methods in org.apache.hudi.common.engine that return HoodieData

Modifier and Type	Method and Description
abstract <T> HoodieData<T>	HoodieEngineContext.emptyHoodieData()
<T> HoodieData<T>	HoodieLocalEngineContext.emptyHoodieData()
<T> HoodieData<T>	HoodieEngineContext.parallelize(List<T> data)
abstract <T> HoodieData<T>	HoodieEngineContext.parallelize(List<T> data, int parallelism)
<T> HoodieData<T>	HoodieLocalEngineContext.parallelize(List<T> data, int parallelism)

Uses of HoodieData in org.apache.hudi.metadata

Methods in org.apache.hudi.metadata that return HoodieData

Modifier and Type	Method and Description
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.convertFilesToBloomFilterRecords(HoodieEngineContext engineContext, Map<String,List<String>> partitionToDeletedFiles, Map<String,Map<String,Long>> partitionToAppendedFiles, MetadataRecordsGenerationParams recordsGenerationParams, String instantTime) Convert added and deleted files metadata to bloom filter index records.
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.convertFilesToColumnStatsRecords(HoodieEngineContext engineContext, Map<String,List<String>> partitionToDeletedFiles, Map<String,Map<String,Long>> partitionToAppendedFiles, MetadataRecordsGenerationParams recordsGenerationParams) Convert added and deleted action metadata to column stats index records.
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.convertMetadataToBloomFilterRecords(HoodieCleanMetadata cleanMetadata, HoodieEngineContext engineContext, String instantTime, MetadataRecordsGenerationParams recordsGenerationParams) Convert clean metadata to bloom filter index records.
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.convertMetadataToBloomFilterRecords(HoodieEngineContext context, HoodieCommitMetadata commitMetadata, String instantTime, MetadataRecordsGenerationParams recordsGenerationParams) Convert commit action metadata to bloom filter records.
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.convertMetadataToColumnStatsRecords(HoodieCleanMetadata cleanMetadata, HoodieEngineContext engineContext, MetadataRecordsGenerationParams recordsGenerationParams) Convert clean metadata to column stats index records.
static HoodieData<HoodieRecord>	HoodieTableMetadataUtil.convertMetadataToColumnStatsRecords(HoodieCommitMetadata commitMetadata, HoodieEngineContext engineContext, MetadataRecordsGenerationParams recordsGenerationParams)
HoodieData<HoodieRecord<HoodieMetadataPayload>>	FileSystemBackedTableMetadata.getRecordsByKeyPrefixes(List<String> keyPrefixes, String partitionName, boolean shouldLoadInMemory)
HoodieData<HoodieRecord<HoodieMetadataPayload>>	HoodieBackedTableMetadata.getRecordsByKeyPrefixes(List<String> keyPrefixes, String partitionName, boolean shouldLoadInMemory)
HoodieData<HoodieRecord<HoodieMetadataPayload>>	HoodieTableMetadata.getRecordsByKeyPrefixes(List<String> keyPrefixes, String partitionName, boolean shouldLoadInMemory) Fetch records by key prefixes.

Methods in org.apache.hudi.metadata that return types with arguments of type HoodieData

Modifier and Type	Method and Description
static Map<MetadataPartitionType,HoodieData<HoodieRecord>>	HoodieTableMetadataUtil.convertMetadataToRecords(HoodieEngineContext engineContext, HoodieActiveTimeline metadataTableTimeline, HoodieRestoreMetadata restoreMetadata, MetadataRecordsGenerationParams recordsGenerationParams, String instantTime, Option<String> lastSyncTs) Convert restore action metadata to metadata table records.
static Map<MetadataPartitionType,HoodieData<HoodieRecord>>	HoodieTableMetadataUtil.convertMetadataToRecords(HoodieEngineContext engineContext, HoodieActiveTimeline metadataTableTimeline, HoodieRollbackMetadata rollbackMetadata, MetadataRecordsGenerationParams recordsGenerationParams, String instantTime, Option<String> lastSyncTs, boolean wasSynced) Convert rollback action metadata to metadata table records.
static Map<MetadataPartitionType,HoodieData<HoodieRecord>>	HoodieTableMetadataUtil.convertMetadataToRecords(HoodieEngineContext engineContext, HoodieCleanMetadata cleanMetadata, MetadataRecordsGenerationParams recordsGenerationParams, String instantTime) Convert the clean action to metadata records.
static Map<MetadataPartitionType,HoodieData<HoodieRecord>>	HoodieTableMetadataUtil.convertMetadataToRecords(HoodieEngineContext context, HoodieCommitMetadata commitMetadata, String instantTime, MetadataRecordsGenerationParams recordsGenerationParams) Convert commit action to metadata records for the enabled partition types.