A B C D F G H I J L M N O P R S T U V W

A

AbstractRealtimeRecordReader - Class in org.apache.hudi.hadoop.realtime

Record Reader implementation to merge fresh avro data with base parquet data, to support real time queries.

AbstractRealtimeRecordReader(RealtimeSplit, JobConf) - Constructor for class org.apache.hudi.hadoop.realtime.AbstractRealtimeRecordReader

accept(Path) - Method in class org.apache.hudi.hadoop.HoodieROTablePathFilter

addPartitionFields(Schema, List<String>) - Static method in class org.apache.hudi.hadoop.utils.HoodieRealtimeRecordReaderUtils

Hive implementation of ParquetRecordReader results in partition columns not present in the original parquet file to also be part of the projected schema.

addRequiredProjectionFields(Configuration, Option<HoodieVirtualKeyInfo>) - Static method in class org.apache.hudi.hadoop.utils.HoodieRealtimeInputFormatUtils

addSplit(FileSplit) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit.Builder

arrayWritableToString(ArrayWritable) - Static method in class org.apache.hudi.hadoop.utils.HoodieRealtimeRecordReaderUtils

Prints a JSON representation of the ArrayWritable for easier debuggability.

avroToArrayWritable(Object, Schema) - Static method in class org.apache.hudi.hadoop.utils.HoodieRealtimeRecordReaderUtils

Convert the projected read from delta record into an array writable.

B

BootstrapBaseFileSplit - Class in org.apache.hudi.hadoop

Sub-type of File Split which encapsulates both skeleton and bootstrap base file splits.

BootstrapBaseFileSplit() - Constructor for class org.apache.hudi.hadoop.BootstrapBaseFileSplit

NOTE: This ctor is necessary for Hive to be able to serialize and then instantiate it when deserializing back

BootstrapBaseFileSplit(FileSplit, FileSplit) - Constructor for class org.apache.hudi.hadoop.BootstrapBaseFileSplit

BootstrapBaseFileSplit.WrapperFileSplit - Class in org.apache.hudi.hadoop

Wrapper for FileSplit just to expose default constructor to the outer class.

BootstrapColumnStichingRecordReader - Class in org.apache.hudi.hadoop

Stitches 2 record reader returned rows and presents a concatenated view to clients.

BootstrapColumnStichingRecordReader(RecordReader<NullWritable, ArrayWritable>, int, RecordReader<NullWritable, ArrayWritable>, int, boolean) - Constructor for class org.apache.hudi.hadoop.BootstrapColumnStichingRecordReader

build(JobConf) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit.Builder

Builder() - Constructor for class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit.Builder

buildMetadataConfig(Configuration) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

C

canAddProjectionToJobConf(RealtimeSplit, JobConf) - Static method in class org.apache.hudi.hadoop.utils.HoodieRealtimeInputFormatUtils

canSkipMerging(JobConf) - Static method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeRecordReader

checkIfValidFileSlice(FileSlice) - Method in class org.apache.hudi.hadoop.HoodieCopyOnWriteTableInputFormat

checkIfValidFileSlice(FileSlice) - Method in class org.apache.hudi.hadoop.realtime.HoodieMergeOnReadTableInputFormat

cleanProjectionColumnIds(Configuration) - Static method in class org.apache.hudi.hadoop.utils.HoodieRealtimeInputFormatUtils

Hive will append read columns' ids to old columns' ids during getRecordReader.

close() - Method in class org.apache.hudi.hadoop.BootstrapColumnStichingRecordReader

close() - Method in class org.apache.hudi.hadoop.HoodieHFileRecordReader

close() - Method in class org.apache.hudi.hadoop.realtime.HoodieCombineRealtimeRecordReader

close() - Method in class org.apache.hudi.hadoop.realtime.HoodieEmptyRecordReader

close() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeRecordReader

close() - Method in class org.apache.hudi.hadoop.RecordReaderValueIterator

close() - Method in class org.apache.hudi.hadoop.SafeParquetRecordReaderWrapper

com.uber.hoodie.hadoop - package com.uber.hoodie.hadoop

com.uber.hoodie.hadoop.realtime - package com.uber.hoodie.hadoop.realtime

CombineHiveInputSplit() - Constructor for class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

CombineHiveInputSplit(CombineFileSplit) - Constructor for class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

CombineHiveInputSplit(JobConf, CombineFileSplit) - Constructor for class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

CombineHiveInputSplit(JobConf, CombineFileSplit, Map<Path, PartitionDesc>) - Constructor for class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

COMPACTION_LAZY_BLOCK_READ_ENABLED_PROP - Static variable in class org.apache.hudi.hadoop.config.HoodieRealtimeConfig

COMPACTION_MEMORY_FRACTION_PROP - Static variable in class org.apache.hudi.hadoop.config.HoodieRealtimeConfig

conf - Variable in class org.apache.hudi.hadoop.HoodieTableInputFormat

createFileStatusUnchecked(FileSlice, Option<HoodieInstant>, String, Option<HoodieVirtualKeyInfo>) - Method in class org.apache.hudi.hadoop.HoodieCopyOnWriteTableInputFormat

createFileStatusUnchecked(FileSlice, Option<HoodieInstant>, String, Option<HoodieVirtualKeyInfo>) - Method in class org.apache.hudi.hadoop.realtime.HoodieMergeOnReadTableInputFormat

createInputFormatShim() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat

createKey() - Method in class org.apache.hudi.hadoop.BootstrapColumnStichingRecordReader

createKey() - Method in class org.apache.hudi.hadoop.HoodieHFileRecordReader

createKey() - Method in class org.apache.hudi.hadoop.realtime.HoodieCombineRealtimeRecordReader

createKey() - Method in class org.apache.hudi.hadoop.realtime.HoodieEmptyRecordReader

createKey() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeRecordReader

createKey() - Method in class org.apache.hudi.hadoop.SafeParquetRecordReaderWrapper

createParquetInputFormat() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.HoodieCombineFileInputFormatShim

createParquetRealtimeInputFormat() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.HoodieCombineFileInputFormatShim

createPool(JobConf, PathFilter...) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.HoodieCombineFileInputFormatShim

createValue() - Method in class org.apache.hudi.hadoop.BootstrapColumnStichingRecordReader

createValue() - Method in class org.apache.hudi.hadoop.HoodieHFileRecordReader

createValue() - Method in class org.apache.hudi.hadoop.realtime.HoodieCombineRealtimeRecordReader

createValue() - Method in class org.apache.hudi.hadoop.realtime.HoodieEmptyRecordReader

createValue() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeRecordReader

createValue() - Method in class org.apache.hudi.hadoop.SafeParquetRecordReaderWrapper

We could be in concurrent fetch and read env.

D

DEFAULT_COMPACTION_LAZY_BLOCK_READ_ENABLED - Static variable in class org.apache.hudi.hadoop.config.HoodieRealtimeConfig

DEFAULT_COMPACTION_MEMORY_FRACTION - Static variable in class org.apache.hudi.hadoop.config.HoodieRealtimeConfig

DEFAULT_MAX_COMMITS - Static variable in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

DEFAULT_MAX_DFS_STREAM_BUFFER_SIZE - Static variable in class org.apache.hudi.hadoop.config.HoodieRealtimeConfig

DEFAULT_REALTIME_SKIP_MERGE - Static variable in class org.apache.hudi.hadoop.realtime.HoodieRealtimeRecordReader

DEFAULT_SCAN_MODE - Static variable in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

DEFAULT_SPILLABLE_MAP_BASE_PATH - Static variable in class org.apache.hudi.hadoop.config.HoodieRealtimeConfig

doesBelongToIncrementalQuery(FileSplit) - Static method in class org.apache.hudi.hadoop.utils.HoodieRealtimeInputFormatUtils

doListStatus(JobConf) - Method in class org.apache.hudi.hadoop.HoodieCopyOnWriteTableInputFormat

Abstracts and exposes FileInputFormat.listStatus(JobConf) operation to subclasses that lists files (returning an array of FileStatus) corresponding to the input paths specified as part of provided JobConf

F

fileSplits - Variable in class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit.Builder

FileStatusWithBootstrapBaseFile - Class in org.apache.hudi.hadoop

Sub-Type of File Status tracking both skeleton and bootstrap base file's status.

FileStatusWithBootstrapBaseFile(FileStatus, FileStatus) - Constructor for class org.apache.hudi.hadoop.FileStatusWithBootstrapBaseFile

filterIncrementalFileStatus(Job, HoodieTableMetaClient, HoodieTimeline, FileStatus[], List<HoodieInstant>) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

Filter a list of FileStatus based on commitsToCheck for incremental view.

filterInstantsTimeline(HoodieDefaultTimeline) - Method in class org.apache.hudi.hadoop.HoodieHFileInputFormat

filterInstantsTimeline(HoodieDefaultTimeline) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

Filter any specific instants that we do not want to process.

G

generateProjectionSchema(Schema, Map<String, Schema.Field>, List<String>) - Static method in class org.apache.hudi.hadoop.utils.HoodieRealtimeRecordReaderUtils

Generate a reader schema off the provided writeSchema, to just project out the provided columns.

getAffectedPartitions(List<HoodieInstant>, HoodieTableMetaClient, HoodieTimeline, List<Path>) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

Extract partitions touched by the commitsToCheck.

getBasePath() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeBootstrapBaseFileSplit

getBasePath() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit

getBasePath() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimePath

getBasePath() - Method in interface org.apache.hudi.hadoop.realtime.RealtimeSplit

Return Base Path of the dataset.

getBelongsToIncrementalQuery() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeBootstrapBaseFileSplit

getBelongsToIncrementalQuery() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit

getBelongsToIncrementalQuery() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimePath

getBelongsToIncrementalQuery() - Method in interface org.apache.hudi.hadoop.realtime.RealtimeSplit

Returns the flag whether this split belongs to an Incremental Query

getBootstrapFileSplit() - Method in class org.apache.hudi.hadoop.BootstrapBaseFileSplit

getBootstrapFileStatus() - Method in class org.apache.hudi.hadoop.PathWithBootstrapFileStatus

getCommitMetadata(HoodieInstant, HoodieTimeline) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

Returns the commit metadata of the given instant.

getCommitsForIncrementalQuery(Job, String, HoodieTimeline) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

Get commits for incremental query from Hive map reduce configuration.

getConf() - Method in class org.apache.hudi.hadoop.HoodieParquetInputFormatBase

getConf() - Method in class org.apache.hudi.hadoop.HoodieROTablePathFilter

getConf() - Method in class org.apache.hudi.hadoop.HoodieTableInputFormat

getDeltaLogFiles() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeBootstrapBaseFileSplit

getDeltaLogFiles() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit

getDeltaLogFiles() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimePath

getDeltaLogFiles() - Method in interface org.apache.hudi.hadoop.realtime.RealtimeSplit

getDeltaLogFiles() - Method in class org.apache.hudi.hadoop.RealtimeFileStatus

getDeltaLogPaths() - Method in interface org.apache.hudi.hadoop.realtime.RealtimeSplit

Return Log File Paths.

getFileStatus(HoodieBaseFile) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

getFileStatusUnchecked(HoodieBaseFile) - Static method in class org.apache.hudi.hadoop.HoodieCopyOnWriteTableInputFormat

getFilteredCommitsTimeline(JobContext, HoodieTableMetaClient) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

Extract HoodieTimeline based on HoodieTableMetaClient.

getGroupedIncrementalPaths() - Method in class org.apache.hudi.hadoop.InputPathHandler

getHiveSchema() - Method in class org.apache.hudi.hadoop.realtime.AbstractRealtimeRecordReader

getHoodieTimelineForIncrementalQuery(JobContext, String, HoodieTimeline) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

Get HoodieTimeline for incremental query from Hive map reduce configuration.

getHoodieVirtualKeyInfo(HoodieTableMetaClient) - Static method in class org.apache.hudi.hadoop.HoodieCopyOnWriteTableInputFormat

getIncrementalTableNames(JobContext) - Static method in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

Returns a list of tableNames for which hoodie..consume.mode is set to incremental else returns empty List

getInputFormat(HoodieFileFormat, boolean, Configuration) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

getInputFormat(String, boolean, Configuration) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

getInputFormatClassName(HoodieFileFormat, boolean) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

getInputPathsShim(JobConf) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.HoodieCombineFileInputFormatShim

getInputSplitShim() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

getInputSplitShim() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.HoodieCombineFileInputFormatShim

getIOColumnNameAndTypes(Configuration) - Static method in class org.apache.hudi.hadoop.HoodieColumnProjectionUtils

getIOColumns(Configuration) - Static method in class org.apache.hudi.hadoop.HoodieColumnProjectionUtils

getIOColumnTypes(Configuration) - Static method in class org.apache.hudi.hadoop.HoodieColumnProjectionUtils

getJob() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

getJobConf() - Method in class org.apache.hudi.hadoop.realtime.AbstractRealtimeRecordReader

getLength() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

getLength(int) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

Returns the length of the i^th Path.

getLength() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit

getLength(int) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit

Returns the length of the i^th Path.

getLength() - Method in interface org.apache.hudi.hadoop.realtime.RealtimeSplit

The number of bytes in the file to process.

getLengths() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

Returns an array containing the lengths of the files in the split.

getLengths() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit

Returns an array containing the lengths of the files in the split.

getLocations() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

Returns all the Paths where this input-split resides.

getLocations() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit

Returns all the Paths where this input-split resides.

getMaxCommit(JobConf, String) - Static method in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

getMaxCommitTime() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeBootstrapBaseFileSplit

getMaxCommitTime() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit

getMaxCommitTime() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimePath

getMaxCommitTime() - Method in interface org.apache.hudi.hadoop.realtime.RealtimeSplit

Return Max Instant Time.

getMaxCommitTime() - Method in class org.apache.hudi.hadoop.RealtimeFileStatus

getMaxCompactionMemoryInBytes(JobConf) - Static method in class org.apache.hudi.hadoop.utils.HoodieRealtimeRecordReaderUtils

get the max compaction memory in bytes from JobConf.

getNameToFieldMap(Schema) - Static method in class org.apache.hudi.hadoop.utils.HoodieRealtimeRecordReaderUtils

getNonCombinablePathIndices(JobConf, Path[], int) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat

Gets all the path indices that should not be combined.

getNonHoodieInputPaths() - Method in class org.apache.hudi.hadoop.InputPathHandler

getNthParent(Path, int) - Static method in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

Gets the n'th parent for the Path.

getNumPaths() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

Returns the number of Paths in the split.

getNumPaths() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit

Returns the number of Paths in the split.

getOffset(int) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

Returns the start offset of the i^th Path.

getOffset(int) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit

Returns the start offset of the i^th Path.

getOutputFormatClassName(HoodieFileFormat) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

getParquetInputFormatClassName() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat

getParquetRealtimeInputFormatClassName() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat

getPartitionPathField() - Method in class org.apache.hudi.hadoop.realtime.HoodieVirtualKeyInfo

getPartitionPathFieldIndex() - Method in class org.apache.hudi.hadoop.realtime.HoodieVirtualKeyInfo

getPath() - Method in class org.apache.hudi.hadoop.FileStatusWithBootstrapBaseFile

getPath(int) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

Returns the i^th Path.

getPath(int) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit

Returns the i^th Path.

getPath() - Method in class org.apache.hudi.hadoop.LocatedFileStatusWithBootstrapBaseFile

getPath() - Method in interface org.apache.hudi.hadoop.realtime.RealtimeSplit

The file containing this split's data.

getPath() - Method in class org.apache.hudi.hadoop.RealtimeFileStatus

getPaths() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

Returns all the Paths in the split.

getPaths() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit

Returns all the Paths in the split.

getPathWithBootstrapFileStatus() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimePath

getPos() - Method in class org.apache.hudi.hadoop.BootstrapColumnStichingRecordReader

getPos() - Method in class org.apache.hudi.hadoop.HoodieHFileRecordReader

getPos() - Method in class org.apache.hudi.hadoop.realtime.HoodieCombineRealtimeRecordReader

getPos() - Method in class org.apache.hudi.hadoop.realtime.HoodieEmptyRecordReader

getPos() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeRecordReader

getPos() - Method in class org.apache.hudi.hadoop.SafeParquetRecordReaderWrapper

getProgress() - Method in class org.apache.hudi.hadoop.BootstrapColumnStichingRecordReader

getProgress() - Method in class org.apache.hudi.hadoop.HoodieHFileRecordReader

getProgress() - Method in class org.apache.hudi.hadoop.realtime.HoodieCombineRealtimeRecordReader

getProgress() - Method in class org.apache.hudi.hadoop.realtime.HoodieEmptyRecordReader

getProgress() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeRecordReader

getProgress() - Method in class org.apache.hudi.hadoop.SafeParquetRecordReaderWrapper

getReadColumnIDs(Configuration) - Static method in class org.apache.hudi.hadoop.HoodieColumnProjectionUtils

Returns an array of column ids(start from zero) which is set in the given parameter conf.

getReadColumnNames(Configuration) - Static method in class org.apache.hudi.hadoop.HoodieColumnProjectionUtils

getReader() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeRecordReader

getReaderSchema() - Method in class org.apache.hudi.hadoop.realtime.AbstractRealtimeRecordReader

getRealtimeFileSplits() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit

getRecordKeyField() - Method in class org.apache.hudi.hadoop.realtime.HoodieVirtualKeyInfo

getRecordKeyFieldIndex() - Method in class org.apache.hudi.hadoop.realtime.HoodieVirtualKeyInfo

getRecordReader(InputSplit, JobConf, Reporter) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat

Create a generic Hive RecordReader than can iterate over all chunks in a CombinedFileSplit.

getRecordReader(InputSplit, JobConf, Reporter) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.HoodieCombineFileInputFormatShim

getRecordReader(JobConf, CombineFileSplit, Reporter, Class<RecordReader<K, V>>) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.HoodieCombineFileInputFormatShim

getRecordReader(InputSplit, JobConf, Reporter) - Method in class org.apache.hudi.hadoop.HoodieCopyOnWriteTableInputFormat

getRecordReader(InputSplit, JobConf, Reporter) - Method in class org.apache.hudi.hadoop.HoodieHFileInputFormat

getRecordReader(InputSplit, JobConf, Reporter) - Method in class org.apache.hudi.hadoop.HoodieParquetInputFormat

getRecordReader(InputSplit, JobConf, Reporter) - Method in class org.apache.hudi.hadoop.realtime.HoodieHFileRealtimeInputFormat

getRecordReader(InputSplit, JobConf, Reporter) - Method in class org.apache.hudi.hadoop.realtime.HoodieParquetRealtimeInputFormat

getSerDeClassName(HoodieFileFormat) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

getSnapshotPaths() - Method in class org.apache.hudi.hadoop.InputPathHandler

getSplit() - Method in class org.apache.hudi.hadoop.realtime.AbstractRealtimeRecordReader

getSplits(JobConf, int) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat

Create Hive splits based on CombineFileSplit.

getSplits(JobConf, int) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.HoodieCombineFileInputFormatShim

getSplits(JobConf, int) - Method in class org.apache.hudi.hadoop.HoodieParquetInputFormatBase

getSplits(JobConf, int) - Method in class org.apache.hudi.hadoop.realtime.HoodieMergeOnReadTableInputFormat

getStart() - Method in interface org.apache.hudi.hadoop.realtime.RealtimeSplit

The position of the first byte in the file to process.

getStartOffsets() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

Returns an array containing the startoffsets of the files in the split.

getStartOffsets() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit

Returns an array containing the start offsets of the files in the split.

getTableMetaClientByPartitionPath(Configuration, Set<Path>) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

Extract HoodieTableMetaClient by partition path.

getTableMetaClientForBasePathUnchecked(Configuration, Path) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

Extract HoodieTableMetaClient from a partition path (not base path)

getTableMetaClientMap() - Method in class org.apache.hudi.hadoop.InputPathHandler

getVirtualKeyInfo() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeBootstrapBaseFileSplit

getVirtualKeyInfo() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit

getVirtualKeyInfo() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimePath

getVirtualKeyInfo() - Method in interface org.apache.hudi.hadoop.realtime.RealtimeSplit

Returns Virtual key info if meta fields are disabled.

getWritePartitionPaths(List<HoodieCommitMetadata>) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

Returns all the incremental write partition paths as a set with the given commits metadata.

getWriterSchema() - Method in class org.apache.hudi.hadoop.realtime.AbstractRealtimeRecordReader

GLOBALLY_CONSISTENT_READ_TIMESTAMP - Static variable in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

groupFileStatusForSnapshotPaths(FileStatus[], String, Collection<HoodieTableMetaClient>) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

Takes in a list of filesStatus and a list of table metadatas.

groupSnapshotPathsByMetaClient(Collection<HoodieTableMetaClient>, List<Path>) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

H

hasNext() - Method in class org.apache.hudi.hadoop.RecordReaderValueIterator

HiveHoodieTableFileIndex - Class in org.apache.hudi.hadoop

Implementation of BaseHoodieTableFileIndex for Hive-based query engines

HiveHoodieTableFileIndex(HoodieEngineContext, HoodieTableMetaClient, TypedProperties, HoodieTableQueryType, List<Path>, Option<String>, boolean) - Constructor for class org.apache.hudi.hadoop.HiveHoodieTableFileIndex

HOODIE_COMMIT_TIME_COL_POS - Static variable in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

HOODIE_CONSUME_COMMIT - Static variable in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

HOODIE_CONSUME_MODE_PATTERN - Static variable in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

HOODIE_CONSUME_MODE_PATTERN_STRING - Static variable in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

HOODIE_CONSUME_PENDING_COMMITS - Static variable in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

HOODIE_INCREMENTAL_USE_DATABASE - Static variable in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

HOODIE_MAX_COMMIT_PATTERN - Static variable in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

HOODIE_PARTITION_PATH_COL_POS - Static variable in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

HOODIE_READ_COLUMNS_PROP - Static variable in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

HOODIE_RECORD_KEY_COL_POS - Static variable in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

HOODIE_START_COMMIT_PATTERN - Static variable in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

HOODIE_STOP_AT_COMPACTION_PATTERN - Static variable in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

HoodieColumnProjectionUtils - Class in org.apache.hudi.hadoop

Utility functions copied from Hive ColumnProjectionUtils.java.

HoodieColumnProjectionUtils() - Constructor for class org.apache.hudi.hadoop.HoodieColumnProjectionUtils

HoodieCombineFileInputFormatShim() - Constructor for class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.HoodieCombineFileInputFormatShim

HoodieCombineHiveInputFormat<K extends org.apache.hadoop.io.WritableComparable,V extends org.apache.hadoop.io.Writable> - Class in org.apache.hudi.hadoop.hive

This is just a copy of the org.apache.hadoop.hive.ql.io.CombineHiveInputFormat from Hive 2.x Search for **MOD** to see minor modifications to support custom inputformat in CombineHiveInputFormat.

HoodieCombineHiveInputFormat() - Constructor for class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat

HoodieCombineHiveInputFormat.AvoidSplitCombination - Interface in org.apache.hudi.hadoop.hive

This is a marker interface that is used to identify the formats where combine split generation is not applicable.

HoodieCombineHiveInputFormat.CombineHiveInputSplit - Class in org.apache.hudi.hadoop.hive

CombineHiveInputSplit encapsulates an InputSplit with its corresponding inputFormatClassName.

HoodieCombineHiveInputFormat.HoodieCombineFileInputFormatShim<K,V> - Class in org.apache.hudi.hadoop.hive

**MOD** This is the implementation of CombineFileInputFormat which is a copy of org.apache.hadoop.hive.shims.HadoopShimsSecure.CombineFileInputFormatShim with changes in listStatus.

HoodieCombineRealtimeFileSplit - Class in org.apache.hudi.hadoop.hive

Represents a CombineFileSplit for realtime tables.

HoodieCombineRealtimeFileSplit() - Constructor for class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit

HoodieCombineRealtimeFileSplit(JobConf, List<FileSplit>) - Constructor for class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit

HoodieCombineRealtimeFileSplit.Builder - Class in org.apache.hudi.hadoop.hive

HoodieCombineRealtimeHiveSplit - Class in org.apache.hudi.hadoop.hive

Represents a CombineHiveInputSplit for realtime tables.

HoodieCombineRealtimeHiveSplit() - Constructor for class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeHiveSplit

HoodieCombineRealtimeHiveSplit(JobConf, CombineFileSplit, Map<Path, PartitionDesc>) - Constructor for class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeHiveSplit

HoodieCombineRealtimeRecordReader - Class in org.apache.hudi.hadoop.realtime

Allows to read multiple realtime file splits grouped together by CombineInputFormat.

HoodieCombineRealtimeRecordReader(JobConf, CombineFileSplit, List<RecordReader>) - Constructor for class org.apache.hudi.hadoop.realtime.HoodieCombineRealtimeRecordReader

HoodieCopyOnWriteTableInputFormat - Class in org.apache.hudi.hadoop

Base implementation of the Hive's FileInputFormat allowing for reading of Hudi's Copy-on-Write (COW) tables in various configurations: Snapshot mode: reading table's state as of particular timestamp (or instant, in Hudi's terms) Incremental mode: reading table's state as of particular timestamp (or instant, in Hudi's terms) External mode: reading non-Hudi partitions NOTE: This class is invariant of the underlying file-format of the files being read

HoodieCopyOnWriteTableInputFormat() - Constructor for class org.apache.hudi.hadoop.HoodieCopyOnWriteTableInputFormat

HoodieEmptyRecordReader - Class in org.apache.hudi.hadoop.realtime

Dummy record for log only realtime split.

HoodieEmptyRecordReader(RealtimeSplit, JobConf) - Constructor for class org.apache.hudi.hadoop.realtime.HoodieEmptyRecordReader

HoodieHFileInputFormat - Class in org.apache.hudi.hadoop

HoodieInputFormat for HUDI datasets which store data in HFile base file format.

HoodieHFileInputFormat() - Constructor for class org.apache.hudi.hadoop.HoodieHFileInputFormat

HoodieHFileRealtimeInputFormat - Class in org.apache.hudi.hadoop.realtime

HoodieRealtimeInputFormat for HUDI datasets which store data in HFile base file format.

HoodieHFileRealtimeInputFormat() - Constructor for class org.apache.hudi.hadoop.realtime.HoodieHFileRealtimeInputFormat

HoodieHFileRecordReader - Class in org.apache.hudi.hadoop

HoodieHFileRecordReader(Configuration, InputSplit, JobConf) - Constructor for class org.apache.hudi.hadoop.HoodieHFileRecordReader

HoodieHiveUtils - Class in org.apache.hudi.hadoop.utils

HoodieHiveUtils() - Constructor for class org.apache.hudi.hadoop.utils.HoodieHiveUtils

HoodieInputFormat - Class in com.uber.hoodie.hadoop

Temporary class to allow seamless migration of com.uber.hoodie to org.apache.hudi .

HoodieInputFormat() - Constructor for class com.uber.hoodie.hadoop.HoodieInputFormat

HoodieInputFormatUtils - Class in org.apache.hudi.hadoop.utils

HoodieInputFormatUtils() - Constructor for class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

HoodieMergeOnReadTableInputFormat - Class in org.apache.hudi.hadoop.realtime

Base implementation of the Hive's FileInputFormat allowing for reading of Hudi's Merge-on-Read (COW) tables in various configurations: Snapshot mode: reading table's state as of particular timestamp (or instant, in Hudi's terms) Incremental mode: reading table's state as of particular timestamp (or instant, in Hudi's terms) External mode: reading non-Hudi partitions

HoodieMergeOnReadTableInputFormat() - Constructor for class org.apache.hudi.hadoop.realtime.HoodieMergeOnReadTableInputFormat

HoodieParquetInputFormat - Class in org.apache.hudi.hadoop

HoodieInputFormat which understands the Hoodie File Structure and filters files based on the Hoodie Mode.

HoodieParquetInputFormat() - Constructor for class org.apache.hudi.hadoop.HoodieParquetInputFormat

HoodieParquetInputFormat(HoodieCopyOnWriteTableInputFormat) - Constructor for class org.apache.hudi.hadoop.HoodieParquetInputFormat

HoodieParquetInputFormatBase - Class in org.apache.hudi.hadoop

!!! PLEASE READ CAREFULLY !!! NOTE: Hive bears optimizations which are based upon validating whether FileInputFormat implementation inherits from MapredParquetInputFormat.

HoodieParquetInputFormatBase(HoodieCopyOnWriteTableInputFormat) - Constructor for class org.apache.hudi.hadoop.HoodieParquetInputFormatBase

HoodieParquetRealtimeInputFormat - Class in org.apache.hudi.hadoop.realtime

Input Format, that provides a real-time view of data in a Hoodie table.

HoodieParquetRealtimeInputFormat() - Constructor for class org.apache.hudi.hadoop.realtime.HoodieParquetRealtimeInputFormat

HoodieParquetSerde - Class in org.apache.hudi.hadoop.realtime

Simply extends ParquetHiveSerDe.

HoodieParquetSerde() - Constructor for class org.apache.hudi.hadoop.realtime.HoodieParquetSerde

HoodieRealtimeBootstrapBaseFileSplit - Class in org.apache.hudi.hadoop.realtime

Realtime FileSplit with external base file NOTE: If you're adding fields here you need to make sure that you appropriately de-/serialize them in RealtimeSplit.readFromInput(DataInput) and RealtimeSplit.writeToOutput(DataOutput)

HoodieRealtimeBootstrapBaseFileSplit() - Constructor for class org.apache.hudi.hadoop.realtime.HoodieRealtimeBootstrapBaseFileSplit

NOTE: This ctor is necessary for Hive to be able to serialize and then instantiate it when deserializing back

HoodieRealtimeBootstrapBaseFileSplit(FileSplit, String, List<HoodieLogFile>, String, FileSplit, boolean, Option<HoodieVirtualKeyInfo>) - Constructor for class org.apache.hudi.hadoop.realtime.HoodieRealtimeBootstrapBaseFileSplit

HoodieRealtimeConfig - Class in org.apache.hudi.hadoop.config

Class to hold props related to Hoodie RealtimeInputFormat and RealtimeRecordReader.

HoodieRealtimeConfig() - Constructor for class org.apache.hudi.hadoop.config.HoodieRealtimeConfig

HoodieRealtimeFileSplit - Class in org.apache.hudi.hadoop.realtime

FileSplit implementation that holds Split corresponding to the base file List of HoodieLogFile that holds the delta to be merged (upon reading) This split is correspondent to a single file-slice in the Hudi terminology.

HoodieRealtimeFileSplit() - Constructor for class org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit

HoodieRealtimeFileSplit(FileSplit, HoodieRealtimePath) - Constructor for class org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit

HoodieRealtimeFileSplit(FileSplit, String, List<HoodieLogFile>, String, boolean, Option<HoodieVirtualKeyInfo>) - Constructor for class org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit

HoodieRealtimeInputFormat - Class in com.uber.hoodie.hadoop.realtime

Temporary class to allow seamless migration of com.uber.hoodie to org.apache.hudi .

HoodieRealtimeInputFormat() - Constructor for class com.uber.hoodie.hadoop.realtime.HoodieRealtimeInputFormat

HoodieRealtimeInputFormatUtils - Class in org.apache.hudi.hadoop.utils

HoodieRealtimeInputFormatUtils() - Constructor for class org.apache.hudi.hadoop.utils.HoodieRealtimeInputFormatUtils

HoodieRealtimePath - Class in org.apache.hudi.hadoop.realtime

Path implementation encoding additional information necessary to appropriately read base files of the MOR tables, such as list of delta log files (holding updated records) associated w/ the base file, etc.

HoodieRealtimePath(Path, String, String, List<HoodieLogFile>, String, boolean, Option<HoodieVirtualKeyInfo>) - Constructor for class org.apache.hudi.hadoop.realtime.HoodieRealtimePath

HoodieRealtimeRecordReader - Class in org.apache.hudi.hadoop.realtime

Realtime Record Reader which can do compacted (merge-on-read) record reading or unmerged reading (parquet and log files read in parallel) based on job configuration.

HoodieRealtimeRecordReader(RealtimeSplit, JobConf, RecordReader<NullWritable, ArrayWritable>) - Constructor for class org.apache.hudi.hadoop.realtime.HoodieRealtimeRecordReader

HoodieRealtimeRecordReaderUtils - Class in org.apache.hudi.hadoop.utils

HoodieRealtimeRecordReaderUtils() - Constructor for class org.apache.hudi.hadoop.utils.HoodieRealtimeRecordReaderUtils

HoodieROTablePathFilter - Class in org.apache.hudi.hadoop

Given a path is a part of - Hoodie table = accepts ONLY the latest version of each path - Non-Hoodie table = then always accept

HoodieROTablePathFilter() - Constructor for class org.apache.hudi.hadoop.HoodieROTablePathFilter

HoodieROTablePathFilter(Configuration) - Constructor for class org.apache.hudi.hadoop.HoodieROTablePathFilter

HoodieTableInputFormat - Class in org.apache.hudi.hadoop

Abstract base class of the Hive's FileInputFormat implementations allowing for reading of Hudi's Copy-on-Write (COW) and Merge-on-Read (MOR) tables

HoodieTableInputFormat() - Constructor for class org.apache.hudi.hadoop.HoodieTableInputFormat

HoodieVirtualKeyInfo - Class in org.apache.hudi.hadoop.realtime

Class to hold virtual key info when meta fields are disabled.

HoodieVirtualKeyInfo(String, Option<String>, int, Option<Integer>) - Constructor for class org.apache.hudi.hadoop.realtime.HoodieVirtualKeyInfo

I

includeBootstrapFilePath() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimePath

INCREMENTAL_SCAN_MODE - Static variable in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

inputFormatClassName() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

Returns the inputFormat class name for the i-th chunk.

InputPathHandler - Class in org.apache.hudi.hadoop

InputPathHandler takes in a set of input paths and incremental tables list.

InputPathHandler(Configuration, Path[], List<String>) - Constructor for class org.apache.hudi.hadoop.InputPathHandler

inputSplitShim - Variable in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

InputSplitUtils - Class in org.apache.hudi.hadoop

InputSplitUtils() - Constructor for class org.apache.hudi.hadoop.InputSplitUtils

isIncrementalUseDatabase(Configuration) - Static method in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

isSplitable(FileSystem, Path) - Method in class org.apache.hudi.hadoop.HoodieCopyOnWriteTableInputFormat

isSplitable(FileSystem, Path) - Method in class org.apache.hudi.hadoop.HoodieHFileInputFormat

isSplitable(FileSystem, Path) - Method in class org.apache.hudi.hadoop.HoodieParquetInputFormatBase

isSplitable(FileSystem, Path) - Method in class org.apache.hudi.hadoop.HoodieTableInputFormat

isSplitable(FileSystem, Path) - Method in class org.apache.hudi.hadoop.realtime.HoodieHFileRealtimeInputFormat

isSplitable(FileSystem, Path) - Method in class org.apache.hudi.hadoop.realtime.HoodieMergeOnReadTableInputFormat

isSplitable() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimePath

J

jobConf - Variable in class org.apache.hudi.hadoop.realtime.AbstractRealtimeRecordReader

L

listAffectedFilesForCommits(Configuration, Path, List<HoodieCommitMetadata>) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

Iterate through a list of commit metadata in natural order, and extract the file status of all affected files from the commits metadata grouping by file full path.

listStatus(JobContext) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.HoodieCombineFileInputFormatShim

listStatus(JobConf) - Method in class org.apache.hudi.hadoop.HoodieCopyOnWriteTableInputFormat

listStatus(JobConf) - Method in class org.apache.hudi.hadoop.HoodieParquetInputFormatBase

listStatus(JobConf) - Method in class org.apache.hudi.hadoop.HoodieTableInputFormat

listStatusForIncrementalMode(JobConf, HoodieTableMetaClient, List<Path>, String) - Method in class org.apache.hudi.hadoop.HoodieCopyOnWriteTableInputFormat

Achieves listStatus functionality for an incrementally queried table.

listStatusForIncrementalMode(JobConf, HoodieTableMetaClient, List<Path>, String) - Method in class org.apache.hudi.hadoop.realtime.HoodieMergeOnReadTableInputFormat

Keep the logic of mor_incr_view as same as spark datasource.

LocatedFileStatusWithBootstrapBaseFile - Class in org.apache.hudi.hadoop

Sub-Type of File Status tracking both skeleton and bootstrap base file's status.

LocatedFileStatusWithBootstrapBaseFile(LocatedFileStatus, FileStatus) - Constructor for class org.apache.hudi.hadoop.LocatedFileStatusWithBootstrapBaseFile

LOG - Static variable in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat

LOG - Static variable in class org.apache.hudi.hadoop.HiveHoodieTableFileIndex

LOG - Static variable in class org.apache.hudi.hadoop.HoodieColumnProjectionUtils

LOG - Static variable in class org.apache.hudi.hadoop.InputPathHandler

LOG - Static variable in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

M

makeSplit(Path, long, long, String[]) - Method in class org.apache.hudi.hadoop.HoodieCopyOnWriteTableInputFormat

makeSplit(Path, long, long, String[], String[]) - Method in class org.apache.hudi.hadoop.HoodieCopyOnWriteTableInputFormat

makeSplit(Path, long, long, String[]) - Method in class org.apache.hudi.hadoop.HoodieParquetInputFormatBase

makeSplit(Path, long, long, String[], String[]) - Method in class org.apache.hudi.hadoop.HoodieParquetInputFormatBase

makeSplit(Path, long, long, String[]) - Method in class org.apache.hudi.hadoop.HoodieTableInputFormat

makeSplit(Path, long, long, String[], String[]) - Method in class org.apache.hudi.hadoop.HoodieTableInputFormat

makeSplit(Path, long, long, String[]) - Method in class org.apache.hudi.hadoop.realtime.HoodieMergeOnReadTableInputFormat

makeSplit(Path, long, long, String[], String[]) - Method in class org.apache.hudi.hadoop.realtime.HoodieMergeOnReadTableInputFormat

MAX_COMMIT_ALL - Static variable in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

MAX_DFS_STREAM_BUFFER_SIZE_PROP - Static variable in class org.apache.hudi.hadoop.config.HoodieRealtimeConfig

N

next(NullWritable, ArrayWritable) - Method in class org.apache.hudi.hadoop.BootstrapColumnStichingRecordReader

next(NullWritable, ArrayWritable) - Method in class org.apache.hudi.hadoop.HoodieHFileRecordReader

next(NullWritable, ArrayWritable) - Method in class org.apache.hudi.hadoop.realtime.HoodieCombineRealtimeRecordReader

next(NullWritable, ArrayWritable) - Method in class org.apache.hudi.hadoop.realtime.HoodieEmptyRecordReader

next(NullWritable, ArrayWritable) - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeRecordReader

next() - Method in class org.apache.hudi.hadoop.RecordReaderValueIterator

next(NullWritable, ArrayWritable) - Method in class org.apache.hudi.hadoop.SafeParquetRecordReaderWrapper

O

orderFields(String, String, List<String>) - Static method in class org.apache.hudi.hadoop.utils.HoodieRealtimeRecordReaderUtils

Given a comma separated list of field names and positions at which they appear on Hive, return an ordered list of field names, that can be passed onto storage.

org.apache.hudi.hadoop - package org.apache.hudi.hadoop

org.apache.hudi.hadoop.config - package org.apache.hudi.hadoop.config

org.apache.hudi.hadoop.hive - package org.apache.hudi.hadoop.hive

org.apache.hudi.hadoop.realtime - package org.apache.hudi.hadoop.realtime

org.apache.hudi.hadoop.utils - package org.apache.hudi.hadoop.utils

P

parsePartitionColumnValues(String[], String) - Method in class org.apache.hudi.hadoop.HiveHoodieTableFileIndex

PathWithBootstrapFileStatus - Class in org.apache.hudi.hadoop

Hacky Workaround !!! With the base input format implementations in Hadoop/Hive, we need to encode additional information in Path to track matching external file.

PathWithBootstrapFileStatus(Path, FileStatus) - Constructor for class org.apache.hudi.hadoop.PathWithBootstrapFileStatus

payloadProps - Variable in class org.apache.hudi.hadoop.realtime.AbstractRealtimeRecordReader

R

READ_COLUMN_IDS_CONF_STR - Static variable in class org.apache.hudi.hadoop.HoodieColumnProjectionUtils

READ_COLUMN_NAMES_CONF_STR - Static variable in class org.apache.hudi.hadoop.HoodieColumnProjectionUtils

the nested column path is the string from the root to the leaf e.g.

readBoolean(DataInput) - Static method in class org.apache.hudi.hadoop.InputSplitUtils

readFields(DataInput) - Method in class org.apache.hudi.hadoop.BootstrapBaseFileSplit

readFields(DataInput) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

Writable interface.

readFields(DataInput) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit

readFields(DataInput) - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeBootstrapBaseFileSplit

readFields(DataInput) - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit

readFromInput(DataInput) - Method in interface org.apache.hudi.hadoop.realtime.RealtimeSplit

readMaxCommits(JobContext, String) - Static method in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

readStartCommitTime(JobContext, String) - Static method in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

readString(DataInput) - Static method in class org.apache.hudi.hadoop.InputSplitUtils

REALTIME_SKIP_MERGE_PROP - Static variable in class org.apache.hudi.hadoop.realtime.HoodieRealtimeRecordReader

RealtimeFileStatus - Class in org.apache.hudi.hadoop

With the base input format implementations in Hadoop/Hive, we need to encode additional information in Path to track base files and logs files for realtime read.

RealtimeFileStatus(FileStatus, String, List<HoodieLogFile>, boolean, Option<HoodieVirtualKeyInfo>) - Constructor for class org.apache.hudi.hadoop.RealtimeFileStatus

RealtimeSplit - Interface in org.apache.hudi.hadoop.realtime

Realtime Input Split Interface.

RecordReaderValueIterator<K,V> - Class in org.apache.hudi.hadoop

Provides Iterator Interface to iterate value entries read from record reader.

RecordReaderValueIterator(RecordReader<K, V>) - Constructor for class org.apache.hudi.hadoop.RecordReaderValueIterator

Construct RecordReaderValueIterator.

refreshFileStatus(Configuration, HoodieBaseFile) - Static method in class org.apache.hudi.hadoop.utils.HoodieInputFormatUtils

Checks the file status for a race condition which can set the file size to 0.

requiredProjectionFieldsExistInConf(Configuration, Option<HoodieVirtualKeyInfo>) - Static method in class org.apache.hudi.hadoop.utils.HoodieRealtimeInputFormatUtils

S

SafeParquetRecordReaderWrapper - Class in org.apache.hudi.hadoop

Record Reader for parquet.

SafeParquetRecordReaderWrapper(RecordReader<NullWritable, ArrayWritable>) - Constructor for class org.apache.hudi.hadoop.SafeParquetRecordReaderWrapper

setBasePath(String) - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeBootstrapBaseFileSplit

setBasePath(String) - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit

setBasePath(String) - Method in interface org.apache.hudi.hadoop.realtime.RealtimeSplit

Set Base Path.

setBelongsToIncrementalQuery(boolean) - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeBootstrapBaseFileSplit

setBelongsToIncrementalQuery(boolean) - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit

setBelongsToIncrementalQuery(boolean) - Method in interface org.apache.hudi.hadoop.realtime.RealtimeSplit

Sets the flag whether this split belongs to an Incremental Query

setBootStrapFileStatus(FileStatus) - Method in class org.apache.hudi.hadoop.RealtimeFileStatus

setConf(Configuration) - Method in class org.apache.hudi.hadoop.HoodieParquetInputFormatBase

setConf(Configuration) - Method in class org.apache.hudi.hadoop.HoodieROTablePathFilter

setConf(Configuration) - Method in class org.apache.hudi.hadoop.HoodieTableInputFormat

setDeltaLogFiles(List<HoodieLogFile>) - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeBootstrapBaseFileSplit

setDeltaLogFiles(List<HoodieLogFile>) - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit

setDeltaLogFiles(List<HoodieLogFile>) - Method in interface org.apache.hudi.hadoop.realtime.RealtimeSplit

setHoodieFilter(boolean) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.HoodieCombineFileInputFormatShim

setInputFormatClassName(String) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

setMaxCommitTime(String) - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeBootstrapBaseFileSplit

setMaxCommitTime(String) - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit

setMaxCommitTime(String) - Method in interface org.apache.hudi.hadoop.realtime.RealtimeSplit

Update Maximum valid instant time.

setMaxCommitTime(String) - Method in class org.apache.hudi.hadoop.RealtimeFileStatus

setPathWithBootstrapFileStatus(PathWithBootstrapFileStatus) - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimePath

setRealTime(boolean) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.HoodieCombineFileInputFormatShim

setVirtualKeyInfo(Option<HoodieVirtualKeyInfo>) - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeBootstrapBaseFileSplit

setVirtualKeyInfo(Option<HoodieVirtualKeyInfo>) - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit

setVirtualKeyInfo(Option<HoodieVirtualKeyInfo>) - Method in interface org.apache.hudi.hadoop.realtime.RealtimeSplit

shouldIncludePendingCommits(JobConf, String) - Static method in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

shouldSkipCombine(Path, Configuration) - Method in interface org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.AvoidSplitCombination

SNAPSHOT_SCAN_MODE - Static variable in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

SPILLABLE_MAP_BASE_PATH_PROP - Static variable in class org.apache.hudi.hadoop.config.HoodieRealtimeConfig

split - Variable in class org.apache.hudi.hadoop.realtime.AbstractRealtimeRecordReader

stopAtCompaction(JobContext, String) - Static method in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

T

toString() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

Prints this obejct as a string.

toString() - Method in class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit

toString() - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit

toString() - Method in class org.apache.hudi.hadoop.realtime.HoodieVirtualKeyInfo

U

UseFileSplitsFromInputFormat - Annotation Type in org.apache.hudi.hadoop

When annotated on a InputFormat, informs the query engines, that they should use the FileSplits provided by the input format to execute the queries.

UseRecordReaderFromInputFormat - Annotation Type in org.apache.hudi.hadoop

When annotated on a InputFormat, informs the query engines, that they should use the RecordReader provided by the input format to execute the queries.

usesCustomPayload - Variable in class org.apache.hudi.hadoop.realtime.AbstractRealtimeRecordReader

V

VIRTUAL_COLUMN_NAMES - Static variable in class org.apache.hudi.hadoop.utils.HoodieHiveUtils

W

WrapperFileSplit() - Constructor for class org.apache.hudi.hadoop.BootstrapBaseFileSplit.WrapperFileSplit

write(DataOutput) - Method in class org.apache.hudi.hadoop.BootstrapBaseFileSplit

write(DataOutput) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineHiveInputFormat.CombineHiveInputSplit

Writable interface.

write(DataOutput) - Method in class org.apache.hudi.hadoop.hive.HoodieCombineRealtimeFileSplit

Writable interface.

write(DataOutput) - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeBootstrapBaseFileSplit

write(DataOutput) - Method in class org.apache.hudi.hadoop.realtime.HoodieRealtimeFileSplit

writeBoolean(Boolean, DataOutput) - Static method in class org.apache.hudi.hadoop.InputSplitUtils

writeString(String, DataOutput) - Static method in class org.apache.hudi.hadoop.InputSplitUtils

writeToOutput(DataOutput) - Method in interface org.apache.hudi.hadoop.realtime.RealtimeSplit

A B C D F G H I J L M N O P R S T U V W