Index (hudi-integ-test 0.6.0 API)

A B C D E F G H I L M N O P R S T U V W

A

addChildNode(DagNode) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
addParentNode(DagNode) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
adjustRDDToGenerateExactNumUpdates(JavaRDD<GenericRecord>, JavaSparkContext, long) - Method in class org.apache.hudi.integ.testsuite.generator.DeltaGenerator
analyzeSingleFile(String) - Method in class org.apache.hudi.integ.testsuite.reader.DFSAvroDeltaInputReader
analyzeSingleFile(String) - Method in class org.apache.hudi.integ.testsuite.reader.DFSDeltaInputReader: Implementation of DeltaInputReaders to provide a way to read a single file on DFS and provide an average number of records across N files.
analyzeSingleFile(String) - Method in class org.apache.hudi.integ.testsuite.reader.DFSHoodieDatasetInputReader
AVRO_EXTENSION - Static variable in class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
AvroFileDeltaInputWriter - Class in org.apache.hudi.integ.testsuite.writer: Implementation of DeltaInputWriter that writes avro records to the result file.
AvroFileDeltaInputWriter(Configuration, String, String, Long) - Constructor for class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter

B

build() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
build() - Method in class org.apache.hudi.integ.testsuite.dag.SimpleWorkflowDagGenerator
build() - Method in interface org.apache.hudi.integ.testsuite.dag.WorkflowDagGenerator: Builds a WorkflowDag.
Builder() - Constructor for class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
bulkInsert() - Method in class org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper
bulkInsert(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
BulkInsertNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
BulkInsertNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.BulkInsertNode

C

canWrite() - Method in class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
canWrite() - Method in interface org.apache.hudi.integ.testsuite.writer.DeltaInputWriter: Check whether more data can/should be written.
canWrite() - Method in class org.apache.hudi.integ.testsuite.writer.SparkAvroDeltaInputWriter
cfg - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
childNodes - Variable in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
CHILDREN - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
CleanNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
CleanNode() - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.CleanNode
close() - Method in class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
close() - Method in class org.apache.hudi.integ.testsuite.writer.SparkAvroDeltaInputWriter
commit(JavaRDD<WriteStatus>, Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
compact() - Method in class org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper
compact(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
CompactNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
CompactNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.CompactNode
compareTo(DagNode<O>) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
computeNext() - Method in class org.apache.hudi.integ.testsuite.generator.LazyRecordGeneratorIterator
Config(Map<String, Object>) - Constructor for class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
config - Variable in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
CONFIG_NAME - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
convert(JavaRDD<I>) - Method in interface org.apache.hudi.integ.testsuite.converter.Converter
convert(JavaRDD<GenericRecord>) - Method in class org.apache.hudi.integ.testsuite.converter.UpdateConverter
convert(Schema) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
convert(Schema) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordPartialPayloadGenerator
convertDagToYaml(WorkflowDag) - Static method in class org.apache.hudi.integ.testsuite.dag.DagUtils: Converts WorkflowDag to a YAML representation.
Converter<I,O> - Interface in org.apache.hudi.integ.testsuite.converter: Implementations of Converter will convert data from one format to another.
convertPartial(Schema) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
convertYamlPathToDag(FileSystem, String) - Static method in class org.apache.hudi.integ.testsuite.dag.DagUtils: Converts a YAML path to WorkflowDag.
convertYamlToDag(String) - Static method in class org.apache.hudi.integ.testsuite.dag.DagUtils: Converts a YAML representation to WorkflowDag.

D

DagNode<O> - Class in org.apache.hudi.integ.testsuite.dag.nodes: Represents a Node in the DAG of operations for a workflow.
DagNode() - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
DagScheduler - Class in org.apache.hudi.integ.testsuite.dag.scheduler
DagScheduler(WorkflowDag, HoodieTestSuiteWriter, DeltaGenerator) - Constructor for class org.apache.hudi.integ.testsuite.dag.scheduler.DagScheduler
DagUtils - Class in org.apache.hudi.integ.testsuite.dag: Utility class to SerDe workflow dag.
DagUtils() - Constructor for class org.apache.hudi.integ.testsuite.dag.DagUtils
DEFAULT_PAYLOAD_SIZE - Static variable in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
DeltaConfig - Class in org.apache.hudi.integ.testsuite.configuration: Configuration to hold the delta output type and delta input format.
DeltaConfig(DeltaOutputMode, DeltaInputType, SerializableConfiguration) - Constructor for class org.apache.hudi.integ.testsuite.configuration.DeltaConfig
DeltaConfig.Config - Class in org.apache.hudi.integ.testsuite.configuration: Represents any kind of workload operation for new data.
DeltaConfig.Config.Builder - Class in org.apache.hudi.integ.testsuite.configuration
DeltaGenerator - Class in org.apache.hudi.integ.testsuite.generator: The delta generator generates all types of workloads (insert, update) for the given configs.
DeltaGenerator(DeltaConfig, JavaSparkContext, SparkSession, String, BuiltinKeyGenerator) - Constructor for class org.apache.hudi.integ.testsuite.generator.DeltaGenerator
DeltaInputReader<O> - Interface in org.apache.hudi.integ.testsuite.reader: Implementations of DeltaInputReader will read the configured input type and provide an RDD of records to the client.
DeltaInputType - Enum in org.apache.hudi.integ.testsuite.reader: Supported delta input data types.
DeltaInputWriter<I> - Interface in org.apache.hudi.integ.testsuite.writer: Implementations of DeltaInputWriter will be able to generate data.
DeltaOutputMode - Enum in org.apache.hudi.integ.testsuite.writer: Supported output destination types for the generated delta workload.
DeltaWriterAdapter<I> - Interface in org.apache.hudi.integ.testsuite.writer
DeltaWriterAdapter.SparkBasedDeltaWriter<J> - Interface in org.apache.hudi.integ.testsuite.writer
DeltaWriterFactory - Class in org.apache.hudi.integ.testsuite.writer: A factory to help instantiate different DeltaWriterAdapters depending on the DeltaOutputMode and DeltaInputType.
DeltaWriteStats - Class in org.apache.hudi.integ.testsuite.writer: This class holds the write statistics for DeltaInputWriter.
DeltaWriteStats() - Constructor for class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
DEPENDENCIES - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
DFSAvroDeltaInputReader - Class in org.apache.hudi.integ.testsuite.reader: A reader of DeltaOutputMode.DFS and DeltaInputType.AVRO.
DFSAvroDeltaInputReader(SparkSession, String, String, Option<String>, Option<String>) - Constructor for class org.apache.hudi.integ.testsuite.reader.DFSAvroDeltaInputReader
DFSDeltaConfig - Class in org.apache.hudi.integ.testsuite.configuration: Configuration to hold details about a DFS based output type, implements DeltaConfig.
DFSDeltaConfig(DeltaOutputMode, DeltaInputType, SerializableConfiguration, String, String, String, Long) - Constructor for class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
DFSDeltaInputReader - Class in org.apache.hudi.integ.testsuite.reader: This class helps to estimate the number of files to read a given number of total records.
DFSDeltaInputReader() - Constructor for class org.apache.hudi.integ.testsuite.reader.DFSDeltaInputReader
DFSDeltaWriterAdapter - Class in org.apache.hudi.integ.testsuite.writer: DistributedFileSystem (or LocalFileSystem) based delta generator.
DFSDeltaWriterAdapter(DeltaInputWriter<GenericRecord>) - Constructor for class org.apache.hudi.integ.testsuite.writer.DFSDeltaWriterAdapter
DFSHoodieDatasetInputReader - Class in org.apache.hudi.integ.testsuite.reader: This class helps to generate updates from an already existing hoodie dataset.
DFSHoodieDatasetInputReader(JavaSparkContext, String, String) - Constructor for class org.apache.hudi.integ.testsuite.reader.DFSHoodieDatasetInputReader
DFSSparkAvroDeltaWriter - Class in org.apache.hudi.integ.testsuite.writer: NEED TO IMPLEMENT A CUSTOM SPARK PARTITIONER TO ENSURE WE WRITE LARGE ENOUGH AVRO FILES.
DFSSparkAvroDeltaWriter(DeltaInputWriter<JavaRDD<GenericRecord>>) - Constructor for class org.apache.hudi.integ.testsuite.writer.DFSSparkAvroDeltaWriter
DFSTestSuitePathSelector - Class in org.apache.hudi.integ.testsuite.helpers: A custom dfs path selector used only for the hudi test suite.
DFSTestSuitePathSelector(TypedProperties, Configuration) - Constructor for class org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector
disableGenerate(boolean) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
disableIngest(boolean) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder

E

end() - Method in class org.apache.hudi.integ.testsuite.generator.LazyRecordGeneratorIterator
equals(Object) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
estimate(Schema) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadSizeEstimator: This method estimates the size of the payload if all entries of this payload were populated with one value.
execute(ExecutionContext) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.CleanNode
execute(ExecutionContext) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.CompactNode
execute(ExecutionContext) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
execute(ExecutionContext) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.HiveQueryNode
execute(ExecutionContext) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.HiveSyncNode
execute(ExecutionContext) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.InsertNode
execute(ExecutionContext) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.RollbackNode
execute(ExecutionContext) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.ScheduleCompactNode
execute(ExecutionContext) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.SparkSQLQueryNode
execute(ExecutionContext) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.ValidateNode
ExecutionContext - Class in org.apache.hudi.integ.testsuite.dag: This wraps the context needed for an execution of a DagNode.execute(ExecutionContext).
ExecutionContext(JavaSparkContext, HoodieTestSuiteWriter, DeltaGenerator) - Constructor for class org.apache.hudi.integ.testsuite.dag.ExecutionContext

F

fetchSource() - Method in class org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper
fetchSource() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
filter - Variable in class org.apache.hudi.integ.testsuite.reader.DFSAvroDeltaInputReader
FlexibleSchemaRecordGenerationIterator - Class in org.apache.hudi.integ.testsuite.generator: A GenericRecordGeneratorIterator for the custom schema of the workload.
FlexibleSchemaRecordGenerationIterator(long, String) - Constructor for class org.apache.hudi.integ.testsuite.generator.FlexibleSchemaRecordGenerationIterator
FlexibleSchemaRecordGenerationIterator(long, int, String, List<String>) - Constructor for class org.apache.hudi.integ.testsuite.generator.FlexibleSchemaRecordGenerationIterator
function - Variable in class org.apache.hudi.integ.testsuite.dag.nodes.ValidateNode

G

generate(DeltaGenerator) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.InsertNode
generate(DeltaGenerator) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.UpsertNode
generateInserts(DeltaConfig.Config) - Method in class org.apache.hudi.integ.testsuite.generator.DeltaGenerator
generateUpdates(DeltaConfig.Config) - Method in class org.apache.hudi.integ.testsuite.generator.DeltaGenerator
GenericRecordFullPayloadGenerator - Class in org.apache.hudi.integ.testsuite.generator: This is a GenericRecord payload generator that generates full generic records GenericRecord.
GenericRecordFullPayloadGenerator(Schema) - Constructor for class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
GenericRecordFullPayloadGenerator(Schema, int) - Constructor for class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
GenericRecordFullPayloadSizeEstimator - Class in org.apache.hudi.integ.testsuite.generator: This is a GenericRecord payload estimator estimates the size of a full generic record GenericRecord.
GenericRecordFullPayloadSizeEstimator(Schema) - Constructor for class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadSizeEstimator
GenericRecordPartialPayloadGenerator - Class in org.apache.hudi.integ.testsuite.generator: This is a GenericRecord payload generator that generates partial generic records GenericRecord.
GenericRecordPartialPayloadGenerator(Schema) - Constructor for class org.apache.hudi.integ.testsuite.generator.GenericRecordPartialPayloadGenerator
GenericRecordPartialPayloadGenerator(Schema, int) - Constructor for class org.apache.hudi.integ.testsuite.generator.GenericRecordPartialPayloadGenerator
getAdjustedPartitionsCount(Map<Integer, Long>, long) - Method in class org.apache.hudi.integ.testsuite.generator.DeltaGenerator
getBatchId() - Method in class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
getBytesWritten() - Method in class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
getCfg() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
getChildNodes() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
getConfig() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
getConfiguration() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig
getConfiguration() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
getDatasetOutputPath() - Method in class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
getDeltaBasePath() - Method in class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
getDeltaGenerator() - Method in class org.apache.hudi.integ.testsuite.dag.ExecutionContext
getDeltaInputType() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig
getDeltaOutputMode() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig
getDeltaStreamerWrapper() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
getDeltaWriterAdapter(DeltaConfig, Integer) - Static method in class org.apache.hudi.integ.testsuite.writer.DeltaWriterFactory
getDeltaWriteStats() - Method in class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
getDeltaWriteStats() - Method in interface org.apache.hudi.integ.testsuite.writer.DeltaInputWriter: Return the statistics of data written.
getDeltaWriteStats() - Method in class org.apache.hudi.integ.testsuite.writer.SparkAvroDeltaInputWriter
getEstimatedFullPayloadSize() - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
getFilePath() - Method in class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
getFilePathsToRead(String, PathFilter, long) - Method in class org.apache.hudi.integ.testsuite.reader.DFSDeltaInputReader
getFileStatusIndexRange(List<FileStatus>, long, long) - Method in class org.apache.hudi.integ.testsuite.reader.DFSDeltaInputReader
getFractionUpsertPerFile() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
getFs() - Method in class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
getHiveProperties() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
getHiveQueries() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
getHiveServiceProvider() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.HiveSyncNode
getHoodieTestSuiteWriter() - Method in class org.apache.hudi.integ.testsuite.dag.ExecutionContext
getJsc() - Method in class org.apache.hudi.integ.testsuite.dag.ExecutionContext
getLocalHiveServer() - Method in class org.apache.hudi.integ.testsuite.helpers.HiveServiceProvider
getMaxFileSize() - Method in class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
getName() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
getNewPayload() - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
getNewWriter() - Method in class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
getNewWriter() - Method in interface org.apache.hudi.integ.testsuite.writer.DeltaInputWriter: Return a new instance of this writer.
getNewWriter() - Method in class org.apache.hudi.integ.testsuite.writer.SparkAvroDeltaInputWriter
getNextFilePathsAndMaxModificationTime(Option<String>, long) - Method in class org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector
getNodeList() - Method in class org.apache.hudi.integ.testsuite.dag.WorkflowDag
getNonNull(Schema) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
getNonNull(Schema) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadSizeEstimator
getNumInsertPartitions() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
getNumRecordsInsert() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
getNumRecordsUpsert() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
getNumUpsertFiles() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
getNumUpsertPartitions() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
getOtherConfigs() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
getParentNodes() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
getPartitionPathRecordKey() - Method in class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
getPartitions(Option<Integer>) - Method in class org.apache.hudi.integ.testsuite.reader.DFSHoodieDatasetInputReader
getPartitionToCountMap(JavaRDD<GenericRecord>) - Method in class org.apache.hudi.integ.testsuite.generator.DeltaGenerator
getPath() - Method in class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
getRecordSize() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
getRecordsWritten() - Method in class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
getRepeatCount() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
getResult() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
getSchemaStr() - Method in class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
getSparkContext() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
getUpdatePayload(GenericRecord, List<String>) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
getWriteClient(DagNode) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter

H

hashCode() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
hasNext() - Method in class org.apache.hudi.integ.testsuite.generator.FlexibleSchemaRecordGenerationIterator
hasNext() - Method in class org.apache.hudi.integ.testsuite.generator.UpdateGeneratorIterator
HIVE_PROPERTIES - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
HIVE_QUERIES - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
HiveQueryNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
HiveQueryNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.HiveQueryNode
HiveServiceProvider - Class in org.apache.hudi.integ.testsuite.helpers: Hive Service provider.
HiveServiceProvider(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.helpers.HiveServiceProvider
HiveSyncNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
HiveSyncNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.HiveSyncNode
HoodieDeltaStreamerWrapper - Class in org.apache.hudi.integ.testsuite: Extends the HoodieDeltaStreamer to expose certain operations helpful in running the Test Suite.
HoodieDeltaStreamerWrapper(HoodieDeltaStreamer.Config, JavaSparkContext) - Constructor for class org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper
HoodieTestSuiteConfig() - Constructor for class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
HoodieTestSuiteJob - Class in org.apache.hudi.integ.testsuite: This is the entry point for running a Hudi Test Suite.
HoodieTestSuiteJob(HoodieTestSuiteJob.HoodieTestSuiteConfig, JavaSparkContext) - Constructor for class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob
HoodieTestSuiteJob.HoodieTestSuiteConfig - Class in org.apache.hudi.integ.testsuite: The Hudi test suite uses HoodieDeltaStreamer to run some operations hence extend delta streamer config.
HoodieTestSuiteWriter - Class in org.apache.hudi.integ.testsuite: A writer abstraction for the Hudi test suite.
HoodieTestSuiteWriter(JavaSparkContext, Properties, HoodieTestSuiteJob.HoodieTestSuiteConfig, String) - Constructor for class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
HoodieTestSuiteWriter(JavaSparkContext, Properties, HoodieTestSuiteJob.HoodieTestSuiteConfig, String, boolean) - Constructor for class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter

I

ingest(HoodieTestSuiteWriter, Option<String>) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.BulkInsertNode
ingest(HoodieTestSuiteWriter, Option<String>) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.InsertNode
ingest(HoodieTestSuiteWriter, Option<String>) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.UpsertNode
inputBasePath - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
inputFormatName - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
insert() - Method in class org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper
insert(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
InsertNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
InsertNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.InsertNode
isCompleted() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
isDisableGenerate() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
isDisableIngest() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
isHiveLocal() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
isOption(Schema) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
isOption(Schema) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadSizeEstimator
isPrimitive(Schema) - Static method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator

L

LazyRecordGeneratorIterator - Class in org.apache.hudi.integ.testsuite.generator: A lazy record generator to generate GenericRecords lazily and not hold a list of records in memory.
LazyRecordGeneratorIterator(Iterator<GenericRecord>) - Constructor for class org.apache.hudi.integ.testsuite.generator.LazyRecordGeneratorIterator
limitFileSize - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
log - Static variable in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode

M

main(String[]) - Static method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob

N

newBuilder() - Static method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
next() - Method in class org.apache.hudi.integ.testsuite.generator.FlexibleSchemaRecordGenerationIterator
next() - Method in class org.apache.hudi.integ.testsuite.generator.UpdateGeneratorIterator
NODE_NAME - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config

O

org.apache.hudi.integ.testsuite - package org.apache.hudi.integ.testsuite
org.apache.hudi.integ.testsuite.configuration - package org.apache.hudi.integ.testsuite.configuration
org.apache.hudi.integ.testsuite.converter - package org.apache.hudi.integ.testsuite.converter
org.apache.hudi.integ.testsuite.dag - package org.apache.hudi.integ.testsuite.dag
org.apache.hudi.integ.testsuite.dag.nodes - package org.apache.hudi.integ.testsuite.dag.nodes
org.apache.hudi.integ.testsuite.dag.scheduler - package org.apache.hudi.integ.testsuite.dag.scheduler
org.apache.hudi.integ.testsuite.generator - package org.apache.hudi.integ.testsuite.generator
org.apache.hudi.integ.testsuite.helpers - package org.apache.hudi.integ.testsuite.helpers
org.apache.hudi.integ.testsuite.reader - package org.apache.hudi.integ.testsuite.reader
org.apache.hudi.integ.testsuite.writer - package org.apache.hudi.integ.testsuite.writer
outputTypeName - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig

P

parentNodes - Variable in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode

R

random - Variable in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
randomize(GenericRecord, List<String>) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
read(long) - Method in interface org.apache.hudi.integ.testsuite.reader.DeltaInputReader: Attempts to reads an approximate number of records close to approxNumRecords.
read(int, long) - Method in interface org.apache.hudi.integ.testsuite.reader.DeltaInputReader
read(int, int, long) - Method in interface org.apache.hudi.integ.testsuite.reader.DeltaInputReader
read(int, int, double) - Method in interface org.apache.hudi.integ.testsuite.reader.DeltaInputReader
read(long) - Method in class org.apache.hudi.integ.testsuite.reader.DFSAvroDeltaInputReader
read(int, long) - Method in class org.apache.hudi.integ.testsuite.reader.DFSAvroDeltaInputReader
read(int, int, long) - Method in class org.apache.hudi.integ.testsuite.reader.DFSAvroDeltaInputReader
read(int, int, double) - Method in class org.apache.hudi.integ.testsuite.reader.DFSAvroDeltaInputReader
read(long) - Method in class org.apache.hudi.integ.testsuite.reader.DFSHoodieDatasetInputReader
read(int, long) - Method in class org.apache.hudi.integ.testsuite.reader.DFSHoodieDatasetInputReader
read(int, int, long) - Method in class org.apache.hudi.integ.testsuite.reader.DFSHoodieDatasetInputReader
read(int, int, double) - Method in class org.apache.hudi.integ.testsuite.reader.DFSHoodieDatasetInputReader
readAvro(SparkSession, String, List<String>, Option<String>, Option<String>) - Static method in class org.apache.hudi.integ.testsuite.reader.SparkBasedReader
readParquet(SparkSession, List<String>, Option<String>, Option<String>) - Static method in class org.apache.hudi.integ.testsuite.reader.SparkBasedReader
result - Variable in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
RollbackNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
RollbackNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.RollbackNode
rollOver() - Method in class org.apache.hudi.integ.testsuite.writer.DFSDeltaWriterAdapter
runTestSuite() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob

S

schedule() - Method in class org.apache.hudi.integ.testsuite.dag.scheduler.DagScheduler
scheduleCompact() - Method in class org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper
scheduleCompaction(Option<Map<String, String>>) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
ScheduleCompactNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
ScheduleCompactNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.ScheduleCompactNode
setBatchId(Integer) - Method in class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
setBytesWritten(long) - Method in class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
setCompleted(boolean) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
setFilePath(String) - Method in class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
setParentNodes(List<DagNode<O>>) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
setPartitionPathRecordKey(List<Pair<String, String>>) - Method in class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
setRecordsWritten(long) - Method in class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
SimpleWorkflowDagGenerator - Class in org.apache.hudi.integ.testsuite.dag: An example of how to generate a workflow dag programmatically.
SimpleWorkflowDagGenerator() - Constructor for class org.apache.hudi.integ.testsuite.dag.SimpleWorkflowDagGenerator
SPARK_AVRO_FORMAT - Static variable in class org.apache.hudi.integ.testsuite.reader.SparkBasedReader
SPARK_PARQUET_FORMAT - Static variable in class org.apache.hudi.integ.testsuite.reader.SparkBasedReader
SparkAvroDeltaInputWriter - Class in org.apache.hudi.integ.testsuite.writer: Spark based avro delta input writer.
SparkAvroDeltaInputWriter(SparkSession, String, String) - Constructor for class org.apache.hudi.integ.testsuite.writer.SparkAvroDeltaInputWriter
SparkBasedReader - Class in org.apache.hudi.integ.testsuite.reader: Helper class to read avro and/or parquet files and generate a RDD of GenericRecord.
SparkBasedReader() - Constructor for class org.apache.hudi.integ.testsuite.reader.SparkBasedReader
sparkSession - Variable in class org.apache.hudi.integ.testsuite.writer.SparkAvroDeltaInputWriter
SparkSQLQueryNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
SparkSQLQueryNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.SparkSQLQueryNode
start() - Method in class org.apache.hudi.integ.testsuite.generator.LazyRecordGeneratorIterator
startCommit() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
startLocalHiveServiceIfNeeded(Configuration) - Method in class org.apache.hudi.integ.testsuite.helpers.HiveServiceProvider
stopLocalHiveServiceIfNeeded() - Method in class org.apache.hudi.integ.testsuite.helpers.HiveServiceProvider
syncToLocalHiveIfNeeded(HoodieTestSuiteWriter) - Method in class org.apache.hudi.integ.testsuite.helpers.HiveServiceProvider

T

toString() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
toString(InputStream) - Static method in class org.apache.hudi.integ.testsuite.dag.DagUtils
TYPE - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
typeEstimateAndNumComplexFields() - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadSizeEstimator

U

UpdateConverter - Class in org.apache.hudi.integ.testsuite.converter: This converter creates an update GenericRecord from an existing GenericRecord.
UpdateConverter(String, int, List<String>, List<String>) - Constructor for class org.apache.hudi.integ.testsuite.converter.UpdateConverter
UpdateGeneratorIterator - Class in org.apache.hudi.integ.testsuite.generator
UpdateGeneratorIterator(Iterator<GenericRecord>, String, List<String>, List<String>, int) - Constructor for class org.apache.hudi.integ.testsuite.generator.UpdateGeneratorIterator
upsert(HoodieDeltaStreamer.Operation) - Method in class org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper
upsert(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
UpsertNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
UpsertNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.UpsertNode
useDeltaStreamer - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig

V

validate(GenericRecord) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
validate(GenericRecord) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordPartialPayloadGenerator
ValidateNode<R> - Class in org.apache.hudi.integ.testsuite.dag.nodes
ValidateNode(DeltaConfig.Config, Function<List<DagNode>, R>) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.ValidateNode
valueOf(String) - Static method in enum org.apache.hudi.integ.testsuite.reader.DeltaInputType: Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum org.apache.hudi.integ.testsuite.writer.DeltaOutputMode: Returns the enum constant of this type with the specified name.
values() - Static method in enum org.apache.hudi.integ.testsuite.reader.DeltaInputType: Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum org.apache.hudi.integ.testsuite.writer.DeltaOutputMode: Returns an array containing the constants of this enum type, in the order they are declared.

W

withConfig(String, Object) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
withConfigsMap(Map<String, Object>) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
withFractionUpsertPerFile(double) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
withHiveLocal(boolean) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
withHiveProperties(List<String>) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
withHiveQueryAndResults(List<Pair<String, Integer>>) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
withNumInsertPartitions(int) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
withNumRecordsToInsert(long) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
withNumRecordsToUpdate(long) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
withNumTimesToRepeat(int) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
withNumUpsertFiles(int) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
withNumUpsertPartitions(int) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
withRecordSize(int) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
WorkflowDag<O> - Class in org.apache.hudi.integ.testsuite.dag: Workflow dag that encapsulates all execute nodes.
WorkflowDag(List<DagNode<O>>) - Constructor for class org.apache.hudi.integ.testsuite.dag.WorkflowDag
WorkflowDagGenerator - Interface in org.apache.hudi.integ.testsuite.dag: A interface which represents a workflow dag generator.
workloadDagGenerator - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
workloadYamlPath - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
write(J) - Method in interface org.apache.hudi.integ.testsuite.writer.DeltaWriterAdapter.SparkBasedDeltaWriter
write(Iterator<I>) - Method in interface org.apache.hudi.integ.testsuite.writer.DeltaWriterAdapter
write(Iterator<GenericRecord>) - Method in class org.apache.hudi.integ.testsuite.writer.DFSDeltaWriterAdapter
write(JavaRDD<GenericRecord>) - Method in class org.apache.hudi.integ.testsuite.writer.DFSSparkAvroDeltaWriter
writeData(GenericRecord) - Method in class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
writeData(I) - Method in interface org.apache.hudi.integ.testsuite.writer.DeltaInputWriter: Generate any type of data.
writeData(JavaRDD<GenericRecord>) - Method in class org.apache.hudi.integ.testsuite.writer.SparkAvroDeltaInputWriter
writeRecords(JavaRDD<GenericRecord>) - Method in class org.apache.hudi.integ.testsuite.generator.DeltaGenerator

A B C D E F G H I L M N O P R S T U V W