Skip navigation links
A B C D E F G H I L M N O P R S T U V W Z 

A

addChildNode(DagNode) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
addParentNode(DagNode) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
adjustRDDToGenerateExactNumUpdates(JavaRDD<GenericRecord>, JavaSparkContext, long) - Method in class org.apache.hudi.integ.testsuite.generator.DeltaGenerator
 
allOfTerminateOnFailure(List<CompletableFuture<Boolean>>) - Static method in class org.apache.hudi.integ.testsuite.HoodieMultiWriterTestSuiteJob
 
analyzeSingleFile(String) - Method in class org.apache.hudi.integ.testsuite.reader.DFSAvroDeltaInputReader
 
analyzeSingleFile(String) - Method in class org.apache.hudi.integ.testsuite.reader.DFSDeltaInputReader
Implementation of DeltaInputReaders to provide a way to read a single file on DFS and provide an average number of records across N files.
analyzeSingleFile(String) - Method in class org.apache.hudi.integ.testsuite.reader.DFSHoodieDatasetInputReader
 
AVRO_EXTENSION - Static variable in class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
 
AvroFileDeltaInputWriter - Class in org.apache.hudi.integ.testsuite.writer
Implementation of DeltaInputWriter that writes avro records to the result file.
AvroFileDeltaInputWriter(Configuration, String, String, Long) - Constructor for class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
 

B

basePath - Variable in class org.apache.hudi.integ.testsuite.SparkDataSourceContinuousIngestTool.Config
 
BaseQueryNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
 
BaseQueryNode() - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.BaseQueryNode
 
BaseValidateDatasetNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
This nodes validates contents from input path are in tact with Hudi.
BaseValidateDatasetNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.BaseValidateDatasetNode
 
build() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
build() - Method in class org.apache.hudi.integ.testsuite.dag.SimpleWorkflowDagGenerator
 
build() - Method in interface org.apache.hudi.integ.testsuite.dag.WorkflowDagGenerator
Builds a WorkflowDag.
Builder() - Constructor for class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
bulkInsert(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieContinousTestSuiteWriter
 
bulkInsert() - Method in class org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper
 
bulkInsert(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieInlineTestSuiteWriter
 
bulkInsert(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
BulkInsertNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
Represents a bulk insert node in the DAG of operations for a workflow.
BulkInsertNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.BulkInsertNode
 

C

canWrite() - Method in class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
 
canWrite() - Method in interface org.apache.hudi.integ.testsuite.writer.DeltaInputWriter
Check whether more data can/should be written.
canWrite() - Method in class org.apache.hudi.integ.testsuite.writer.SparkAvroDeltaInputWriter
 
cfg - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
checkpointFilePath - Variable in class org.apache.hudi.integ.testsuite.SparkDataSourceContinuousIngestTool.Config
 
childNodes - Variable in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
CHILDREN - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
cleanInput - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
CleanNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
Represents a clean node in the DAG of operations for a workflow.
CleanNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.CleanNode
 
cleanOutput - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
clone() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
close() - Method in class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
 
close() - Method in class org.apache.hudi.integ.testsuite.writer.SparkAvroDeltaInputWriter
 
ClusteringNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
Triggers inline clustering.
ClusteringNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.ClusteringNode
 
commit(JavaRDD<WriteStatus>, JavaRDD<DeltaWriteStats>, Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieContinousTestSuiteWriter
 
commit(JavaRDD<WriteStatus>, JavaRDD<DeltaWriteStats>, Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieInlineTestSuiteWriter
 
commit(JavaRDD<WriteStatus>, JavaRDD<DeltaWriteStats>, Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
commitCompaction(JavaRDD<WriteStatus>, JavaRDD<DeltaWriteStats>, Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieContinousTestSuiteWriter
 
commitCompaction(JavaRDD<WriteStatus>, JavaRDD<DeltaWriteStats>, Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieInlineTestSuiteWriter
 
commitCompaction(JavaRDD<WriteStatus>, JavaRDD<DeltaWriteStats>, Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
compact(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieContinousTestSuiteWriter
 
compact() - Method in class org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper
 
compact(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieInlineTestSuiteWriter
 
compact(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
CompactNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
Represents a compact node in the DAG of operations for a workflow.
CompactNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.CompactNode
 
compareTo(DagNode<O>) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
computeNext() - Method in class org.apache.hudi.integ.testsuite.generator.LazyRecordGeneratorIterator
 
Config(Map<String, Object>) - Constructor for class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
config - Variable in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
Config() - Constructor for class org.apache.hudi.integ.testsuite.SparkDataSourceContinuousIngestTool.Config
 
CONFIG_NAME - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
configs - Variable in class org.apache.hudi.integ.testsuite.SparkDataSourceContinuousIngestTool.Config
 
configuration - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
convert(JavaRDD<I>) - Method in interface org.apache.hudi.integ.testsuite.converter.Converter
Convert data from one format to another.
convert(JavaRDD<GenericRecord>) - Method in class org.apache.hudi.integ.testsuite.converter.DeleteConverter
 
convert(JavaRDD<GenericRecord>) - Method in class org.apache.hudi.integ.testsuite.converter.UpdateConverter
 
convertDagToYaml(WorkflowDag) - Static method in class org.apache.hudi.integ.testsuite.dag.DagUtils
Converts WorkflowDag to a YAML representation.
Converter<I,O> - Interface in org.apache.hudi.integ.testsuite.converter
Implementations of Converter will convert data from one format to another.
convertPartial(Schema) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
Create a new GenericRecord with random values.
convertYamlPathToDag(FileSystem, String) - Static method in class org.apache.hudi.integ.testsuite.dag.DagUtils
Converts a YAML path to WorkflowDag.
convertYamlToDag(String) - Static method in class org.apache.hudi.integ.testsuite.dag.DagUtils
Converts a YAML representation to WorkflowDag.
create(Schema, Set<String>) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
 
createWorkflowDag() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob
 

D

DAG_CONTENT - Static variable in class org.apache.hudi.integ.testsuite.dag.DagUtils
 
DAG_INTERMITTENT_DELAY_MINS - Static variable in class org.apache.hudi.integ.testsuite.dag.DagUtils
 
DAG_NAME - Static variable in class org.apache.hudi.integ.testsuite.dag.DagUtils
 
DAG_ROUNDS - Static variable in class org.apache.hudi.integ.testsuite.dag.DagUtils
 
DagNode<O> - Class in org.apache.hudi.integ.testsuite.dag.nodes
Base abstraction of an compute node in the DAG of operations for a workflow.
DagNode() - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
DagScheduler - Class in org.apache.hudi.integ.testsuite.dag.scheduler
The Dag scheduler schedules the workflow DAGs.
DagScheduler(WorkflowDag, WriterContext, JavaSparkContext) - Constructor for class org.apache.hudi.integ.testsuite.dag.scheduler.DagScheduler
 
DagUtils - Class in org.apache.hudi.integ.testsuite.dag
Utility class to SerDe workflow dag.
DagUtils() - Constructor for class org.apache.hudi.integ.testsuite.dag.DagUtils
 
DEFAULT_DAG_NAME - Static variable in class org.apache.hudi.integ.testsuite.dag.DagUtils
 
DEFAULT_DAG_ROUNDS - Static variable in class org.apache.hudi.integ.testsuite.dag.DagUtils
 
DEFAULT_HOODIE_IS_DELETED_COL - Static variable in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
 
DEFAULT_INTERMITTENT_DELAY_MINS - Static variable in class org.apache.hudi.integ.testsuite.dag.DagUtils
 
DEFAULT_NUM_DATE_PARTITIONS - Static variable in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
 
DEFAULT_PAYLOAD_SIZE - Static variable in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
 
DEFAULT_START_PARTITION - Static variable in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
 
DelayNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
Delay Node to add delays between each group of test runs.
DelayNode(int) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.DelayNode
 
DeleteConverter - Class in org.apache.hudi.integ.testsuite.converter
 
DeleteConverter(String, int) - Constructor for class org.apache.hudi.integ.testsuite.converter.DeleteConverter
 
DeleteGeneratorIterator - Class in org.apache.hudi.integ.testsuite.generator
Lazy delete record generator.
DeleteGeneratorIterator(Iterator<GenericRecord>, String, int) - Constructor for class org.apache.hudi.integ.testsuite.generator.DeleteGeneratorIterator
 
DeleteInputDatasetNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
Deletes all input except latest batch.
DeleteInputDatasetNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.DeleteInputDatasetNode
 
DeleteNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
Delete node to assist in issuing deletes.
DeleteNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.DeleteNode
 
deleteOldInput - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
DeltaConfig - Class in org.apache.hudi.integ.testsuite.configuration
Configuration to hold the delta output type and delta input format.
DeltaConfig(DeltaOutputMode, DeltaInputType, SerializableConfiguration) - Constructor for class org.apache.hudi.integ.testsuite.configuration.DeltaConfig
 
DeltaConfig.Config - Class in org.apache.hudi.integ.testsuite.configuration
Represents any kind of workload operation for new data.
DeltaConfig.Config.Builder - Class in org.apache.hudi.integ.testsuite.configuration
 
DeltaGenerator - Class in org.apache.hudi.integ.testsuite.generator
The delta generator generates all types of workloads (insert, update) for the given configs.
DeltaGenerator(DFSDeltaConfig, JavaSparkContext, SparkSession, String, BuiltinKeyGenerator) - Constructor for class org.apache.hudi.integ.testsuite.generator.DeltaGenerator
 
DeltaInputReader<O> - Interface in org.apache.hudi.integ.testsuite.reader
Implementations of DeltaInputReader will read the configured input type and provide an RDD of records to the client.
DeltaInputType - Enum in org.apache.hudi.integ.testsuite.reader
Supported delta input data types.
DeltaInputWriter<I> - Interface in org.apache.hudi.integ.testsuite.writer
Implementations of DeltaInputWriter will be able to generate data.
DeltaOutputMode - Enum in org.apache.hudi.integ.testsuite.writer
Supported output destination types for the generated delta workload.
deltaStreamerWrapper - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
DeltaWriterAdapter<I> - Interface in org.apache.hudi.integ.testsuite.writer
Adapter use Delta Writer.
DeltaWriterAdapter.SparkBasedDeltaWriter<J> - Interface in org.apache.hudi.integ.testsuite.writer
 
DeltaWriterFactory - Class in org.apache.hudi.integ.testsuite.writer
A factory to help instantiate different DeltaWriterAdapters depending on the DeltaOutputMode and DeltaInputType.
DeltaWriteStats - Class in org.apache.hudi.integ.testsuite.writer
This class holds the write statistics for DeltaInputWriter.
DeltaWriteStats() - Constructor for class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
 
deltaWriteStatsRDD - Variable in class org.apache.hudi.integ.testsuite.dag.nodes.InsertNode
 
DEPENDENCIES - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
DFSAvroDeltaInputReader - Class in org.apache.hudi.integ.testsuite.reader
DFSAvroDeltaInputReader(SparkSession, String, String, Option<String>, Option<String>) - Constructor for class org.apache.hudi.integ.testsuite.reader.DFSAvroDeltaInputReader
 
DFSDeltaConfig - Class in org.apache.hudi.integ.testsuite.configuration
Configuration to hold details about a DFS based output type, implements DeltaConfig.
DFSDeltaConfig(DeltaOutputMode, DeltaInputType, SerializableConfiguration, String, String, String, Long, int, boolean, boolean) - Constructor for class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
 
DFSDeltaInputReader - Class in org.apache.hudi.integ.testsuite.reader
This class helps to estimate the number of files to read a given number of total records.
DFSDeltaInputReader() - Constructor for class org.apache.hudi.integ.testsuite.reader.DFSDeltaInputReader
 
DFSDeltaWriterAdapter - Class in org.apache.hudi.integ.testsuite.writer
DistributedFileSystem (or LocalFileSystem) based delta generator.
DFSDeltaWriterAdapter(DeltaInputWriter<GenericRecord>, int) - Constructor for class org.apache.hudi.integ.testsuite.writer.DFSDeltaWriterAdapter
 
DFSDeltaWriterAdapter(DeltaInputWriter<GenericRecord>) - Constructor for class org.apache.hudi.integ.testsuite.writer.DFSDeltaWriterAdapter
 
DFSHoodieDatasetInputReader - Class in org.apache.hudi.integ.testsuite.reader
This class helps to generate updates from an already existing hoodie dataset.
DFSHoodieDatasetInputReader(JavaSparkContext, String, String) - Constructor for class org.apache.hudi.integ.testsuite.reader.DFSHoodieDatasetInputReader
 
DFSSparkAvroDeltaWriter - Class in org.apache.hudi.integ.testsuite.writer
NEED TO IMPLEMENT A CUSTOM SPARK PARTITIONER TO ENSURE WE WRITE LARGE ENOUGH AVRO FILES.
DFSSparkAvroDeltaWriter(DeltaInputWriter<JavaRDD<GenericRecord>>) - Constructor for class org.apache.hudi.integ.testsuite.writer.DFSSparkAvroDeltaWriter
 
DFSTestSuitePathSelector - Class in org.apache.hudi.integ.testsuite.helpers
A custom dfs path selector used only for the hudi test suite.
DFSTestSuitePathSelector(TypedProperties, Configuration) - Constructor for class org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector
 
disableGenerate(boolean) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
disableIngest(boolean) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 

E

enableMetadataOnRead - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
enablePrestoValidation - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
enableRowWriting() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
end() - Method in class org.apache.hudi.integ.testsuite.generator.LazyRecordGeneratorIterator
 
equals(Object) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
estimate(Schema) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadSizeEstimator
This method estimates the size of the payload if all entries of this payload were populated with one value.
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.BaseValidateDatasetNode
 
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.CleanNode
 
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.ClusteringNode
 
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.CompactNode
Method helps to start the compact operation.
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
Execute the DagNode.
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DelayNode
 
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DeleteInputDatasetNode
 
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.HiveQueryNode
 
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.HiveSyncNode
 
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.InsertNode
 
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.PrestoQueryNode
 
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.RollbackNode
Method helps to rollback the last commit instant in the timeline, if it has one.
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.ScheduleCompactNode
 
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.SparkSQLQueryNode
Method helps to execute a sparkSql query from a hive table.
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.TrinoQueryNode
 
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.ValidateAsyncOperations
 
execute(ExecutionContext, int) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.ValidateNode
Method to start the validate operation.
executeAndValidateQueries(List<Pair<String, Integer>>, Statement) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.BaseQueryNode
 
executeNode(DagNode, int) - Method in class org.apache.hudi.integ.testsuite.dag.scheduler.DagScheduler
Execute the given node.
executeNode(DagNode, int) - Method in class org.apache.hudi.integ.testsuite.dag.scheduler.SaferSchemaDagScheduler
 
ExecutionContext - Class in org.apache.hudi.integ.testsuite.dag
This wraps the context needed for an execution of a DagNode.execute(ExecutionContext, int).
ExecutionContext(JavaSparkContext, WriterContext) - Constructor for class org.apache.hudi.integ.testsuite.dag.ExecutionContext
 

F

fetchSource() - Method in class org.apache.hudi.integ.testsuite.HoodieContinousTestSuiteWriter
 
fetchSource() - Method in class org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper
 
fetchSource() - Method in class org.apache.hudi.integ.testsuite.HoodieInlineTestSuiteWriter
 
fetchSource() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
filter - Variable in class org.apache.hudi.integ.testsuite.reader.DFSAvroDeltaInputReader
 
FlexibleSchemaRecordGenerationIterator - Class in org.apache.hudi.integ.testsuite.generator
A GenericRecordGeneratorIterator for the custom schema of the workload.
FlexibleSchemaRecordGenerationIterator(long, String) - Constructor for class org.apache.hudi.integ.testsuite.generator.FlexibleSchemaRecordGenerationIterator
 
FlexibleSchemaRecordGenerationIterator(long, int, String, List<String>, int, int) - Constructor for class org.apache.hudi.integ.testsuite.generator.FlexibleSchemaRecordGenerationIterator
 
function - Variable in class org.apache.hudi.integ.testsuite.dag.nodes.ValidateNode
 

G

generate(DeltaGenerator) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DeleteNode
 
generate(DeltaGenerator) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.InsertNode
 
generate(DeltaGenerator) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.UpsertNode
 
generateDeleteRecord(GenericRecord) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
Set _hoodie_is_deleted column value to true.
generateDeletes(DeltaConfig.Config) - Method in class org.apache.hudi.integ.testsuite.generator.DeltaGenerator
 
generateInserts(DeltaConfig.Config) - Method in class org.apache.hudi.integ.testsuite.generator.DeltaGenerator
 
generateUpdates(DeltaConfig.Config) - Method in class org.apache.hudi.integ.testsuite.generator.DeltaGenerator
 
GenericRecordFullPayloadGenerator - Class in org.apache.hudi.integ.testsuite.generator
This is a GenericRecord payload generator that generates full generic records GenericRecord.
GenericRecordFullPayloadGenerator(Schema) - Constructor for class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
 
GenericRecordFullPayloadGenerator(Schema, int, int, int) - Constructor for class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
 
GenericRecordFullPayloadGenerator(Schema, int) - Constructor for class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
 
GenericRecordFullPayloadSizeEstimator - Class in org.apache.hudi.integ.testsuite.generator
This is a GenericRecord payload estimator estimates the size of a full generic record GenericRecord.
GenericRecordFullPayloadSizeEstimator(Schema) - Constructor for class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadSizeEstimator
 
GenericRecordPartialPayloadGenerator - Class in org.apache.hudi.integ.testsuite.generator
This is a GenericRecord payload generator that generates partial generic records GenericRecord.
GenericRecordPartialPayloadGenerator(Schema) - Constructor for class org.apache.hudi.integ.testsuite.generator.GenericRecordPartialPayloadGenerator
 
GenericRecordPartialPayloadGenerator(Schema, int) - Constructor for class org.apache.hudi.integ.testsuite.generator.GenericRecordPartialPayloadGenerator
 
getAdjustedPartitionsCount(Map<Integer, Long>, long) - Method in class org.apache.hudi.integ.testsuite.generator.DeltaGenerator
 
getBatchId() - Method in class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
 
getBatchId() - Method in class org.apache.hudi.integ.testsuite.generator.DeltaGenerator
 
getBytesWritten() - Method in class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
 
getCfg() - Method in class org.apache.hudi.integ.testsuite.dag.WriterContext
 
getCfg() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
getChildNodes() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
getConfig() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
getConfiguration() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig
 
getConfiguration() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
getDagName() - Method in class org.apache.hudi.integ.testsuite.dag.WorkflowDag
 
getDatasetOutputPath() - Method in class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
 
getDatasetToValidate(SparkSession, ExecutionContext, StructType) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.BaseValidateDatasetNode
 
getDatasetToValidate(SparkSession, ExecutionContext, StructType) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.ValidateDatasetNode
 
getDeltaBasePath() - Method in class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
 
getDeltaGenerator() - Method in class org.apache.hudi.integ.testsuite.dag.ExecutionContext
 
getDeltaGenerator() - Method in class org.apache.hudi.integ.testsuite.dag.WriterContext
 
getDeltaInputType() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig
 
getDeltaOutputMode() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig
 
getDeltaStreamerWrapper() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
getDeltaWriterAdapter(DeltaConfig, Integer) - Static method in class org.apache.hudi.integ.testsuite.writer.DeltaWriterFactory
 
getDeltaWriteStats() - Method in class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
 
getDeltaWriteStats() - Method in interface org.apache.hudi.integ.testsuite.writer.DeltaInputWriter
Return the statistics of data written.
getDeltaWriteStats() - Method in class org.apache.hudi.integ.testsuite.writer.SparkAvroDeltaInputWriter
 
getEstimatedFullPayloadSize() - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
 
getFilePath() - Method in class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
 
getFilePathsToRead(String, PathFilter, long) - Method in class org.apache.hudi.integ.testsuite.reader.DFSDeltaInputReader
 
getFileStatusIndexRange(List<FileStatus>, long, long) - Method in class org.apache.hudi.integ.testsuite.reader.DFSDeltaInputReader
 
getFractionUpsertPerFile() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getFs() - Method in class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
 
getHiveProperties() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getHiveQueries() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getHoodieTestSuiteWriter() - Method in class org.apache.hudi.integ.testsuite.dag.ExecutionContext
 
getHoodieTestSuiteWriter() - Method in class org.apache.hudi.integ.testsuite.dag.WriterContext
 
getInputParallelism() - Method in class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
 
getIntermittentDelayMins() - Method in class org.apache.hudi.integ.testsuite.dag.WorkflowDag
 
getIterationCountToExecute() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getJsc() - Method in class org.apache.hudi.integ.testsuite.dag.ExecutionContext
 
getLastCheckpoint() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
getLocalHiveServer() - Method in class org.apache.hudi.integ.testsuite.helpers.HiveServiceProvider
 
getLogger() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.BaseValidateDatasetNode
 
getLogger() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.ValidateDatasetNode
 
getMatchedAction() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getMaxFileSize() - Method in class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
 
getMergeCondition() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getName() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
getNewPayload() - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
Create a new GenericRecord with random value according to given schema.
getNewPayload(Schema) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
 
getNewPayload(Set<String>) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
Create a new GenericRecord with random value according to given schema.
getNewPayload(Schema) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordPartialPayloadGenerator
 
getNewPayloadWithTimestamp(String) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
 
getNewWriter() - Method in class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
 
getNewWriter() - Method in interface org.apache.hudi.integ.testsuite.writer.DeltaInputWriter
Return a new instance of this writer.
getNewWriter() - Method in class org.apache.hudi.integ.testsuite.writer.SparkAvroDeltaInputWriter
 
getNextBatch() - Method in class org.apache.hudi.integ.testsuite.HoodieContinousTestSuiteWriter
 
getNextBatch() - Method in class org.apache.hudi.integ.testsuite.HoodieInlineTestSuiteWriter
 
getNextBatch() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
getNextFilePathsAndMaxModificationTime(Option<String>, long) - Method in class org.apache.hudi.integ.testsuite.helpers.DFSTestSuitePathSelector
 
getNodeList() - Method in class org.apache.hudi.integ.testsuite.dag.WorkflowDag
 
getNonNull(Schema) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
 
getNonNull(Schema) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadSizeEstimator
Get the nonNull Schema of a given UNION Schema.
getNotMatchedAction() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getNumDeletePartitions() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getNumInsertPartitions() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getNumRecordsDelete() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getNumRecordsInsert() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getNumRecordsUpsert() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getNumRollbacks() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getNumUpsertFiles() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getNumUpsertPartitions() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getOtherConfigs() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getParentNodes() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
getPartitionField() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getPartitionPathRecordKey() - Method in class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
 
getPartitions(Option<Integer>) - Method in class org.apache.hudi.integ.testsuite.reader.DFSHoodieDatasetInputReader
 
getPartitionsToDelete() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getPartitionToCountMap(JavaRDD<GenericRecord>) - Method in class org.apache.hudi.integ.testsuite.generator.DeltaGenerator
 
getPath() - Method in class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
 
getPreCombineField() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getPrestoProperties() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getPrestoQueries() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getPrimaryKey() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getProps() - Method in class org.apache.hudi.integ.testsuite.dag.WriterContext
 
getProps() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
getRatioRecordsChange() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getRecordSize() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getRecordsWritten() - Method in class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
 
getReinitContext() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getRepeatCount() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getResult() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
getRounds() - Method in class org.apache.hudi.integ.testsuite.dag.WorkflowDag
 
getSchema() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
getSchemaStr() - Method in class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
 
getSchemaVersion() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getSparkContext() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
getSparkSession() - Method in class org.apache.hudi.integ.testsuite.dag.WriterContext
 
getStartPartition() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getTableType() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getTrinoProperties() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getTrinoQueries() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getUpdateColumn() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getUpdatePayload(GenericRecord, Set<String>) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
Update a given GenericRecord with random value.
getUpdatePayloadWithTimestamp(GenericRecord, Set<String>, String) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
 
getWhereConditionColumn() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
getWriteClient(DagNode) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
getWriteConfig() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
getWriterContext() - Method in class org.apache.hudi.integ.testsuite.dag.ExecutionContext
 

H

hashCode() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
hasNext() - Method in class org.apache.hudi.integ.testsuite.generator.DeleteGeneratorIterator
 
hasNext() - Method in class org.apache.hudi.integ.testsuite.generator.FlexibleSchemaRecordGenerationIterator
 
hasNext() - Method in class org.apache.hudi.integ.testsuite.generator.UpdateGeneratorIterator
 
help - Variable in class org.apache.hudi.integ.testsuite.SparkDataSourceContinuousIngestTool.Config
 
HIVE_PROPERTIES - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
HIVE_QUERIES - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
HiveQueryNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
A hive query node in the DAG of operations for a workflow.
HiveQueryNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.HiveQueryNode
 
HiveServiceProvider - Class in org.apache.hudi.integ.testsuite.helpers
Hive Service provider.
HiveServiceProvider(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.helpers.HiveServiceProvider
 
HiveSyncNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
Represents a hive sync node in the DAG of operations for a workflow.
HiveSyncNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.HiveSyncNode
 
HoodieContinousTestSuiteWriter - Class in org.apache.hudi.integ.testsuite
Test suite Writer that assists in testing async table operations with Deltastreamer continuous mode.
HoodieContinousTestSuiteWriter(JavaSparkContext, Properties, HoodieTestSuiteJob.HoodieTestSuiteConfig, String) - Constructor for class org.apache.hudi.integ.testsuite.HoodieContinousTestSuiteWriter
 
HoodieDeltaStreamerWrapper - Class in org.apache.hudi.integ.testsuite
Extends the HoodieDeltaStreamer to expose certain operations helpful in running the Test Suite.
HoodieDeltaStreamerWrapper(HoodieDeltaStreamer.Config, JavaSparkContext) - Constructor for class org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper
 
HoodieInlineTestSuiteWriter - Class in org.apache.hudi.integ.testsuite
A writer abstraction for the Hudi test suite.
HoodieInlineTestSuiteWriter(JavaSparkContext, Properties, HoodieTestSuiteJob.HoodieTestSuiteConfig, String) - Constructor for class org.apache.hudi.integ.testsuite.HoodieInlineTestSuiteWriter
 
HoodieMultiWriterTestSuiteConfig() - Constructor for class org.apache.hudi.integ.testsuite.HoodieMultiWriterTestSuiteJob.HoodieMultiWriterTestSuiteConfig
 
HoodieMultiWriterTestSuiteJob - Class in org.apache.hudi.integ.testsuite
Multi write test suite job to assist in testing multi-writer scenarios.
HoodieMultiWriterTestSuiteJob() - Constructor for class org.apache.hudi.integ.testsuite.HoodieMultiWriterTestSuiteJob
 
HoodieMultiWriterTestSuiteJob.HoodieMultiWriterTestSuiteConfig - Class in org.apache.hudi.integ.testsuite
 
hoodieReadClient - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
HoodieTestSuiteConfig() - Constructor for class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
HoodieTestSuiteJob - Class in org.apache.hudi.integ.testsuite
This is the entry point for running a Hudi Test Suite.
HoodieTestSuiteJob(HoodieTestSuiteJob.HoodieTestSuiteConfig, JavaSparkContext) - Constructor for class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob
 
HoodieTestSuiteJob(HoodieTestSuiteJob.HoodieTestSuiteConfig, JavaSparkContext, boolean) - Constructor for class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob
 
HoodieTestSuiteJob.HoodieTestSuiteConfig - Class in org.apache.hudi.integ.testsuite
The Hudi test suite uses HoodieDeltaStreamer to run some operations hence extend delta streamer config.
HoodieTestSuiteWriter - Class in org.apache.hudi.integ.testsuite
 
HoodieTestSuiteWriter(JavaSparkContext, Properties, HoodieTestSuiteJob.HoodieTestSuiteConfig, String) - Constructor for class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 

I

indexType - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
ingest(HoodieTestSuiteWriter, Option<String>) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.BulkInsertNode
 
ingest(HoodieTestSuiteWriter, Option<String>) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DeleteNode
 
ingest(HoodieTestSuiteWriter, Option<String>) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.InsertNode
 
ingest(HoodieTestSuiteWriter, Option<String>) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.InsertOverwriteNode
 
ingest(HoodieTestSuiteWriter, Option<String>) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.InsertOverwriteTableNode
 
ingest(HoodieTestSuiteWriter, Option<String>) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.UpsertNode
 
initContext(JavaSparkContext) - Method in class org.apache.hudi.integ.testsuite.dag.WriterContext
 
inlineClustering() - Method in class org.apache.hudi.integ.testsuite.HoodieContinousTestSuiteWriter
 
inlineClustering() - Method in class org.apache.hudi.integ.testsuite.HoodieInlineTestSuiteWriter
 
inlineClustering() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
inputBasePath - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
inputBasePaths - Variable in class org.apache.hudi.integ.testsuite.HoodieMultiWriterTestSuiteJob.HoodieMultiWriterTestSuiteConfig
 
inputFormatName - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
inputParallelism - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
inputPartitonsToSkipWithValidate() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
insert(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieContinousTestSuiteWriter
 
insert() - Method in class org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper
 
insert(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieInlineTestSuiteWriter
 
insert(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
InsertNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
An insert node in the DAG of operations for a workflow.
InsertNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.InsertNode
 
insertOverwrite(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieContinousTestSuiteWriter
 
insertOverwrite() - Method in class org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper
 
insertOverwrite(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieInlineTestSuiteWriter
 
insertOverwrite(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
InsertOverwriteNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
 
InsertOverwriteNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.InsertOverwriteNode
 
insertOverwriteTable(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieContinousTestSuiteWriter
 
insertOverwriteTable() - Method in class org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper
 
insertOverwriteTable(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieInlineTestSuiteWriter
 
insertOverwriteTable(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
InsertOverwriteTableNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
 
InsertOverwriteTableNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.InsertOverwriteTableNode
 
isCompleted() - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
isDeleteInputData() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
isDeleteInputDataExceptLatest() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
isDisableGenerate() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
isDisableIngest() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
isEnableMetadataValidate() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
isHiveLocal() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
isOption(Schema) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
Check whether a schema is option.
isOption(Schema) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadSizeEstimator
 
isPrimitive(Schema) - Static method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
 
isTableExternal() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
isValidateFullData() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
isValidateHive() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 

L

lastCheckpoint - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
LazyRecordGeneratorIterator - Class in org.apache.hudi.integ.testsuite.generator
A lazy record generator to generate GenericRecords lazily and not hold a list of records in memory.
LazyRecordGeneratorIterator(Iterator<GenericRecord>) - Constructor for class org.apache.hudi.integ.testsuite.generator.LazyRecordGeneratorIterator
 
limitFileSize - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
log - Static variable in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
log - Static variable in class org.apache.hudi.integ.testsuite.dag.WriterContext
 
log - Static variable in class org.apache.hudi.integ.testsuite.schema.TestSuiteFileBasedSchemaProvider
 

M

main(String[]) - Static method in class org.apache.hudi.integ.testsuite.HoodieMultiWriterTestSuiteJob
 
main(String[]) - Static method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob
 
main(String[]) - Static method in class org.apache.hudi.integ.testsuite.SparkDataSourceContinuousIngestTool
 
maxWaitTimeForDeltastreamerToCatchupMs() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
minSyncIntervalSeconds - Variable in class org.apache.hudi.integ.testsuite.SparkDataSourceContinuousIngestTool.Config
 

N

newBuilder() - Static method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
next() - Method in class org.apache.hudi.integ.testsuite.generator.DeleteGeneratorIterator
 
next() - Method in class org.apache.hudi.integ.testsuite.generator.FlexibleSchemaRecordGenerationIterator
 
next() - Method in class org.apache.hudi.integ.testsuite.generator.UpdateGeneratorIterator
 
NO_DEPENDENCY_VALUE - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
NODE_NAME - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 

O

org.apache.hudi.integ.testsuite - package org.apache.hudi.integ.testsuite
 
org.apache.hudi.integ.testsuite.configuration - package org.apache.hudi.integ.testsuite.configuration
 
org.apache.hudi.integ.testsuite.converter - package org.apache.hudi.integ.testsuite.converter
 
org.apache.hudi.integ.testsuite.dag - package org.apache.hudi.integ.testsuite.dag
 
org.apache.hudi.integ.testsuite.dag.nodes - package org.apache.hudi.integ.testsuite.dag.nodes
 
org.apache.hudi.integ.testsuite.dag.scheduler - package org.apache.hudi.integ.testsuite.dag.scheduler
 
org.apache.hudi.integ.testsuite.generator - package org.apache.hudi.integ.testsuite.generator
 
org.apache.hudi.integ.testsuite.helpers - package org.apache.hudi.integ.testsuite.helpers
 
org.apache.hudi.integ.testsuite.reader - package org.apache.hudi.integ.testsuite.reader
 
org.apache.hudi.integ.testsuite.schema - package org.apache.hudi.integ.testsuite.schema
 
org.apache.hudi.integ.testsuite.writer - package org.apache.hudi.integ.testsuite.writer
 
outputTypeName - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 

P

parentNodes - Variable in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
PRESTO_PROPERTIES - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
PRESTO_QUERIES - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
prestoJdbcUrl - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
prestoPassword - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
PrestoQueryNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
 
PrestoQueryNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.PrestoQueryNode
 
prestoUsername - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
props - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
propsFilePath - Variable in class org.apache.hudi.integ.testsuite.SparkDataSourceContinuousIngestTool.Config
 
propsFilePaths - Variable in class org.apache.hudi.integ.testsuite.HoodieMultiWriterTestSuiteJob.HoodieMultiWriterTestSuiteConfig
 

R

random - Variable in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
 
randomize(GenericRecord, Set<String>) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
Set random value to GenericRecord according to the schema type of field.
read(long) - Method in interface org.apache.hudi.integ.testsuite.reader.DeltaInputReader
Attempts to reads an approximate number of records close to approxNumRecords.
read(int, long) - Method in interface org.apache.hudi.integ.testsuite.reader.DeltaInputReader
 
read(int, int, long) - Method in interface org.apache.hudi.integ.testsuite.reader.DeltaInputReader
 
read(int, int, double) - Method in interface org.apache.hudi.integ.testsuite.reader.DeltaInputReader
 
read(long) - Method in class org.apache.hudi.integ.testsuite.reader.DFSAvroDeltaInputReader
 
read(int, long) - Method in class org.apache.hudi.integ.testsuite.reader.DFSAvroDeltaInputReader
 
read(int, int, long) - Method in class org.apache.hudi.integ.testsuite.reader.DFSAvroDeltaInputReader
 
read(int, int, double) - Method in class org.apache.hudi.integ.testsuite.reader.DFSAvroDeltaInputReader
 
read(long) - Method in class org.apache.hudi.integ.testsuite.reader.DFSHoodieDatasetInputReader
 
read(int, long) - Method in class org.apache.hudi.integ.testsuite.reader.DFSHoodieDatasetInputReader
 
read(int, int, long) - Method in class org.apache.hudi.integ.testsuite.reader.DFSHoodieDatasetInputReader
 
read(int, int, double) - Method in class org.apache.hudi.integ.testsuite.reader.DFSHoodieDatasetInputReader
 
readAvro(SparkSession, String, List<String>, Option<String>, Option<String>) - Static method in class org.apache.hudi.integ.testsuite.reader.SparkBasedReader
 
readOrc(SparkSession, List<String>, Option<String>, Option<String>) - Static method in class org.apache.hudi.integ.testsuite.reader.SparkBasedReader
 
readParquet(SparkSession, List<String>, Option<String>, Option<String>) - Static method in class org.apache.hudi.integ.testsuite.reader.SparkBasedReader
 
reinitContext(Map<String, Object>) - Method in class org.apache.hudi.integ.testsuite.dag.WriterContext
 
reinitializeContext(boolean) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
result - Variable in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
RollbackNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
A rollback node in the DAG helps to perform rollback operations.
RollbackNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.RollbackNode
 
rollOver() - Method in class org.apache.hudi.integ.testsuite.writer.DFSDeltaWriterAdapter
 
run() - Method in class org.apache.hudi.integ.testsuite.SparkDataSourceContinuousIngestTool
 
runTestSuite() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob
 

S

SaferSchemaDagScheduler - Class in org.apache.hudi.integ.testsuite.dag.scheduler
 
SaferSchemaDagScheduler(WorkflowDag, WriterContext, JavaSparkContext) - Constructor for class org.apache.hudi.integ.testsuite.dag.scheduler.SaferSchemaDagScheduler
 
SaferSchemaDagScheduler(WorkflowDag, WriterContext, JavaSparkContext, int) - Constructor for class org.apache.hudi.integ.testsuite.dag.scheduler.SaferSchemaDagScheduler
 
saferSchemaEvolution - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
schedule() - Method in class org.apache.hudi.integ.testsuite.dag.scheduler.DagScheduler
Method to start executing workflow DAGs.
scheduleCompact() - Method in class org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper
 
scheduleCompaction(Option<Map<String, String>>) - Method in class org.apache.hudi.integ.testsuite.HoodieContinousTestSuiteWriter
 
scheduleCompaction(Option<Map<String, String>>) - Method in class org.apache.hudi.integ.testsuite.HoodieInlineTestSuiteWriter
 
scheduleCompaction(Option<Map<String, String>>) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
ScheduleCompactNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
A schedule node in the DAG of operations for a workflow helps to schedule compact operation.
ScheduleCompactNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.ScheduleCompactNode
 
schema - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
SchemaUtils - Class in org.apache.hudi.integ.testsuite.schema
 
SchemaUtils() - Constructor for class org.apache.hudi.integ.testsuite.schema.SchemaUtils
 
setBatchId(Integer) - Method in class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
 
setBytesWritten(long) - Method in class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
 
setCompleted(boolean) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
setFilePath(String) - Method in class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
 
setParentNodes(List<DagNode<O>>) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.DagNode
 
setPartitionPathRecordKey(List<Pair<String, String>>) - Method in class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
 
setRecordsWritten(long) - Method in class org.apache.hudi.integ.testsuite.writer.DeltaWriteStats
 
setSessionProperties(List<String>, Statement) - Method in class org.apache.hudi.integ.testsuite.dag.nodes.BaseQueryNode
 
shouldDeleteOldInputData() - Method in class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
 
shouldUseCtas() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
shouldUseHudiToGenerateUpdates() - Method in class org.apache.hudi.integ.testsuite.configuration.DFSDeltaConfig
 
shutdownResources() - Method in class org.apache.hudi.integ.testsuite.dag.WriterContext
 
shutdownResources() - Method in class org.apache.hudi.integ.testsuite.HoodieContinousTestSuiteWriter
 
shutdownResources() - Method in class org.apache.hudi.integ.testsuite.HoodieInlineTestSuiteWriter
 
shutdownResources() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
SimpleWorkflowDagGenerator - Class in org.apache.hudi.integ.testsuite.dag
An example of how to generate a workflow dag programmatically.
SimpleWorkflowDagGenerator() - Constructor for class org.apache.hudi.integ.testsuite.dag.SimpleWorkflowDagGenerator
 
SOURCE_ORDERING_FIELD - Static variable in class org.apache.hudi.integ.testsuite.schema.SchemaUtils
 
sourcePath - Variable in class org.apache.hudi.integ.testsuite.SparkDataSourceContinuousIngestTool.Config
 
SPARK_AVRO_FORMAT - Static variable in class org.apache.hudi.integ.testsuite.reader.SparkBasedReader
 
SPARK_PARQUET_FORMAT - Static variable in class org.apache.hudi.integ.testsuite.reader.SparkBasedReader
 
SparkAvroDeltaInputWriter - Class in org.apache.hudi.integ.testsuite.writer
Spark based avro delta input writer.
SparkAvroDeltaInputWriter(SparkSession, String, String) - Constructor for class org.apache.hudi.integ.testsuite.writer.SparkAvroDeltaInputWriter
 
SparkBasedReader - Class in org.apache.hudi.integ.testsuite.reader
Helper class to read avro and/or parquet files and generate a RDD of GenericRecord.
SparkBasedReader() - Constructor for class org.apache.hudi.integ.testsuite.reader.SparkBasedReader
 
sparkContext - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
SparkDataSourceContinuousIngestTool - Class in org.apache.hudi.integ.testsuite
Sample command ./bin/spark-submit --packages org.apache.spark:spark-avro_2.11:2.4.4 --driver-memory 4g --executor-memory 4g \ --conf spark.serializer=org.apache.spark.serializer.KryoSerializer --conf spark.sql.catalogImplementation=hive \ --class org.apache.hudi.integ.testsuite.SparkDSContinuousIngestTool \ ${HUDI_ROOT_DIR}/packaging/hudi-integ-test-bundle/target/hudi-integ-test-bundle-0.11.0-SNAPSHOT.jar \ --source-path file:${SOURCE_DIR}/spark_ds_continuous --checkpoint-file-path /tmp/hudi/checkpoint \ --base-path file:///tmp/hudi/tbl_path/ --props /tmp/hudi_props.out Contents of hudi.properties hoodie.insert.shuffle.parallelism=4 hoodie.upsert.shuffle.parallelism=4 hoodie.bulkinsert.shuffle.parallelism=4 hoodie.delete.shuffle.parallelism=4 hoodie.datasource.write.recordkey.field=VendorID hoodie.datasource.write.partitionpath.field=date_col hoodie.datasource.write.operation=upsert hoodie.datasource.write.precombine.field=tpep_pickup_datetime hoodie.metadata.enable=false hoodie.table.name=hudi_tbl
SparkDataSourceContinuousIngestTool(JavaSparkContext, SparkDataSourceContinuousIngestTool.Config) - Constructor for class org.apache.hudi.integ.testsuite.SparkDataSourceContinuousIngestTool
 
SparkDataSourceContinuousIngestTool.Config - Class in org.apache.hudi.integ.testsuite
 
sparkFormat - Variable in class org.apache.hudi.integ.testsuite.SparkDataSourceContinuousIngestTool.Config
 
sparkMaster - Variable in class org.apache.hudi.integ.testsuite.SparkDataSourceContinuousIngestTool.Config
 
sparkMemory - Variable in class org.apache.hudi.integ.testsuite.SparkDataSourceContinuousIngestTool.Config
 
sparkSession - Variable in class org.apache.hudi.integ.testsuite.writer.SparkAvroDeltaInputWriter
 
SparkSQLQueryNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
A SparkSQL query node in the DAG of operations for a workflow.
SparkSQLQueryNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.SparkSQLQueryNode
 
start() - Method in class org.apache.hudi.integ.testsuite.generator.LazyRecordGeneratorIterator
 
startCommit() - Method in class org.apache.hudi.integ.testsuite.HoodieContinousTestSuiteWriter
 
startCommit() - Method in class org.apache.hudi.integ.testsuite.HoodieInlineTestSuiteWriter
 
startCommit() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
startHiveMetastore - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
startLocalHiveServiceIfNeeded(Configuration) - Method in class org.apache.hudi.integ.testsuite.helpers.HiveServiceProvider
 
startLocalZookeeperIfNeeded() - Method in class org.apache.hudi.integ.testsuite.helpers.ZookeeperServiceProvider
 
startZookeeper - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
stopLocalHiveServiceIfNeeded() - Method in class org.apache.hudi.integ.testsuite.helpers.HiveServiceProvider
 
stopLocalZookeeperIfNeeded() - Method in class org.apache.hudi.integ.testsuite.helpers.ZookeeperServiceProvider
 
stopQuietly() - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob
 
syncToLocalHiveIfNeeded(HoodieTestSuiteWriter) - Method in class org.apache.hudi.integ.testsuite.helpers.HiveServiceProvider
 

T

testContinousMode - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
TestSuiteFileBasedSchemaProvider - Class in org.apache.hudi.integ.testsuite.schema
Appends source ordering field to both source and target schemas.
TestSuiteFileBasedSchemaProvider(TypedProperties, JavaSparkContext) - Constructor for class org.apache.hudi.integ.testsuite.schema.TestSuiteFileBasedSchemaProvider
 
toString() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
toString(InputStream) - Static method in class org.apache.hudi.integ.testsuite.dag.DagUtils
 
toString() - Method in class org.apache.hudi.integ.testsuite.dag.WriterContext
 
TRINO_PROPERTIES - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
TRINO_QUERIES - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
trinoJdbcUrl - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
trinoPassword - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
TrinoQueryNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
 
TrinoQueryNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.TrinoQueryNode
 
trinoUsername - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
TYPE - Static variable in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
typeEstimateAndNumComplexFields() - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadSizeEstimator
 

U

UpdateConverter - Class in org.apache.hudi.integ.testsuite.converter
This converter creates an update GenericRecord from an existing GenericRecord.
UpdateConverter(String, int, List<String>, List<String>) - Constructor for class org.apache.hudi.integ.testsuite.converter.UpdateConverter
 
UpdateGeneratorIterator - Class in org.apache.hudi.integ.testsuite.generator
A lazy update payload generator to generate GenericRecords lazily.
UpdateGeneratorIterator(Iterator<GenericRecord>, String, List<String>, List<String>, int) - Constructor for class org.apache.hudi.integ.testsuite.generator.UpdateGeneratorIterator
 
updateTimestamp(GenericRecord, String) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
Generates a sequential timestamp (daily increment), and updates the timestamp field of the record.
upsert(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieContinousTestSuiteWriter
 
upsert(WriteOperationType) - Method in class org.apache.hudi.integ.testsuite.HoodieDeltaStreamerWrapper
 
upsert(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieInlineTestSuiteWriter
 
upsert(Option<String>) - Method in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
UpsertNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
Represents an upsert node in the DAG of operations for a workflow.
UpsertNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.UpsertNode
 
useDeltaStreamer - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
useHudiToGenerateUpdates - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 

V

VALID_DAG_NODES_TO_ALLOW_WRITE_CLIENT_IN_DELTASTREAMER_MODE - Static variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
validate(GenericRecord) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordFullPayloadGenerator
Validate whether the record match schema.
validate(GenericRecord) - Method in class org.apache.hudi.integ.testsuite.generator.GenericRecordPartialPayloadGenerator
 
validateArchival() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
ValidateAsyncOperations - Class in org.apache.hudi.integ.testsuite.dag.nodes
Node to validate data set sanity like total file versions retained, has cleaning happened, has archival happened, etc.
ValidateAsyncOperations(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.ValidateAsyncOperations
 
validateClean() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
ValidateDatasetNode - Class in org.apache.hudi.integ.testsuite.dag.nodes
This validation node uses spark datasource for comparison purposes.
ValidateDatasetNode(DeltaConfig.Config) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.ValidateDatasetNode
 
ValidateNode<R> - Class in org.apache.hudi.integ.testsuite.dag.nodes
A validate node helps to validate its parent nodes with given function.
ValidateNode(DeltaConfig.Config, Function<List<DagNode>, R>) - Constructor for class org.apache.hudi.integ.testsuite.dag.nodes.ValidateNode
 
validateOnceEveryIteration() - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config
 
valueOf(String) - Static method in enum org.apache.hudi.integ.testsuite.reader.DeltaInputType
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum org.apache.hudi.integ.testsuite.writer.DeltaOutputMode
Returns the enum constant of this type with the specified name.
values() - Static method in enum org.apache.hudi.integ.testsuite.reader.DeltaInputType
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum org.apache.hudi.integ.testsuite.writer.DeltaOutputMode
Returns an array containing the constants of this enum type, in the order they are declared.

W

withConfig(String, Object) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withConfigsMap(Map<String, Object>) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withFractionUpsertPerFile(double) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withHiveLocal(boolean) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withHiveProperties(List<String>) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withHiveQueryAndResults(List<Pair<String, Integer>>) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withName(String) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withNumDeletePartitions(int) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withNumInsertPartitions(int) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withNumRecordsToDelete(long) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withNumRecordsToInsert(long) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withNumRecordsToUpdate(long) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withNumRollbacks(int) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withNumTimesToRepeat(int) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withNumUpsertFiles(int) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withNumUpsertPartitions(int) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withPrestoProperties(List<String>) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withPrestoQueryAndResults(List<Pair<String, Integer>>) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withRecordSize(int) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withSchemaVersion(int) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withStartPartition(int) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withTrinoProperties(List<String>) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
withTrinoQueryAndResults(List<Pair<String, Integer>>) - Method in class org.apache.hudi.integ.testsuite.configuration.DeltaConfig.Config.Builder
 
WorkflowDag<O> - Class in org.apache.hudi.integ.testsuite.dag
Workflow dag that encapsulates all execute nodes.
WorkflowDag(List<DagNode<O>>) - Constructor for class org.apache.hudi.integ.testsuite.dag.WorkflowDag
 
WorkflowDag(String, int, int, List<DagNode<O>>) - Constructor for class org.apache.hudi.integ.testsuite.dag.WorkflowDag
 
WorkflowDagGenerator - Interface in org.apache.hudi.integ.testsuite.dag
A interface which represents a workflow dag generator.
workloadDagGenerator - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
workloadYamlPath - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteJob.HoodieTestSuiteConfig
 
workloadYamlPaths - Variable in class org.apache.hudi.integ.testsuite.HoodieMultiWriterTestSuiteJob.HoodieMultiWriterTestSuiteConfig
 
write(J) - Method in interface org.apache.hudi.integ.testsuite.writer.DeltaWriterAdapter.SparkBasedDeltaWriter
 
write(Iterator<I>) - Method in interface org.apache.hudi.integ.testsuite.writer.DeltaWriterAdapter
 
write(Iterator<GenericRecord>) - Method in class org.apache.hudi.integ.testsuite.writer.DFSDeltaWriterAdapter
 
write(JavaRDD<GenericRecord>) - Method in class org.apache.hudi.integ.testsuite.writer.DFSSparkAvroDeltaWriter
 
writeClient - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
writeConfig - Variable in class org.apache.hudi.integ.testsuite.HoodieTestSuiteWriter
 
writeData(GenericRecord) - Method in class org.apache.hudi.integ.testsuite.writer.AvroFileDeltaInputWriter
 
writeData(I) - Method in interface org.apache.hudi.integ.testsuite.writer.DeltaInputWriter
Generate any type of data.
writeData(JavaRDD<GenericRecord>) - Method in class org.apache.hudi.integ.testsuite.writer.SparkAvroDeltaInputWriter
 
WriterContext - Class in org.apache.hudi.integ.testsuite.dag
WriterContext wraps the delta writer/data generator related configuration needed to init/reinit.
WriterContext(JavaSparkContext, TypedProperties, HoodieTestSuiteJob.HoodieTestSuiteConfig, BuiltinKeyGenerator, SparkSession) - Constructor for class org.apache.hudi.integ.testsuite.dag.WriterContext
 
writeRecords(JavaRDD<GenericRecord>) - Method in class org.apache.hudi.integ.testsuite.generator.DeltaGenerator
 

Z

ZookeeperServiceProvider - Class in org.apache.hudi.integ.testsuite.helpers
 
ZookeeperServiceProvider(DeltaConfig.Config, Configuration) - Constructor for class org.apache.hudi.integ.testsuite.helpers.ZookeeperServiceProvider
 
A B C D E F G H I L M N O P R S T U V W Z 
Skip navigation links

Copyright © 2023 The Apache Software Foundation. All rights reserved.