Serialized Form (hudi-utilities 0.5.0-incubating API)

Package org.apache.hudi.utilities
- Class org.apache.hudi.utilities.HDFSParquetImporter extends Object implements Serializable
  - Serialized Fields
    - cfg
```
HDFSParquetImporter.Config cfg
```
    - props
```
TypedProperties props
```
      Bag of properties with source, hoodie client, key generator etc.
- Class org.apache.hudi.utilities.HDFSParquetImporter.Config extends Object implements Serializable
  - Serialized Fields
    - command
```
String command
```
    - srcPath
```
String srcPath
```
    - targetPath
```
String targetPath
```
    - tableName
```
String tableName
```
    - tableType
```
String tableType
```
    - rowKey
```
String rowKey
```
    - partitionKey
```
String partitionKey
```
    - parallelism
```
int parallelism
```
    - schemaFile
```
String schemaFile
```
    - format
```
String format
```
    - sparkMaster
```
String sparkMaster
```
    - sparkMemory
```
String sparkMemory
```
    - retry
```
int retry
```
    - propsFilePath
```
String propsFilePath
```
    - configs
```
List<E> configs
```
    - help
```
Boolean help
```
- Class org.apache.hudi.utilities.HiveIncrementalPuller.Config extends Object implements Serializable
  - Serialized Fields
    - hiveJDBCUrl
```
String hiveJDBCUrl
```
    - hiveUsername
```
String hiveUsername
```
    - hivePassword
```
String hivePassword
```
    - yarnQueueName
```
String yarnQueueName
```
    - hoodieTmpDir
```
String hoodieTmpDir
```
    - incrementalSQLFile
```
String incrementalSQLFile
```
    - sourceDb
```
String sourceDb
```
    - sourceTable
```
String sourceTable
```
    - targetDb
```
String targetDb
```
    - targetTable
```
String targetTable
```
    - tmpDb
```
String tmpDb
```
    - fromCommitTime
```
String fromCommitTime
```
    - maxCommits
```
int maxCommits
```
    - help
```
Boolean help
```
- Class org.apache.hudi.utilities.HoodieCleaner.Config extends Object implements Serializable
  - Serialized Fields
    - basePath
```
String basePath
```
    - propsFilePath
```
String propsFilePath
```
    - configs
```
List<E> configs
```
    - sparkMaster
```
String sparkMaster
```
    - help
```
Boolean help
```
- Class org.apache.hudi.utilities.HoodieCompactionAdminTool.Config extends Object implements Serializable
  - Serialized Fields
    - operation
```
HoodieCompactionAdminTool.Operation operation
```
    - basePath
```
String basePath
```
    - compactionInstantTime
```
String compactionInstantTime
```
    - partitionPath
```
String partitionPath
```
    - fileId
```
String fileId
```
    - parallelism
```
int parallelism
```
    - sparkMaster
```
String sparkMaster
```
    - sparkMemory
```
String sparkMemory
```
    - dryRun
```
boolean dryRun
```
    - skipValidation
```
boolean skipValidation
```
    - outputPath
```
String outputPath
```
    - printOutput
```
boolean printOutput
```
    - help
```
Boolean help
```
- Class org.apache.hudi.utilities.HoodieCompactor.Config extends Object implements Serializable
  - Serialized Fields
    - basePath
```
String basePath
```
    - tableName
```
String tableName
```
    - compactionInstantTime
```
String compactionInstantTime
```
    - parallelism
```
int parallelism
```
    - schemaFile
```
String schemaFile
```
    - sparkMaster
```
String sparkMaster
```
    - sparkMemory
```
String sparkMemory
```
    - retry
```
int retry
```
    - runSchedule
```
Boolean runSchedule
```
    - strategyClassName
```
String strategyClassName
```
    - help
```
Boolean help
```
    - propsFilePath
```
String propsFilePath
```
    - configs
```
List<E> configs
```
- Class org.apache.hudi.utilities.HoodieSnapshotCopier extends Object implements Serializable
- Class org.apache.hudi.utilities.HoodieWithTimelineServer extends Object implements Serializable
  - Serialized Fields
    - cfg
```
HoodieWithTimelineServer.Config cfg
```
- Class org.apache.hudi.utilities.HoodieWithTimelineServer.Config extends Object implements Serializable
  - Serialized Fields
    - sparkMaster
```
String sparkMaster
```
    - sparkMemory
```
String sparkMemory
```
    - numPartitions
```
Integer numPartitions
```
    - serverPort
```
Integer serverPort
```
    - delaySecs
```
Integer delaySecs
```
    - help
```
Boolean help
```
Package org.apache.hudi.utilities.adhoc
- Class org.apache.hudi.utilities.adhoc.UpgradePayloadFromUberToApache extends Object implements Serializable
  - Serialized Fields
    - cfg
```
UpgradePayloadFromUberToApache.Config cfg
```
- Class org.apache.hudi.utilities.adhoc.UpgradePayloadFromUberToApache.Config extends Object implements Serializable
  - Serialized Fields
    - inputPath
```
String inputPath
```
    - help
```
Boolean help
```
Package org.apache.hudi.utilities.deltastreamer
- Class org.apache.hudi.utilities.deltastreamer.AbstractDeltaStreamerService extends Object implements Serializable
  - Serialized Fields
    - started
```
boolean started
```
    - shutdownRequested
```
boolean shutdownRequested
```
    - shutdown
```
boolean shutdown
```
- Class org.apache.hudi.utilities.deltastreamer.Compactor extends Object implements Serializable
- Class org.apache.hudi.utilities.deltastreamer.DeltaSync extends Object implements Serializable
  - Serialized Fields
    - cfg
```
HoodieDeltaStreamer.Config cfg
```
      Delta Sync Config
    - keyGenerator
```
KeyGenerator keyGenerator
```
      Extract the key for the target dataset
    - props
```
TypedProperties props
```
      Bag of properties with source, hoodie client, key generator etc.
    - tableType
```
HoodieTableType tableType
```
      Table Type
- Class org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer extends Object implements Serializable
- Class org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.AsyncCompactService extends AbstractDeltaStreamerService implements Serializable
  - Serialized Fields
    - maxConcurrentCompaction
```
int maxConcurrentCompaction
```
- Class org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.Config extends Object implements Serializable
  - Serialized Fields
    - targetBasePath
```
String targetBasePath
```
    - targetTableName
```
String targetTableName
```
    - storageType
```
String storageType
```
    - propsFilePath
```
String propsFilePath
```
    - configs
```
List<E> configs
```
    - sourceClassName
```
String sourceClassName
```
    - sourceOrderingField
```
String sourceOrderingField
```
    - payloadClassName
```
String payloadClassName
```
    - schemaProviderClassName
```
String schemaProviderClassName
```
    - transformerClassName
```
String transformerClassName
```
    - sourceLimit
```
long sourceLimit
```
    - operation
```
HoodieDeltaStreamer.Operation operation
```
    - filterDupes
```
Boolean filterDupes
```
    - enableHiveSync
```
Boolean enableHiveSync
```
    - maxPendingCompactions
```
Integer maxPendingCompactions
```
    - continuousMode
```
Boolean continuousMode
```
    - minSyncIntervalSeconds
```
Integer minSyncIntervalSeconds
```
    - sparkMaster
```
String sparkMaster
```
    - commitOnErrors
```
Boolean commitOnErrors
```
    - deltaSyncSchedulingWeight
```
Integer deltaSyncSchedulingWeight
```
    - compactSchedulingWeight
```
Integer compactSchedulingWeight
```
    - deltaSyncSchedulingMinShare
```
Integer deltaSyncSchedulingMinShare
```
    - compactSchedulingMinShare
```
Integer compactSchedulingMinShare
```
    - forceDisableCompaction
```
Boolean forceDisableCompaction
```
      Compaction is enabled for MoR table by default. This flag disables it
    - checkpoint
```
String checkpoint
```
      Resume Delta Streamer from this checkpoint.
    - help
```
Boolean help
```
- Class org.apache.hudi.utilities.deltastreamer.HoodieDeltaStreamer.DeltaSyncService extends AbstractDeltaStreamerService implements Serializable
  - Serialized Fields
    - cfg
```
HoodieDeltaStreamer.Config cfg
```
      Delta Sync Config
    - props
```
TypedProperties props
```
      Bag of properties with source, hoodie client, key generator etc.
    - asyncCompactService
```
HoodieDeltaStreamer.AsyncCompactService asyncCompactService
```
      Async Compactor Service
    - tableType
```
HoodieTableType tableType
```
      Table Type
Package org.apache.hudi.utilities.exception
- Class org.apache.hudi.utilities.exception.HoodieDeltaStreamerException extends HoodieException implements Serializable
- Class org.apache.hudi.utilities.exception.HoodieIncrementalPullException extends HoodieException implements Serializable
- Class org.apache.hudi.utilities.exception.HoodieIncrementalPullSQLException extends HoodieIncrementalPullException implements Serializable
Package org.apache.hudi.utilities.keygen
- Class org.apache.hudi.utilities.keygen.TimestampBasedKeyGenerator extends SimpleKeyGenerator implements Serializable
  - Serialized Fields
    - timestampType
```
org.apache.hudi.utilities.keygen.TimestampBasedKeyGenerator.TimestampType timestampType
```
    - inputDateFormat
```
SimpleDateFormat inputDateFormat
```
    - outputDateFormat
```
String outputDateFormat
```
Package org.apache.hudi.utilities.perf
- Class org.apache.hudi.utilities.perf.TimelineServerPerf extends Object implements Serializable
  - Serialized Fields
    - cfg
```
TimelineServerPerf.Config cfg
```
    - useExternalTimelineServer
```
boolean useExternalTimelineServer
```
    - hostAddr
```
String hostAddr
```
- Class org.apache.hudi.utilities.perf.TimelineServerPerf.Config extends Object implements Serializable
  - Serialized Fields
    - basePath
```
String basePath
```
    - reportDir
```
String reportDir
```
    - maxPartitions
```
Integer maxPartitions
```
    - numExecutors
```
Integer numExecutors
```
    - numCoresPerExecutor
```
Integer numCoresPerExecutor
```
    - numIterations
```
Integer numIterations
```
    - sparkMaster
```
String sparkMaster
```
    - serverPort
```
Integer serverPort
```
    - serverHost
```
String serverHost
```
    - viewStorageType
```
FileSystemViewStorageType viewStorageType
```
    - maxViewMemPerTableInMB
```
Integer maxViewMemPerTableInMB
```
    - memFractionForCompactionPerTable
```
Double memFractionForCompactionPerTable
```
    - baseStorePathForFileGroups
```
String baseStorePathForFileGroups
```
    - rocksDBPath
```
String rocksDBPath
```
    - waitForManualQueries
```
Boolean waitForManualQueries
```
    - help
```
Boolean help
```
Package org.apache.hudi.utilities.schema
- Class org.apache.hudi.utilities.schema.FilebasedSchemaProvider extends SchemaProvider implements Serializable
  - Serialized Fields
    - fs
```
org.apache.hadoop.fs.FileSystem fs
```
    - sourceSchema
```
org.apache.avro.Schema sourceSchema
```
    - targetSchema
```
org.apache.avro.Schema targetSchema
```
- Class org.apache.hudi.utilities.schema.NullTargetSchemaRegistryProvider extends SchemaRegistryProvider implements Serializable
- Class org.apache.hudi.utilities.schema.RowBasedSchemaProvider extends SchemaProvider implements Serializable
  - Serialized Fields
    - rowStruct
```
org.apache.spark.sql.types.StructType rowStruct
```
- Class org.apache.hudi.utilities.schema.SchemaProvider extends Object implements Serializable
  - Serialized Fields
    - config
```
TypedProperties config
```
    - jssc
```
org.apache.spark.api.java.JavaSparkContext jssc
```
- Class org.apache.hudi.utilities.schema.SchemaRegistryProvider extends SchemaProvider implements Serializable
  - Serialized Fields
    - schema
```
org.apache.avro.Schema schema
```
    - targetSchema
```
org.apache.avro.Schema targetSchema
```
Package org.apache.hudi.utilities.sources
- Class org.apache.hudi.utilities.sources.AvroDFSSource extends AvroSource implements Serializable
  - Serialized Fields
    - pathSelector
```
DFSPathSelector pathSelector
```
- Class org.apache.hudi.utilities.sources.AvroKafkaSource extends AvroSource implements Serializable
  - Serialized Fields
    - offsetGen
```
KafkaOffsetGen offsetGen
```
- Class org.apache.hudi.utilities.sources.AvroSource extends Source<org.apache.spark.api.java.JavaRDD<org.apache.avro.generic.GenericRecord>> implements Serializable
- Class org.apache.hudi.utilities.sources.HiveIncrPullSource extends AvroSource implements Serializable
  - Serialized Fields
    - incrPullRootPath
```
String incrPullRootPath
```
- Class org.apache.hudi.utilities.sources.HoodieIncrSource extends RowSource implements Serializable
- Class org.apache.hudi.utilities.sources.JsonDFSSource extends JsonSource implements Serializable
  - Serialized Fields
    - pathSelector
```
DFSPathSelector pathSelector
```
- Class org.apache.hudi.utilities.sources.JsonKafkaSource extends JsonSource implements Serializable
  - Serialized Fields
    - offsetGen
```
KafkaOffsetGen offsetGen
```
- Class org.apache.hudi.utilities.sources.JsonSource extends Source<org.apache.spark.api.java.JavaRDD<String>> implements Serializable
- Class org.apache.hudi.utilities.sources.RowSource extends Source<org.apache.spark.sql.Dataset<org.apache.spark.sql.Row>> implements Serializable
- Class org.apache.hudi.utilities.sources.Source extends Object implements Serializable
  - Serialized Fields
    - sourceType
```
Source.SourceType sourceType
```
Package org.apache.hudi.utilities.sources.helpers
- Class org.apache.hudi.utilities.sources.helpers.AvroConvertor extends Object implements Serializable
  - Serialized Fields
    - schemaStr
```
String schemaStr
```

Serialized Form

Package org.apache.hudi.utilities

Class org.apache.hudi.utilities.HDFSParquetImporter extends Object implements Serializable

Serialized Fields

cfg

props

Class org.apache.hudi.utilities.HDFSParquetImporter.Config extends Object implements Serializable

Serialized Fields

command

srcPath

targetPath

tableName

tableType

rowKey

partitionKey

parallelism

schemaFile

format

sparkMaster

sparkMemory

retry

propsFilePath

configs

help

Class org.apache.hudi.utilities.HiveIncrementalPuller.Config extends Object implements Serializable

Serialized Fields

hiveJDBCUrl

hiveUsername

hivePassword

yarnQueueName

hoodieTmpDir

incrementalSQLFile

sourceDb

sourceTable

targetDb

targetTable

tmpDb

fromCommitTime

maxCommits

help

Class org.apache.hudi.utilities.HoodieCleaner.Config extends Object implements Serializable

Serialized Fields

basePath

propsFilePath

configs

sparkMaster

help

Class org.apache.hudi.utilities.HoodieCompactionAdminTool.Config extends Object implements Serializable

Serialized Fields

operation

basePath

compactionInstantTime

partitionPath

fileId

parallelism

sparkMaster

sparkMemory

dryRun

skipValidation

outputPath

printOutput

help

Class org.apache.hudi.utilities.HoodieCompactor.Config extends Object implements Serializable

Serialized Fields

basePath

tableName

compactionInstantTime

parallelism

schemaFile

sparkMaster

sparkMemory

retry

runSchedule

strategyClassName

help

propsFilePath

configs

Class org.apache.hudi.utilities.HoodieSnapshotCopier extends Object implements Serializable

Class org.apache.hudi.utilities.HoodieWithTimelineServer extends Object implements Serializable

Serialized Fields