Uses of Class
org.apache.hudi.common.util.Option

Packages that use Option

Package	Description
org.apache.hudi
org.apache.hudi.avro
org.apache.hudi.common
org.apache.hudi.common.config
org.apache.hudi.common.data
org.apache.hudi.common.engine
org.apache.hudi.common.fs
org.apache.hudi.common.model
org.apache.hudi.common.model.debezium
org.apache.hudi.common.table
org.apache.hudi.common.table.log
org.apache.hudi.common.table.log.block
org.apache.hudi.common.table.timeline
org.apache.hudi.common.table.view
org.apache.hudi.common.util
org.apache.hudi.common.util.queue
org.apache.hudi.internal.schema.io
org.apache.hudi.internal.schema.utils
org.apache.hudi.io.storage
org.apache.hudi.metadata

Uses of Option in org.apache.hudi

Methods in org.apache.hudi that return Option

Modifier and Type	Method and Description
Option<org.apache.hadoop.fs.FileStatus[]>	BaseHoodieTableFileIndex.FileStatusCache.get(org.apache.hadoop.fs.Path path)
Option<HoodieInstant>	BaseHoodieTableFileIndex.getLatestCompletedInstant() Returns latest completed instant as seen by this instance of the file-index

Constructors in org.apache.hudi with parameters of type Option

Constructor and Description
BaseHoodieTableFileIndex(HoodieEngineContext engineContext, HoodieTableMetaClient metaClient, TypedProperties configProperties, HoodieTableQueryType queryType, List<org.apache.hadoop.fs.Path> queryPaths, Option<String> specifiedQueryInstant, boolean shouldIncludePendingCommits, boolean shouldValidateInstant, BaseHoodieTableFileIndex.FileStatusCache fileStatusCache)

Uses of Option in org.apache.hudi.avro

Methods in org.apache.hudi.avro that return Option

Modifier and Type	Method and Description
static Option<String>	HoodieAvroUtils.getNullableValAsString(org.apache.avro.generic.GenericRecord rec, String fieldName) Returns the string value of the given record rec and field fieldName.

Constructors in org.apache.hudi.avro with parameters of type Option

Constructor and Description
HoodieAvroWriteSupport(org.apache.parquet.schema.MessageType schema, org.apache.avro.Schema avroSchema, Option<BloomFilter> bloomFilterOpt)

Uses of Option in org.apache.hudi.common

Methods in org.apache.hudi.common that return Option

Modifier and Type	Method and Description
Option<org.apache.avro.generic.IndexedRecord>	HoodieJsonPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord oldRec, org.apache.avro.Schema schema)
Option<org.apache.avro.generic.IndexedRecord>	HoodieJsonPayload.getInsertValue(org.apache.avro.Schema schema)

Methods in org.apache.hudi.common with parameters of type Option

Modifier and Type	Method and Description
HoodieCleanStat.Builder	HoodieCleanStat.Builder.withEarliestCommitRetained(Option<HoodieInstant> earliestCommitToRetain)

Uses of Option in org.apache.hudi.common.config

Methods in org.apache.hudi.common.config that return Option

Modifier and Type	Method and Description
Option<String>	ConfigProperty.getDeprecatedVersion()
Option<String>	ConfigProperty.getSinceVersion()

Method parameters in org.apache.hudi.common.config with type arguments of type Option

Modifier and Type	Method and Description
ConfigProperty<T>	ConfigProperty.withInferFunction(Function<HoodieConfig,Option<T>> inferFunction)

Uses of Option in org.apache.hudi.common.data

Methods in org.apache.hudi.common.data that return types with arguments of type Option

Modifier and Type	Method and Description
<W> HoodiePairData<K,Pair<V,Option<W>>>	HoodieListPairData.leftOuterJoin(HoodiePairData<K,W> other)
<W> HoodiePairData<K,Pair<V,Option<W>>>	HoodiePairData.leftOuterJoin(HoodiePairData<K,W> other) Performs a left outer join of this dataset against other.

Uses of Option in org.apache.hudi.common.engine

Methods in org.apache.hudi.common.engine that return Option

Modifier and Type	Method and Description
abstract Option<String>	TaskContextSupplier.getProperty(EngineProperty prop)
abstract Option<String>	HoodieEngineContext.getProperty(EngineProperty key)
Option<String>	HoodieLocalEngineContext.getProperty(EngineProperty key)
Option<String>	LocalTaskContextSupplier.getProperty(EngineProperty prop)

Uses of Option in org.apache.hudi.common.fs

Methods in org.apache.hudi.common.fs that return Option

Modifier and Type	Method and Description
static Option<HoodieLogFile>	FSUtils.getLatestLogFile(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path partitionPath, String fileId, String logFileExtension, String baseCommitTime) Get the latest log file for the passed in file-id in the partition path
static Option<Pair<Integer,String>>	FSUtils.getLatestLogVersion(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path partitionPath, String fileId, String logFileExtension, String baseCommitTime) Get the latest log version for the fileId in the partition path.

Methods in org.apache.hudi.common.fs with parameters of type Option

Modifier and Type	Method and Description
void	HoodieWrapperFileSystem.createImmutableFileInPath(org.apache.hadoop.fs.Path fullPath, Option<byte[]> content) Creates a new file with overwrite set to false.

Uses of Option in org.apache.hudi.common.model

Methods in org.apache.hudi.common.model that return Option

Modifier and Type	Method and Description
static Option<org.apache.hadoop.fs.Path>	HoodiePartitionMetadata.baseFormatMetaPathIfExists(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path partitionPath)
Option<org.apache.avro.generic.IndexedRecord>	HoodieAvroPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema)
Option<org.apache.avro.generic.IndexedRecord>	OverwriteWithLatestAvroPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema)
Option<org.apache.avro.generic.IndexedRecord>	AWSDmsAvroPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema)
Option<org.apache.avro.generic.IndexedRecord>	OverwriteNonDefaultsWithLatestAvroPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema)
Option<org.apache.avro.generic.IndexedRecord>	RewriteAvroPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema)
Option<org.apache.avro.generic.IndexedRecord>	HoodieRecordPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema) Deprecated.
Option<org.apache.avro.generic.IndexedRecord>	EmptyHoodieRecordPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema)
Option<org.apache.avro.generic.IndexedRecord>	EventTimeAvroPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema, Properties properties)
Option<org.apache.avro.generic.IndexedRecord>	AWSDmsAvroPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema, Properties properties)
Option<org.apache.avro.generic.IndexedRecord>	DefaultHoodieRecordPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema, Properties properties)
default Option<org.apache.avro.generic.IndexedRecord>	HoodieRecordPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema, Properties properties) This methods lets you write custom merging/combining logic to produce new values as a function of current value on storage and whats contained in this object.
Option<HoodieBaseFile>	FileSlice.getBaseFile()
Option<HoodieBaseFile>	CompactionOperation.getBaseFile(String basePath, String partitionPath)
Option<BaseFile>	HoodieBaseFile.getBootstrapBaseFile()
Option<String>	CompactionOperation.getBootstrapFilePath()
Option<Map<String,HoodieColumnRangeMetadata<Comparable>>>	HoodieDeltaWriteStat.getColumnStats()
Option<String>	CompactionOperation.getDataFileCommitTime()
Option<String>	CompactionOperation.getDataFileName()
Option<org.apache.avro.generic.IndexedRecord>	HoodieAvroPayload.getInsertValue(org.apache.avro.Schema schema)
Option<org.apache.avro.generic.IndexedRecord>	OverwriteWithLatestAvroPayload.getInsertValue(org.apache.avro.Schema schema)
Option<org.apache.avro.generic.IndexedRecord>	AWSDmsAvroPayload.getInsertValue(org.apache.avro.Schema schema)
Option<org.apache.avro.generic.IndexedRecord>	RewriteAvroPayload.getInsertValue(org.apache.avro.Schema schema)
Option<org.apache.avro.generic.IndexedRecord>	HoodieRecordPayload.getInsertValue(org.apache.avro.Schema schema) Deprecated.
Option<org.apache.avro.generic.IndexedRecord>	EmptyHoodieRecordPayload.getInsertValue(org.apache.avro.Schema schema)
Option<org.apache.avro.generic.IndexedRecord>	EventTimeAvroPayload.getInsertValue(org.apache.avro.Schema schema, Properties properties)
Option<org.apache.avro.generic.IndexedRecord>	AWSDmsAvroPayload.getInsertValue(org.apache.avro.Schema schema, Properties properties)
Option<org.apache.avro.generic.IndexedRecord>	DefaultHoodieRecordPayload.getInsertValue(org.apache.avro.Schema schema, Properties properties)
default Option<org.apache.avro.generic.IndexedRecord>	HoodieRecordPayload.getInsertValue(org.apache.avro.Schema schema, Properties properties) Generates an avro record out of the given HoodieRecordPayload, to be written out to storage.
Option<HoodieBaseFile>	HoodieFileGroup.getLatestDataFile() Gets the latest data file.
Option<FileSlice>	HoodieFileGroup.getLatestFileSlice() Gets the latest slice - this can contain either.
Option<FileSlice>	HoodieFileGroup.getLatestFileSliceBefore(String maxInstantTime) Obtain the latest file slice, upto an instantTime i.e < maxInstantTime.
Option<FileSlice>	HoodieFileGroup.getLatestFileSliceBeforeOrOn(String maxInstantTime) Obtain the latest file slice, upto a instantTime i.e <= maxInstantTime.
Option<FileSlice>	HoodieFileGroup.getLatestFileSliceInRange(List<String> commitRange)
Option<FileSlice>	HoodieFileGroup.getLatestFileSlicesIncludingInflight() Get the latest file slices including inflight ones.
Option<HoodieLogFile>	FileSlice.getLatestLogFile()
Option<Map<String,String>>	EventTimeAvroPayload.getMetadata()
Option<Map<String,String>>	DefaultHoodieRecordPayload.getMetadata()
default Option<Map<String,String>>	HoodieRecordPayload.getMetadata() This method can be used to extract some metadata from HoodieRecordPayload.
Option<HoodieRecordLocation>	HoodieRecord.getNewLocation()
static Option<org.apache.hadoop.fs.Path>	HoodiePartitionMetadata.getPartitionMetafilePath(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path partitionPath) Returns the name of the partition metadata.
Option<String>	HoodiePartitionMetadata.readPartitionCreatedCommitTime() Read out the COMMIT_TIME_KEY metadata for this partition.
static Option<org.apache.hadoop.fs.Path>	HoodiePartitionMetadata.textFormatMetaPathIfExists(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path partitionPath)

Methods in org.apache.hudi.common.model that return types with arguments of type Option

Modifier and Type	Method and Description
Pair<Option<Long>,Option<Long>>	HoodieCommitMetadata.getMinAndMaxEventTime()
Pair<Option<Long>,Option<Long>>	HoodieCommitMetadata.getMinAndMaxEventTime()

Methods in org.apache.hudi.common.model with parameters of type Option

Modifier and Type	Method and Description
static HoodieOperation	HoodieOperation.fromName(Option<String> nameOpt)

Constructors in org.apache.hudi.common.model with parameters of type Option

Constructor and Description
AWSDmsAvroPayload(Option<org.apache.avro.generic.GenericRecord> record)
CompactionOperation(Option<HoodieBaseFile> dataFile, String partitionPath, List<HoodieLogFile> logFiles, Map<String,Double> metrics)
CompactionOperation(String fileId, String partitionPath, String baseInstantTime, Option<String> dataFileCommitTime, List<String> deltaFileNames, Option<String> dataFileName, Option<String> bootstrapFilePath, Map<String,Double> metrics)
CompactionOperation(String fileId, String partitionPath, String baseInstantTime, Option<String> dataFileCommitTime, List<String> deltaFileNames, Option<String> dataFileName, Option<String> bootstrapFilePath, Map<String,Double> metrics)
CompactionOperation(String fileId, String partitionPath, String baseInstantTime, Option<String> dataFileCommitTime, List<String> deltaFileNames, Option<String> dataFileName, Option<String> bootstrapFilePath, Map<String,Double> metrics)
DefaultHoodieRecordPayload(Option<org.apache.avro.generic.GenericRecord> record)
EventTimeAvroPayload(Option<org.apache.avro.generic.GenericRecord> record)
HoodieAvroPayload(Option<org.apache.avro.generic.GenericRecord> record)
HoodiePartitionMetadata(org.apache.hadoop.fs.FileSystem fs, String instantTime, org.apache.hadoop.fs.Path basePath, org.apache.hadoop.fs.Path partitionPath, Option<HoodieFileFormat> format) Construct metadata object to be written out.
OverwriteNonDefaultsWithLatestAvroPayload(Option<org.apache.avro.generic.GenericRecord> record)
OverwriteWithLatestAvroPayload(Option<org.apache.avro.generic.GenericRecord> record)

Uses of Option in org.apache.hudi.common.model.debezium

Methods in org.apache.hudi.common.model.debezium that return Option

Modifier and Type	Method and Description
Option<org.apache.avro.generic.IndexedRecord>	AbstractDebeziumAvroPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema)
Option<org.apache.avro.generic.IndexedRecord>	PostgresDebeziumAvroPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema)
Option<org.apache.avro.generic.IndexedRecord>	PostgresDebeziumAvroPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord currentValue, org.apache.avro.Schema schema, Properties properties)
Option<org.apache.avro.generic.IndexedRecord>	AbstractDebeziumAvroPayload.getInsertValue(org.apache.avro.Schema schema)

Constructors in org.apache.hudi.common.model.debezium with parameters of type Option

Constructor and Description
AbstractDebeziumAvroPayload(Option<org.apache.avro.generic.GenericRecord> record)
MySqlDebeziumAvroPayload(Option<org.apache.avro.generic.GenericRecord> record)
PostgresDebeziumAvroPayload(Option<org.apache.avro.generic.GenericRecord> record)

Uses of Option in org.apache.hudi.common.table

Methods in org.apache.hudi.common.table that return Option

Modifier and Type	Method and Description
Option<String>	HoodieTableConfig.getBootstrapBasePath()
Option<String[]>	HoodieTableConfig.getPartitionFields()
Option<HoodieFileFormat>	HoodieTableConfig.getPartitionMetafileFormat() Returns the format to use for partition meta files.
Option<String[]>	HoodieTableConfig.getRecordKeyFields()
Option<org.apache.avro.Schema>	TableSchemaResolver.getTableAvroSchemaFromLatestCommit(boolean includeMetadataFields) Returns table's latest Avro Schema iff table is non-empty (ie there's at least a single commit) This method differs from TableSchemaResolver.getTableAvroSchema(boolean) in that it won't fallback to use table's schema used at creation
Option<org.apache.avro.Schema>	HoodieTableConfig.getTableCreateSchema()
Option<String>	TableSchemaResolver.getTableHistorySchemaStrFromCommitMetadata() Gets the history schemas as String for a hoodie table from the HoodieCommitMetadata of the instant.
Option<InternalSchema>	TableSchemaResolver.getTableInternalSchemaFromCommitMetadata() Gets the InternalSchema for a hoodie table from the HoodieCommitMetadata of the instant.
Option<InternalSchema>	TableSchemaResolver.getTableInternalSchemaFromCommitMetadata(String timestamp) Gets the InternalSchema for a hoodie table from the HoodieCommitMetadata of the instant.
Option<TimelineLayoutVersion>	HoodieTableConfig.getTimelineLayoutVersion()

Methods in org.apache.hudi.common.table with parameters of type Option

Modifier and Type	Method and Description
static org.apache.avro.Schema	TableSchemaResolver.appendPartitionColumns(org.apache.avro.Schema dataSchema, Option<String[]> partitionFields)
org.apache.parquet.schema.MessageType	TableSchemaResolver.readSchemaFromLastCompaction(Option<HoodieInstant> lastCompactionCommitOpt) Deprecated. please use TableSchemaResolver.getTableAvroSchema(HoodieInstant, boolean) instead
HoodieTableMetaClient.Builder	HoodieTableMetaClient.Builder.setLayoutVersion(Option<TimelineLayoutVersion> layoutVersion)

Constructors in org.apache.hudi.common.table with parameters of type Option

Constructor and Description
HoodieTableMetaClient(org.apache.hadoop.conf.Configuration conf, String basePath, boolean loadActiveTimelineOnLoad, ConsistencyGuardConfig consistencyGuardConfig, Option<TimelineLayoutVersion> layoutVersion, String payloadClassName, FileSystemRetryConfig fileSystemRetryConfig)

Uses of Option in org.apache.hudi.common.table.log

Fields in org.apache.hudi.common.table.log declared as Option

Modifier and Type	Field and Description
protected Option<InstantRange>	HoodieMergedLogRecordScanner.Builder.instantRange

Methods in org.apache.hudi.common.table.log that return Option

Modifier and Type	Method and Description
Option<String>	AbstractHoodieLogRecordReader.getPartitionName()

Methods in org.apache.hudi.common.table.log with parameters of type Option

Modifier and Type	Method and Description
protected HoodieAvroRecord<?>	AbstractHoodieLogRecordReader.createHoodieRecord(org.apache.avro.generic.IndexedRecord rec, HoodieTableConfig hoodieTableConfig, String payloadClassFQN, String preCombineField, boolean withOperationField, Option<Pair<String,String>> simpleKeyGenFields, Option<String> partitionName) Create @HoodieRecord from the @IndexedRecord.
protected HoodieAvroRecord<?>	AbstractHoodieLogRecordReader.createHoodieRecord(org.apache.avro.generic.IndexedRecord rec, HoodieTableConfig hoodieTableConfig, String payloadClassFQN, String preCombineField, boolean withOperationField, Option<Pair<String,String>> simpleKeyGenFields, Option<String> partitionName) Create @HoodieRecord from the @IndexedRecord.
protected void	AbstractHoodieLogRecordReader.scanInternal(Option<AbstractHoodieLogRecordReader.KeySpec> keySpecOpt)
AbstractHoodieLogRecordReader.Builder	AbstractHoodieLogRecordReader.Builder.withInstantRange(Option<InstantRange> instantRange)
HoodieMergedLogRecordScanner.Builder	HoodieMergedLogRecordScanner.Builder.withInstantRange(Option<InstantRange> instantRange)
HoodieUnMergedLogRecordScanner.Builder	HoodieUnMergedLogRecordScanner.Builder.withInstantRange(Option<InstantRange> instantRange)

Constructors in org.apache.hudi.common.table.log with parameters of type Option

Constructor and Description
AbstractHoodieLogRecordReader(org.apache.hadoop.fs.FileSystem fs, String basePath, List<String> logFilePaths, org.apache.avro.Schema readerSchema, String latestInstantTime, boolean readBlocksLazily, boolean reverseReader, int bufferSize, Option<InstantRange> instantRange, boolean withOperationField)
AbstractHoodieLogRecordReader(org.apache.hadoop.fs.FileSystem fs, String basePath, List<String> logFilePaths, org.apache.avro.Schema readerSchema, String latestInstantTime, boolean readBlocksLazily, boolean reverseReader, int bufferSize, Option<InstantRange> instantRange, boolean withOperationField, boolean forceFullScan, Option<String> partitionName, InternalSchema internalSchema)
AbstractHoodieLogRecordReader(org.apache.hadoop.fs.FileSystem fs, String basePath, List<String> logFilePaths, org.apache.avro.Schema readerSchema, String latestInstantTime, boolean readBlocksLazily, boolean reverseReader, int bufferSize, Option<InstantRange> instantRange, boolean withOperationField, boolean forceFullScan, Option<String> partitionName, InternalSchema internalSchema)
HoodieMergedLogRecordScanner(org.apache.hadoop.fs.FileSystem fs, String basePath, List<String> logFilePaths, org.apache.avro.Schema readerSchema, String latestInstantTime, Long maxMemorySizeInBytes, boolean readBlocksLazily, boolean reverseReader, int bufferSize, String spillableMapBasePath, Option<InstantRange> instantRange, ExternalSpillableMap.DiskMapType diskMapType, boolean isBitCaskDiskMapCompressionEnabled, boolean withOperationField, boolean forceFullScan, Option<String> partitionName, InternalSchema internalSchema)
HoodieMergedLogRecordScanner(org.apache.hadoop.fs.FileSystem fs, String basePath, List<String> logFilePaths, org.apache.avro.Schema readerSchema, String latestInstantTime, Long maxMemorySizeInBytes, boolean readBlocksLazily, boolean reverseReader, int bufferSize, String spillableMapBasePath, Option<InstantRange> instantRange, ExternalSpillableMap.DiskMapType diskMapType, boolean isBitCaskDiskMapCompressionEnabled, boolean withOperationField, boolean forceFullScan, Option<String> partitionName, InternalSchema internalSchema)

Uses of Option in org.apache.hudi.common.table.log.block

Methods in org.apache.hudi.common.table.log.block that return Option

Modifier and Type	Method and Description
Option<HoodieLogBlock.HoodieLogBlockContentLocation>	HoodieLogBlock.getBlockContentLocation()
Option<byte[]>	HoodieLogBlock.getContent()
protected Option<org.apache.avro.Schema.Field>	HoodieDataBlock.getKeyField(org.apache.avro.Schema schema)
protected Option<String>	HoodieDataBlock.getRecordKey(org.apache.avro.generic.IndexedRecord record)
static Option<byte[]>	HoodieLogBlock.tryReadContent(org.apache.hadoop.fs.FSDataInputStream inputStream, Integer contentLength, boolean readLazily) Read or Skip block content of a log block in the log file.

Constructors in org.apache.hudi.common.table.log.block with parameters of type Option

Constructor and Description
HoodieAvroDataBlock(org.apache.hadoop.fs.FSDataInputStream inputStream, Option<byte[]> content, boolean readBlockLazily, HoodieLogBlock.HoodieLogBlockContentLocation logBlockContentLocation, Option<org.apache.avro.Schema> readerSchema, Map<HoodieLogBlock.HeaderMetadataType,String> header, Map<HoodieLogBlock.HeaderMetadataType,String> footer, String keyField)
HoodieAvroDataBlock(org.apache.hadoop.fs.FSDataInputStream inputStream, Option<byte[]> content, boolean readBlockLazily, HoodieLogBlock.HoodieLogBlockContentLocation logBlockContentLocation, Option<org.apache.avro.Schema> readerSchema, Map<HoodieLogBlock.HeaderMetadataType,String> header, Map<HoodieLogBlock.HeaderMetadataType,String> footer, String keyField)
HoodieAvroDataBlock(org.apache.hadoop.fs.FSDataInputStream inputStream, Option<byte[]> content, boolean readBlockLazily, HoodieLogBlock.HoodieLogBlockContentLocation logBlockContentLocation, Option<org.apache.avro.Schema> readerSchema, Map<HoodieLogBlock.HeaderMetadataType,String> header, Map<HoodieLogBlock.HeaderMetadataType,String> footer, String keyField, InternalSchema internalSchema)
HoodieAvroDataBlock(org.apache.hadoop.fs.FSDataInputStream inputStream, Option<byte[]> content, boolean readBlockLazily, HoodieLogBlock.HoodieLogBlockContentLocation logBlockContentLocation, Option<org.apache.avro.Schema> readerSchema, Map<HoodieLogBlock.HeaderMetadataType,String> header, Map<HoodieLogBlock.HeaderMetadataType,String> footer, String keyField, InternalSchema internalSchema)
HoodieCommandBlock(Option<byte[]> content, org.apache.hadoop.fs.FSDataInputStream inputStream, boolean readBlockLazily, Option<HoodieLogBlock.HoodieLogBlockContentLocation> blockContentLocation, Map<HoodieLogBlock.HeaderMetadataType,String> header, Map<HoodieLogBlock.HeaderMetadataType,String> footer)
HoodieCommandBlock(Option<byte[]> content, org.apache.hadoop.fs.FSDataInputStream inputStream, boolean readBlockLazily, Option<HoodieLogBlock.HoodieLogBlockContentLocation> blockContentLocation, Map<HoodieLogBlock.HeaderMetadataType,String> header, Map<HoodieLogBlock.HeaderMetadataType,String> footer)
HoodieCorruptBlock(Option<byte[]> corruptedBytes, org.apache.hadoop.fs.FSDataInputStream inputStream, boolean readBlockLazily, Option<HoodieLogBlock.HoodieLogBlockContentLocation> blockContentLocation, Map<HoodieLogBlock.HeaderMetadataType,String> header, Map<HoodieLogBlock.HeaderMetadataType,String> footer)
HoodieCorruptBlock(Option<byte[]> corruptedBytes, org.apache.hadoop.fs.FSDataInputStream inputStream, boolean readBlockLazily, Option<HoodieLogBlock.HoodieLogBlockContentLocation> blockContentLocation, Map<HoodieLogBlock.HeaderMetadataType,String> header, Map<HoodieLogBlock.HeaderMetadataType,String> footer)
HoodieDataBlock(Option<byte[]> content, org.apache.hadoop.fs.FSDataInputStream inputStream, boolean readBlockLazily, Option<HoodieLogBlock.HoodieLogBlockContentLocation> blockContentLocation, Option<org.apache.avro.Schema> readerSchema, Map<HoodieLogBlock.HeaderMetadataType,String> headers, Map<HoodieLogBlock.HeaderMetadataType,String> footer, String keyFieldName, boolean enablePointLookups) NOTE: This ctor is used on the write-path (ie when records ought to be written into the log)
HoodieDataBlock(Option<byte[]> content, org.apache.hadoop.fs.FSDataInputStream inputStream, boolean readBlockLazily, Option<HoodieLogBlock.HoodieLogBlockContentLocation> blockContentLocation, Option<org.apache.avro.Schema> readerSchema, Map<HoodieLogBlock.HeaderMetadataType,String> headers, Map<HoodieLogBlock.HeaderMetadataType,String> footer, String keyFieldName, boolean enablePointLookups) NOTE: This ctor is used on the write-path (ie when records ought to be written into the log)
HoodieDataBlock(Option<byte[]> content, org.apache.hadoop.fs.FSDataInputStream inputStream, boolean readBlockLazily, Option<HoodieLogBlock.HoodieLogBlockContentLocation> blockContentLocation, Option<org.apache.avro.Schema> readerSchema, Map<HoodieLogBlock.HeaderMetadataType,String> headers, Map<HoodieLogBlock.HeaderMetadataType,String> footer, String keyFieldName, boolean enablePointLookups) NOTE: This ctor is used on the write-path (ie when records ought to be written into the log)
HoodieDataBlock(Option<byte[]> content, org.apache.hadoop.fs.FSDataInputStream inputStream, boolean readBlockLazily, Option<HoodieLogBlock.HoodieLogBlockContentLocation> blockContentLocation, Option<org.apache.avro.Schema> readerSchema, Map<HoodieLogBlock.HeaderMetadataType,String> headers, Map<HoodieLogBlock.HeaderMetadataType,String> footer, String keyFieldName, boolean enablePointLookups, InternalSchema internalSchema)
HoodieDataBlock(Option<byte[]> content, org.apache.hadoop.fs.FSDataInputStream inputStream, boolean readBlockLazily, Option<HoodieLogBlock.HoodieLogBlockContentLocation> blockContentLocation, Option<org.apache.avro.Schema> readerSchema, Map<HoodieLogBlock.HeaderMetadataType,String> headers, Map<HoodieLogBlock.HeaderMetadataType,String> footer, String keyFieldName, boolean enablePointLookups, InternalSchema internalSchema)
HoodieDataBlock(Option<byte[]> content, org.apache.hadoop.fs.FSDataInputStream inputStream, boolean readBlockLazily, Option<HoodieLogBlock.HoodieLogBlockContentLocation> blockContentLocation, Option<org.apache.avro.Schema> readerSchema, Map<HoodieLogBlock.HeaderMetadataType,String> headers, Map<HoodieLogBlock.HeaderMetadataType,String> footer, String keyFieldName, boolean enablePointLookups, InternalSchema internalSchema)
HoodieDeleteBlock(Option<byte[]> content, org.apache.hadoop.fs.FSDataInputStream inputStream, boolean readBlockLazily, Option<HoodieLogBlock.HoodieLogBlockContentLocation> blockContentLocation, Map<HoodieLogBlock.HeaderMetadataType,String> header, Map<HoodieLogBlock.HeaderMetadataType,String> footer)
HoodieDeleteBlock(Option<byte[]> content, org.apache.hadoop.fs.FSDataInputStream inputStream, boolean readBlockLazily, Option<HoodieLogBlock.HoodieLogBlockContentLocation> blockContentLocation, Map<HoodieLogBlock.HeaderMetadataType,String> header, Map<HoodieLogBlock.HeaderMetadataType,String> footer)
HoodieHFileDataBlock(org.apache.hadoop.fs.FSDataInputStream inputStream, Option<byte[]> content, boolean readBlockLazily, HoodieLogBlock.HoodieLogBlockContentLocation logBlockContentLocation, Option<org.apache.avro.Schema> readerSchema, Map<HoodieLogBlock.HeaderMetadataType,String> header, Map<HoodieLogBlock.HeaderMetadataType,String> footer, boolean enablePointLookups, org.apache.hadoop.fs.Path pathForReader)
HoodieHFileDataBlock(org.apache.hadoop.fs.FSDataInputStream inputStream, Option<byte[]> content, boolean readBlockLazily, HoodieLogBlock.HoodieLogBlockContentLocation logBlockContentLocation, Option<org.apache.avro.Schema> readerSchema, Map<HoodieLogBlock.HeaderMetadataType,String> header, Map<HoodieLogBlock.HeaderMetadataType,String> footer, boolean enablePointLookups, org.apache.hadoop.fs.Path pathForReader)
HoodieLogBlock(Map<HoodieLogBlock.HeaderMetadataType,String> logBlockHeader, Map<HoodieLogBlock.HeaderMetadataType,String> logBlockFooter, Option<HoodieLogBlock.HoodieLogBlockContentLocation> blockContentLocation, Option<byte[]> content, org.apache.hadoop.fs.FSDataInputStream inputStream, boolean readBlockLazily)
HoodieLogBlock(Map<HoodieLogBlock.HeaderMetadataType,String> logBlockHeader, Map<HoodieLogBlock.HeaderMetadataType,String> logBlockFooter, Option<HoodieLogBlock.HoodieLogBlockContentLocation> blockContentLocation, Option<byte[]> content, org.apache.hadoop.fs.FSDataInputStream inputStream, boolean readBlockLazily)
HoodieParquetDataBlock(org.apache.hadoop.fs.FSDataInputStream inputStream, Option<byte[]> content, boolean readBlockLazily, HoodieLogBlock.HoodieLogBlockContentLocation logBlockContentLocation, Option<org.apache.avro.Schema> readerSchema, Map<HoodieLogBlock.HeaderMetadataType,String> header, Map<HoodieLogBlock.HeaderMetadataType,String> footer, String keyField)
HoodieParquetDataBlock(org.apache.hadoop.fs.FSDataInputStream inputStream, Option<byte[]> content, boolean readBlockLazily, HoodieLogBlock.HoodieLogBlockContentLocation logBlockContentLocation, Option<org.apache.avro.Schema> readerSchema, Map<HoodieLogBlock.HeaderMetadataType,String> header, Map<HoodieLogBlock.HeaderMetadataType,String> footer, String keyField)

Uses of Option in org.apache.hudi.common.table.timeline

Fields in org.apache.hudi.common.table.timeline with type parameters of type Option

Modifier and Type	Field and Description
protected Function<HoodieInstant,Option<byte[]>>	HoodieDefaultTimeline.details

Methods in org.apache.hudi.common.table.timeline that return Option

Modifier and Type	Method and Description
Option<HoodieInstant>	HoodieDefaultTimeline.firstInstant()
Option<HoodieInstant>	HoodieTimeline.firstInstant()
Option<HoodieInstant>	HoodieDefaultTimeline.firstInstant(String action, HoodieInstant.State state)
Option<HoodieInstant>	HoodieTimeline.firstInstant(String action, HoodieInstant.State state)
static Option<String>	TimelineUtils.getExtraMetadataFromLatest(HoodieTableMetaClient metaClient, String extraMetadataKey) Get extra metadata for specified key from latest commit/deltacommit/replacecommit(eg.
static Option<String>	TimelineUtils.getExtraMetadataFromLatestIncludeClustering(HoodieTableMetaClient metaClient, String extraMetadataKey) Get extra metadata for specified key from latest commit/deltacommit/replacecommit instant including internal commits such as clustering.
Option<HoodieInstant>	HoodieDefaultTimeline.getFirstNonSavepointCommit()
Option<HoodieInstant>	HoodieTimeline.getFirstNonSavepointCommit() First non-savepoint commit in the active data timeline.
Option<byte[]>	HoodieActiveTimeline.getInstantDetails(HoodieInstant instant)
Option<byte[]>	HoodieDefaultTimeline.getInstantDetails(HoodieInstant instant)
Option<byte[]>	HoodieTimeline.getInstantDetails(HoodieInstant instant) Read the completed instant details.
Option<byte[]>	HoodieArchivedTimeline.getInstantDetails(HoodieInstant instant)
Option<Pair<HoodieInstant,HoodieCommitMetadata>>	HoodieActiveTimeline.getLastCommitMetadataWithValidData() Get the last instant with valid data, and convert this to HoodieCommitMetadata
Option<Pair<HoodieInstant,HoodieCommitMetadata>>	HoodieActiveTimeline.getLastCommitMetadataWithValidSchema() Returns most recent instant having valid schema in its HoodieCommitMetadata
Option<HoodieInstant>	HoodieDefaultTimeline.lastInstant()
Option<HoodieInstant>	HoodieTimeline.lastInstant()
Option<HoodieInstant>	HoodieDefaultTimeline.nthFromLastInstant(int n)
Option<HoodieInstant>	HoodieTimeline.nthFromLastInstant(int n)
Option<HoodieInstant>	HoodieDefaultTimeline.nthInstant(int n)
Option<HoodieInstant>	HoodieTimeline.nthInstant(int n)
static Option<Date>	HoodieActiveTimeline.parseDateFromInstantTimeSafely(String timestamp) The same parsing method as above, but this method will mute ParseException.
Option<byte[]>	HoodieActiveTimeline.readCleanerInfoAsBytes(HoodieInstant instant)
Option<byte[]>	HoodieActiveTimeline.readCompactionPlanAsBytes(HoodieInstant instant)
Option<byte[]>	HoodieActiveTimeline.readIndexPlanAsBytes(HoodieInstant instant)
Option<byte[]>	HoodieActiveTimeline.readRestoreInfoAsBytes(HoodieInstant instant)
Option<byte[]>	HoodieActiveTimeline.readRollbackInfoAsBytes(HoodieInstant instant)
static <T extends org.apache.avro.specific.SpecificRecordBase> Option<byte[]>	TimelineMetadataUtils.serializeAvroMetadata(T metadata, Class<T> clazz)
static Option<byte[]>	TimelineMetadataUtils.serializeCleanerPlan(HoodieCleanerPlan cleanPlan)
static Option<byte[]>	TimelineMetadataUtils.serializeCleanMetadata(HoodieCleanMetadata metadata)
static Option<byte[]>	TimelineMetadataUtils.serializeCompactionPlan(HoodieCompactionPlan compactionWorkload)
static Option<byte[]>	TimelineMetadataUtils.serializeIndexCommitMetadata(HoodieIndexCommitMetadata indexCommitMetadata)
static Option<byte[]>	TimelineMetadataUtils.serializeIndexPlan(HoodieIndexPlan indexPlan)
static Option<byte[]>	TimelineMetadataUtils.serializeRequestedReplaceMetadata(HoodieRequestedReplaceMetadata clusteringPlan)
static Option<byte[]>	TimelineMetadataUtils.serializeRestoreMetadata(HoodieRestoreMetadata restoreMetadata)
static Option<byte[]>	TimelineMetadataUtils.serializeRestorePlan(HoodieRestorePlan restorePlan)
static Option<byte[]>	TimelineMetadataUtils.serializeRollbackMetadata(HoodieRollbackMetadata rollbackMetadata)
static Option<byte[]>	TimelineMetadataUtils.serializeRollbackPlan(HoodieRollbackPlan rollbackPlan)
static Option<byte[]>	TimelineMetadataUtils.serializeSavepointMetadata(HoodieSavepointMetadata metadata)

Methods in org.apache.hudi.common.table.timeline that return types with arguments of type Option

Modifier and Type	Method and Description
static Map<String,Option<String>>	TimelineUtils.getAllExtraMetadataForKey(HoodieTableMetaClient metaClient, String extraMetadataKey) Get extra metadata for specified key from all active commit/deltacommit instants.

Methods in org.apache.hudi.common.table.timeline with parameters of type Option

Modifier and Type	Method and Description
static HoodieRollbackMetadata	TimelineMetadataUtils.convertRollbackMetadata(String startRollbackTime, Option<Long> durationInMs, List<HoodieInstant> instants, List<HoodieRollbackStat> rollbackStats)
protected void	HoodieActiveTimeline.createFileInMetaPath(String filename, Option<byte[]> content, boolean allowOverwrite)
void	HoodieActiveTimeline.saveAsComplete(HoodieInstant instant, Option<byte[]> data)
void	HoodieActiveTimeline.saveToCleanRequested(HoodieInstant instant, Option<byte[]> content)
void	HoodieActiveTimeline.saveToCompactionRequested(HoodieInstant instant, Option<byte[]> content)
void	HoodieActiveTimeline.saveToCompactionRequested(HoodieInstant instant, Option<byte[]> content, boolean overwrite)
void	HoodieActiveTimeline.saveToPendingIndexAction(HoodieInstant instant, Option<byte[]> content) Save content for inflight/requested index instant.
void	HoodieActiveTimeline.saveToPendingReplaceCommit(HoodieInstant instant, Option<byte[]> content) Saves content for requested REPLACE instant.
void	HoodieActiveTimeline.saveToRestoreRequested(HoodieInstant instant, Option<byte[]> content)
void	HoodieActiveTimeline.saveToRollbackRequested(HoodieInstant instant, Option<byte[]> content)
HoodieInstant	HoodieActiveTimeline.transitionCleanInflightToComplete(HoodieInstant inflightInstant, Option<byte[]> data) Transition Clean State from inflight to Committed.
HoodieInstant	HoodieActiveTimeline.transitionCleanRequestedToInflight(HoodieInstant requestedInstant, Option<byte[]> data) Transition Clean State from requested to inflight.
HoodieInstant	HoodieActiveTimeline.transitionCompactionInflightToComplete(HoodieInstant inflightInstant, Option<byte[]> data) Transition Compaction State from inflight to Committed.
HoodieInstant	HoodieActiveTimeline.transitionIndexInflightToComplete(HoodieInstant inflightInstant, Option<byte[]> data) Transition index instant state from inflight to completed.
HoodieInstant	HoodieActiveTimeline.transitionIndexRequestedToInflight(HoodieInstant requestedInstant, Option<byte[]> data) Transition index instant state from requested to inflight.
HoodieInstant	HoodieActiveTimeline.transitionReplaceInflightToComplete(HoodieInstant inflightInstant, Option<byte[]> data) Transition replace inflight to Committed.
HoodieInstant	HoodieActiveTimeline.transitionReplaceRequestedToInflight(HoodieInstant requestedInstant, Option<byte[]> data) Transition replace requested file to replace inflight.
void	HoodieActiveTimeline.transitionRequestedToInflight(HoodieInstant requested, Option<byte[]> content)
void	HoodieActiveTimeline.transitionRequestedToInflight(HoodieInstant requested, Option<byte[]> content, boolean allowRedundantTransitions)
HoodieInstant	HoodieActiveTimeline.transitionRollbackInflightToComplete(HoodieInstant inflightInstant, Option<byte[]> data) Transition Rollback State from inflight to Committed.
protected void	HoodieActiveTimeline.transitionState(HoodieInstant fromInstant, HoodieInstant toInstant, Option<byte[]> data, boolean allowRedundantTransitions)

Constructor parameters in org.apache.hudi.common.table.timeline with type arguments of type Option

Constructor and Description
HoodieDefaultTimeline(Stream<HoodieInstant> instants, Function<HoodieInstant,Option<byte[]>> details)

Uses of Option in org.apache.hudi.common.table.view

Methods in org.apache.hudi.common.table.view that return Option

Modifier and Type	Method and Description
protected Option<HoodieBaseFile>	RocksDbBasedFileSystemView.fetchLatestBaseFile(String partitionPath, String fileId)
protected Option<HoodieBaseFile>	AbstractTableFileSystemView.fetchLatestBaseFile(String partitionPath, String fileId) Default implementation for fetching latest base-file.
protected Option<FileSlice>	RocksDbBasedFileSystemView.fetchLatestFileSlice(String partitionPath, String fileId)
protected Option<FileSlice>	AbstractTableFileSystemView.fetchLatestFileSlice(String partitionPath, String fileId) Default implementation for fetching file-slice.
Option<HoodieBaseFile>	TableFileSystemView.BaseFileOnlyView.getBaseFileOn(String partitionPath, String instantTime, String fileId) Get the version of data file matching the instant time in the given partition.
Option<HoodieBaseFile>	AbstractTableFileSystemView.getBaseFileOn(String partitionStr, String instantTime, String fileId)
Option<HoodieBaseFile>	PriorityBasedFileSystemView.getBaseFileOn(String partitionPath, String instantTime, String fileId)
Option<HoodieBaseFile>	RemoteHoodieTableFileSystemView.getBaseFileOn(String partitionPath, String instantTime, String fileId)
protected Option<BootstrapBaseFileMapping>	HoodieTableFileSystemView.getBootstrapBaseFile(HoodieFileGroupId fileGroupId)
protected Option<BootstrapBaseFileMapping>	RocksDbBasedFileSystemView.getBootstrapBaseFile(HoodieFileGroupId fileGroupId)
protected abstract Option<BootstrapBaseFileMapping>	AbstractTableFileSystemView.getBootstrapBaseFile(HoodieFileGroupId fileGroupId) Return pending compaction operation for a file-group.
Option<HoodieInstant>	TableFileSystemView.getLastInstant() Last Known Instant on which the view is built.
Option<HoodieInstant>	AbstractTableFileSystemView.getLastInstant()
Option<HoodieInstant>	PriorityBasedFileSystemView.getLastInstant()
Option<HoodieInstant>	RemoteHoodieTableFileSystemView.getLastInstant()
protected Option<HoodieBaseFile>	AbstractTableFileSystemView.getLatestBaseFile(HoodieFileGroup fileGroup)
Option<HoodieBaseFile>	TableFileSystemView.BaseFileOnlyViewWithLatestSlice.getLatestBaseFile(String partitionPath, String fileId) Get Latest data file for a partition and file-Id.
Option<HoodieBaseFile>	AbstractTableFileSystemView.getLatestBaseFile(String partitionStr, String fileId) Get Latest base file for a partition and file-Id.
Option<HoodieBaseFile>	PriorityBasedFileSystemView.getLatestBaseFile(String partitionPath, String fileId)
Option<HoodieBaseFile>	RemoteHoodieTableFileSystemView.getLatestBaseFile(String partitionPath, String fileId)
Option<FileSlice>	TableFileSystemView.SliceViewWithLatestSlice.getLatestFileSlice(String partitionPath, String fileId) Get Latest File Slice for a given fileId in a given partition.
Option<FileSlice>	AbstractTableFileSystemView.getLatestFileSlice(String partitionStr, String fileId) Get Latest File Slice for a given fileId in a given partition.
Option<FileSlice>	PriorityBasedFileSystemView.getLatestFileSlice(String partitionPath, String fileId)
Option<FileSlice>	RemoteHoodieTableFileSystemView.getLatestFileSlice(String partitionPath, String fileId)
protected Option<HoodieInstant>	HoodieTableFileSystemView.getPendingClusteringInstant(HoodieFileGroupId fgId)
protected Option<HoodieInstant>	RocksDbBasedFileSystemView.getPendingClusteringInstant(HoodieFileGroupId fgId)
protected abstract Option<HoodieInstant>	AbstractTableFileSystemView.getPendingClusteringInstant(HoodieFileGroupId fileGroupId) Get pending clustering instant time for specified file group.
protected Option<Pair<String,CompactionOperation>>	HoodieTableFileSystemView.getPendingCompactionOperationWithInstant(HoodieFileGroupId fgId)
protected Option<Pair<String,CompactionOperation>>	RocksDbBasedFileSystemView.getPendingCompactionOperationWithInstant(HoodieFileGroupId fgId)
protected abstract Option<Pair<String,CompactionOperation>>	AbstractTableFileSystemView.getPendingCompactionOperationWithInstant(HoodieFileGroupId fileGroupId) Return pending compaction operation for a file-group.
protected Option<HoodieInstant>	HoodieTableFileSystemView.getReplaceInstant(HoodieFileGroupId fileGroupId)
protected Option<HoodieInstant>	RocksDbBasedFileSystemView.getReplaceInstant(HoodieFileGroupId fileGroupId)
protected abstract Option<HoodieInstant>	AbstractTableFileSystemView.getReplaceInstant(HoodieFileGroupId fileGroupId) Track instant time for file groups replaced.

Uses of Option in org.apache.hudi.common.util

Methods in org.apache.hudi.common.util that return Option

Modifier and Type	Method and Description
static <T> Option<T>	Option.empty()
<U> Option<U>	Option.flatMap(Function<? super T,Option<U>> mapper)
static <T> Option<T>	Option.fromJavaOptional(Optional<T> v) Convert from java.util.Optional.
static Option<Pair<HoodieInstant,HoodieClusteringPlan>>	ClusteringUtils.getClusteringPlan(HoodieTableMetaClient metaClient, HoodieInstant pendingReplaceInstant) Get Clustering plan from timeline.
static Option<Pair<HoodieTimeline,HoodieInstant>>	CompactionUtils.getDeltaCommitsSinceLatestCompaction(HoodieActiveTimeline activeTimeline) Returns a pair of (timeline containing the delta commits after the latest completed compaction commit, the completed compaction commit instant), if the latest completed compaction commit is present; a pair of (timeline containing all the delta commits, the first delta commit instant), if there is no completed compaction commit.
static Option<Pair<HoodieTimeline,HoodieInstant>>	CompactionUtils.getDeltaCommitsSinceLatestCompactionRequest(HoodieActiveTimeline activeTimeline)
static Option<HoodieCommitMetadata>	CommitUtils.getLatestCommitMetadataWithValidCheckpointInfo(HoodieTimeline timeline, String checkpointKey) Process previous commits metadata in the timeline to determine the checkpoint given a checkpoint key.
static Option<HoodieInstant>	CompactionUtils.getOldestInstantToRetainForCompaction(HoodieActiveTimeline activeTimeline, int maxDeltaCommits) Gets the oldest instant to retain for MOR compaction.
static Option<org.apache.hadoop.fs.Path>	TablePathUtils.getTablePath(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path path)
<U> Option<U>	Option.map(Function<? super T,? extends U> mapper)
static <T> Option<T>	Option.of(T value)
static <T> Option<T>	Option.ofNullable(T value)
Option<T>	Option.or(Supplier<? extends Option<T>> other) Returns this Option if not empty, otherwise evaluates the provided supplier and returns the alternative
static Option<byte[]>	FileIOUtils.readDataFromPath(org.apache.hadoop.fs.FileSystem fileSystem, org.apache.hadoop.fs.Path detailPath)
static Option<byte[]>	FileIOUtils.readDataFromPath(org.apache.hadoop.fs.FileSystem fileSystem, org.apache.hadoop.fs.Path detailPath, boolean ignoreIOE)
static Option<MarkerType>	MarkerUtils.readMarkerType(org.apache.hadoop.fs.FileSystem fileSystem, String markerDir) Reads the marker type from `MARKERS.type` file.

Methods in org.apache.hudi.common.util that return types with arguments of type Option

Modifier and Type	Method and Description
static Pair<Option<String>,Option<String>>	InternalSchemaCache.getInternalSchemaAndAvroSchemaForClusteringAndCompaction(HoodieTableMetaClient metaClient, String compactionAndClusteringInstant) Get internalSchema and avroSchema for compaction/cluster operation.
static Pair<Option<String>,Option<String>>	InternalSchemaCache.getInternalSchemaAndAvroSchemaForClusteringAndCompaction(HoodieTableMetaClient metaClient, String compactionAndClusteringInstant) Get internalSchema and avroSchema for compaction/cluster operation.

Methods in org.apache.hudi.common.util with parameters of type Option

Modifier and Type	Method and Description
static HoodieCompactionOperation	CompactionUtils.buildFromFileSlice(String partitionPath, FileSlice fileSlice, Option<Function<Pair<String,FileSlice>,Map<String,Double>>> metricsCaptureFunction) Generate compaction operation from file-slice.
static HoodieCompactionPlan	CompactionUtils.buildFromFileSlices(List<Pair<String,FileSlice>> partitionFileSlicePairs, Option<Map<String,String>> extraMetadata, Option<Function<Pair<String,FileSlice>,Map<String,Double>>> metricsCaptureFunction) Generate compaction plan from file-slices.
static HoodieCompactionPlan	CompactionUtils.buildFromFileSlices(List<Pair<String,FileSlice>> partitionFileSlicePairs, Option<Map<String,String>> extraMetadata, Option<Function<Pair<String,FileSlice>,Map<String,Double>>> metricsCaptureFunction) Generate compaction plan from file-slices.
static HoodieCommitMetadata	CommitUtils.buildMetadata(List<HoodieWriteStat> writeStats, Map<String,List<String>> partitionToReplaceFileIds, Option<Map<String,String>> extraMetadata, WriteOperationType operationType, String schemaToStoreInCommit, String commitActionType)
static HoodieCleanMetadata	CleanerUtils.convertCleanMetadata(String startCleanTime, Option<Long> durationInMs, List<HoodieCleanStat> cleanStats)
static <R> R	SpillableMapUtils.convertToHoodieRecordPayload(org.apache.avro.generic.GenericRecord record, String payloadClazz, String preCombineField, boolean withOperationField, Option<String> partitionName)
static <R> R	SpillableMapUtils.convertToHoodieRecordPayload(org.apache.avro.generic.GenericRecord record, String payloadClazz, String preCombineField, Pair<String,String> recordKeyPartitionPathFieldPair, boolean withOperationField, Option<String> partitionName) Utility method to convert bytes to HoodieRecord using schema and payload class.
static void	FileIOUtils.createFileInPath(org.apache.hadoop.fs.FileSystem fileSystem, org.apache.hadoop.fs.Path fullPath, Option<byte[]> content)
static void	FileIOUtils.createFileInPath(org.apache.hadoop.fs.FileSystem fileSystem, org.apache.hadoop.fs.Path fullPath, Option<byte[]> content, boolean ignoreIOE)
abstract List<HoodieKey>	BaseFileUtils.fetchHoodieKeys(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path filePath, Option<BaseKeyGenerator> keyGeneratorOpt) Fetch HoodieKeys from the given data file.
List<HoodieKey>	ParquetUtils.fetchHoodieKeys(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path filePath, Option<BaseKeyGenerator> keyGeneratorOpt) Fetch HoodieKeys from the given parquet file.
List<HoodieKey>	OrcUtils.fetchHoodieKeys(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path filePath, Option<BaseKeyGenerator> keyGeneratorOpt)
abstract ClosableIterator<HoodieKey>	BaseFileUtils.getHoodieKeyIterator(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path filePath, Option<BaseKeyGenerator> keyGeneratorOpt) Provides a closable iterator for reading the given data file.
ClosableIterator<HoodieKey>	ParquetUtils.getHoodieKeyIterator(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path filePath, Option<BaseKeyGenerator> keyGeneratorOpt) Returns a closable iterator for reading the given parquet file.
ClosableIterator<HoodieKey>	OrcUtils.getHoodieKeyIterator(org.apache.hadoop.conf.Configuration configuration, org.apache.hadoop.fs.Path filePath, Option<BaseKeyGenerator> keyGeneratorOpt)

Method parameters in org.apache.hudi.common.util with type arguments of type Option

Modifier and Type	Method and Description
<U> Option<U>	Option.flatMap(Function<? super T,Option<U>> mapper)
Option<T>	Option.or(Supplier<? extends Option<T>> other) Returns this Option if not empty, otherwise evaluates the provided supplier and returns the alternative

Uses of Option in org.apache.hudi.common.util.queue

Constructors in org.apache.hudi.common.util.queue with parameters of type Option

Constructor and Description
BoundedInMemoryExecutor(long bufferLimitInBytes, BoundedInMemoryQueueProducer<I> producer, Option<BoundedInMemoryQueueConsumer<O,E>> consumer, Function<I,O> transformFunction)
BoundedInMemoryExecutor(long bufferLimitInBytes, BoundedInMemoryQueueProducer<I> producer, Option<BoundedInMemoryQueueConsumer<O,E>> consumer, Function<I,O> transformFunction, Runnable preExecuteRunnable)
BoundedInMemoryExecutor(long bufferLimitInBytes, List<BoundedInMemoryQueueProducer<I>> producers, Option<BoundedInMemoryQueueConsumer<O,E>> consumer, Function<I,O> transformFunction, SizeEstimator<O> sizeEstimator, Runnable preExecuteRunnable)

Uses of Option in org.apache.hudi.internal.schema.io

Methods in org.apache.hudi.internal.schema.io that return Option

Modifier and Type	Method and Description
Option<InternalSchema>	FileBasedInternalSchemaStorageManager.getSchemaByKey(String versionId)

Uses of Option in org.apache.hudi.internal.schema.utils

Methods in org.apache.hudi.internal.schema.utils that return Option

Modifier and Type	Method and Description
static Option<InternalSchema>	SerDeHelper.fromJson(String json) Convert string to internalSchema.

Uses of Option in org.apache.hudi.io.storage

Methods in org.apache.hudi.io.storage that return Option

Modifier and Type	Method and Description
default Option<R>	HoodieFileReader.getRecordByKey(String key)
Option<R>	HoodieHFileReader.getRecordByKey(String key, org.apache.avro.Schema readerSchema)
default Option<R>	HoodieFileReader.getRecordByKey(String key, org.apache.avro.Schema readerSchema)

Constructors in org.apache.hudi.io.storage with parameters of type Option

Constructor and Description
HoodieHFileReader(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path dummyPath, byte[] content, Option<org.apache.avro.Schema> schemaOpt)
HoodieHFileReader(org.apache.hadoop.fs.Path path, org.apache.hadoop.hbase.io.hfile.HFile.Reader reader, Option<org.apache.avro.Schema> schemaOpt)

Uses of Option in org.apache.hudi.metadata

Fields in org.apache.hudi.metadata declared as Option

Modifier and Type	Field and Description
protected Option<HoodieMetadataMetrics>	BaseTableMetadata.metrics

Methods in org.apache.hudi.metadata that return Option

Modifier and Type	Method and Description
Option<org.apache.avro.generic.IndexedRecord>	HoodieMetadataPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord oldRecord, org.apache.avro.Schema schema)
Option<org.apache.avro.generic.IndexedRecord>	HoodieMetadataPayload.combineAndGetUpdateValue(org.apache.avro.generic.IndexedRecord oldRecord, org.apache.avro.Schema schema, Properties properties)
Option<BloomFilter>	FileSystemBackedTableMetadata.getBloomFilter(String partitionName, String fileName)
Option<BloomFilter>	BaseTableMetadata.getBloomFilter(String partitionName, String fileName)
Option<BloomFilter>	HoodieTableMetadata.getBloomFilter(String partitionName, String fileName) Get the bloom filter for the FileID from the metadata table.
Option<HoodieMetadataBloomFilter>	HoodieMetadataPayload.getBloomFilterMetadata() Get the bloom filter metadata from this payload.
Option<HoodieMetadataColumnStats>	HoodieMetadataPayload.getColumnStatMetadata() Get the bloom filter metadata from this payload.
Option<org.apache.avro.generic.IndexedRecord>	HoodieMetadataPayload.getInsertValue(org.apache.avro.Schema schema)
Option<org.apache.avro.generic.IndexedRecord>	HoodieMetadataPayload.getInsertValue(org.apache.avro.Schema schemaIgnored, Properties propertiesIgnored)
Option<String>	FileSystemBackedTableMetadata.getLatestCompactionTime()
Option<String>	HoodieBackedTableMetadata.getLatestCompactionTime()
Option<String>	HoodieTableMetadata.getLatestCompactionTime() Returns the timestamp of the latest compaction.
protected Option<HoodieRecord<HoodieMetadataPayload>>	HoodieBackedTableMetadata.getRecordByKey(String key, String partitionName)
protected abstract Option<HoodieRecord<HoodieMetadataPayload>>	BaseTableMetadata.getRecordByKey(String key, String partitionName)
Option<String>	FileSystemBackedTableMetadata.getSyncedInstantTime()
Option<String>	HoodieBackedTableMetadata.getSyncedInstantTime()
Option<String>	HoodieTableMetadata.getSyncedInstantTime() Get the instant time to which the metadata is synced w.r.t data timeline.

Methods in org.apache.hudi.metadata that return types with arguments of type Option

Modifier and Type	Method and Description
List<Pair<String,Option<HoodieRecord<HoodieMetadataPayload>>>>	HoodieMetadataMergedLogRecordReader.getRecordByKey(String key) Retrieve a record given its key.
List<Pair<String,Option<HoodieRecord<HoodieMetadataPayload>>>>	HoodieMetadataMergedLogRecordReader.getRecordsByKeys(List<String> keys)
List<Pair<String,Option<HoodieRecord<HoodieMetadataPayload>>>>	HoodieBackedTableMetadata.getRecordsByKeys(List<String> keys, String partitionName)
abstract List<Pair<String,Option<HoodieRecord<HoodieMetadataPayload>>>>	BaseTableMetadata.getRecordsByKeys(List<String> key, String partitionName)

Methods in org.apache.hudi.metadata with parameters of type Option

Modifier and Type	Method and Description
static Map<MetadataPartitionType,HoodieData<HoodieRecord>>	HoodieTableMetadataUtil.convertMetadataToRecords(HoodieEngineContext engineContext, HoodieActiveTimeline metadataTableTimeline, HoodieRestoreMetadata restoreMetadata, MetadataRecordsGenerationParams recordsGenerationParams, String instantTime, Option<String> lastSyncTs) Convert restore action metadata to metadata table records.
static Map<MetadataPartitionType,HoodieData<HoodieRecord>>	HoodieTableMetadataUtil.convertMetadataToRecords(HoodieEngineContext engineContext, HoodieActiveTimeline metadataTableTimeline, HoodieRollbackMetadata rollbackMetadata, MetadataRecordsGenerationParams recordsGenerationParams, String instantTime, Option<String> lastSyncTs, boolean wasSynced) Convert rollback action metadata to metadata table records.
protected HoodieAvroRecord<?>	HoodieMetadataMergedLogRecordReader.createHoodieRecord(org.apache.avro.generic.IndexedRecord rec, HoodieTableConfig hoodieTableConfig, String payloadClassFQN, String preCombineField, boolean withOperationField, Option<Pair<String,String>> simpleKeyGenFields, Option<String> partitionName)
protected HoodieAvroRecord<?>	HoodieMetadataMergedLogRecordReader.createHoodieRecord(org.apache.avro.generic.IndexedRecord rec, HoodieTableConfig hoodieTableConfig, String payloadClassFQN, String preCombineField, boolean withOperationField, Option<Pair<String,String>> simpleKeyGenFields, Option<String> partitionName)
static HoodieRecord<HoodieMetadataPayload>	HoodieMetadataPayload.createPartitionFilesRecord(String partition, Option<Map<String,Long>> filesAdded, Option<List<String>> filesDeleted) Create and return a HoodieMetadataPayload to save list of files within a partition.
static HoodieRecord<HoodieMetadataPayload>	HoodieMetadataPayload.createPartitionFilesRecord(String partition, Option<Map<String,Long>> filesAdded, Option<List<String>> filesDeleted) Create and return a HoodieMetadataPayload to save list of files within a partition.
Pair<HoodieMetadataMergedLogRecordReader,Long>	HoodieBackedTableMetadata.getLogRecordScanner(List<HoodieLogFile> logFiles, String partitionName, Option<Boolean> allowFullScanOverride)
static int	HoodieTableMetadataUtil.getPartitionFileGroupCount(MetadataPartitionType partitionType, Option<HoodieTableMetaClient> metaClient, Option<HoodieTableFileSystemView> fsView, HoodieMetadataConfig metadataConfig, boolean isBootstrapCompleted) Get file group count for a metadata table partition.
static int	HoodieTableMetadataUtil.getPartitionFileGroupCount(MetadataPartitionType partitionType, Option<HoodieTableMetaClient> metaClient, Option<HoodieTableFileSystemView> fsView, HoodieMetadataConfig metadataConfig, boolean isBootstrapCompleted) Get file group count for a metadata table partition.
static List<FileSlice>	HoodieTableMetadataUtil.getPartitionLatestFileSlices(HoodieTableMetaClient metaClient, Option<HoodieTableFileSystemView> fsView, String partition) Get the latest file slices for a Metadata Table partition.
static List<FileSlice>	HoodieTableMetadataUtil.getPartitionLatestFileSlicesIncludingInflight(HoodieTableMetaClient metaClient, Option<HoodieTableFileSystemView> fileSystemView, String partition) Get the latest file slices for a given partition including the inflight ones.

Constructors in org.apache.hudi.metadata with parameters of type Option

Constructor and Description
HoodieMetadataPayload(Option<org.apache.avro.generic.GenericRecord> recordOpt)