HoodieCommitMetadata (hudi-common 0.15.0 API)

java.lang.Object
- org.apache.hudi.common.model.HoodieCommitMetadata

All Implemented Interfaces:

Serializable

Direct Known Subclasses:

HoodieReplaceCommitMetadata
```
public class HoodieCommitMetadata
extends Object
implements Serializable
```
All the metadata that gets stored along with a commit.

See Also:

Serialized Form

Field Summary

Fields
Modifier and Type	Field and Description
`protected Boolean`	`compacted`
`protected Map<String,String>`	`extraMetadata`
`protected WriteOperationType`	`operationType`
`protected Map<String,List<HoodieWriteStat>>`	`partitionToWriteStats`
`static String`	`SCHEMA_KEY`

Constructor Summary

Constructors
Constructor and Description

HoodieCommitMetadata()

HoodieCommitMetadata(boolean compacted)

Constructors
Constructor and Description
`HoodieCommitMetadata()`
`HoodieCommitMetadata(boolean compacted)`

Method Summary

All Methods Static Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`void`	`addMetadata(String metaKey, String value)`
`void`	`addWriteStat(String partitionPath, HoodieWriteStat stat)`
`boolean`	`equals(Object o)`
`long`	`fetchTotalBytesWritten()`
`long`	`fetchTotalFilesInsert()`
`long`	`fetchTotalFilesUpdated()`
`long`	`fetchTotalInsertRecordsWritten()`
`long`	`fetchTotalPartitionsWritten()`
`long`	`fetchTotalRecordsWritten()`
`long`	`fetchTotalUpdateRecordsWritten()`
`long`	`fetchTotalWriteErrors()`
`static <T> T`	`fromBytes(byte[] bytes, Class<T> clazz)`
`static <T> T`	`fromJsonString(String jsonStr, Class<T> clazz)`
`Boolean`	`getCompacted()`
`Map<String,String>`	`getExtraMetadata()`
`Map<HoodieFileGroupId,String>`	`getFileGroupIdAndFullPaths(String basePath)`
`HashMap<String,String>`	`getFileIdAndFullPaths(StoragePath basePath)`
`HashMap<String,String>`	`getFileIdAndRelativePaths()`
`Map<String,StoragePathInfo>`	`getFileIdToInfo(String basePath)` Extract the file status of all affected files from the commit metadata.
`static Option<Pair<String,List<String>>>`	`getFileSliceForFileGroupFromDeltaCommit(byte[] bytes, HoodieFileGroupId fileGroupId)` parse the bytes of deltacommit, and get the base file and the log files belonging to this provided file group.
`List<String>`	`getFullPathsByPartitionPath(String basePath, String partitionPath)`
`Map<String,StoragePathInfo>`	`getFullPathToInfo(HoodieStorage storage, String basePath)` Extract the file status of all affected files from the commit metadata.
`String`	`getMetadata(String metaKey)`
`Pair<Option<Long>,Option<Long>>`	`getMinAndMaxEventTime()`
`WriteOperationType`	`getOperationType()`
`Map<String,List<HoodieWriteStat>>`	`getPartitionToWriteStats()`
`Long`	`getTotalCompactedRecordsUpdated()`
`Long`	`getTotalCorruptLogBlocks()`
`Long`	`getTotalCreateTime()`
`Long`	`getTotalLogFilesCompacted()`
`Long`	`getTotalLogFilesSize()`
`Long`	`getTotalLogRecordsCompacted()`
`long`	`getTotalRecordsDeleted()`
`Long`	`getTotalRollbackLogBlocks()`
`Long`	`getTotalScanTime()`
`Long`	`getTotalUpsertTime()`
`HashSet<String>`	`getWritePartitionPaths()`
`List<HoodieWriteStat>`	`getWriteStats()`
`List<HoodieWriteStat>`	`getWriteStats(String partitionPath)`
`int`	`hashCode()`
`void`	`setCompacted(Boolean compacted)`
`void`	`setOperationType(WriteOperationType type)`
`String`	`toJsonString()`
`String`	`toString()`

Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Field Detail

SCHEMA_KEY
```
public static final String SCHEMA_KEY
```
See Also:

Constant Field Values

partitionToWriteStats

protected Map<String,List<HoodieWriteStat>> partitionToWriteStats

compacted
```
protected Boolean compacted
```

extraMetadata

protected Map<String,String> extraMetadata

operationType

protected WriteOperationType operationType

Constructor Detail

HoodieCommitMetadata
```
public HoodieCommitMetadata()
```

HoodieCommitMetadata

public HoodieCommitMetadata(boolean compacted)

Method Detail

addWriteStat

public void addWriteStat(String partitionPath,
                         HoodieWriteStat stat)

addMetadata

public void addMetadata(String metaKey,
                        String value)

getWriteStats

public List<HoodieWriteStat> getWriteStats(String partitionPath)

getExtraMetadata

public Map<String,String> getExtraMetadata()

getPartitionToWriteStats

public Map<String,List<HoodieWriteStat>> getPartitionToWriteStats()

getWriteStats

public List<HoodieWriteStat> getWriteStats()

getMetadata

public String getMetadata(String metaKey)

getCompacted
```
public Boolean getCompacted()
```

setCompacted

public void setCompacted(Boolean compacted)

getFileIdAndRelativePaths

public HashMap<String,String> getFileIdAndRelativePaths()

setOperationType

public void setOperationType(WriteOperationType type)

getOperationType

public WriteOperationType getOperationType()

getFileIdAndFullPaths

public HashMap<String,String> getFileIdAndFullPaths(StoragePath basePath)

getFullPathsByPartitionPath

public List<String> getFullPathsByPartitionPath(String basePath,
                                                String partitionPath)

getFileGroupIdAndFullPaths

public Map<HoodieFileGroupId,String> getFileGroupIdAndFullPaths(String basePath)

getFullPathToInfo
```
public Map<String,StoragePathInfo> getFullPathToInfo(HoodieStorage storage,
                                                     String basePath)
```
Extract the file status of all affected files from the commit metadata. If a file has been touched multiple times in the given commits, the return value will keep the one from the latest commit.

Parameters:

storage - HoodieStorage instance.

basePath - The base path

Returns:

the file full path to file status mapping

getFileIdToInfo
```
public Map<String,StoragePathInfo> getFileIdToInfo(String basePath)
```
Extract the file status of all affected files from the commit metadata. If a file has been touched multiple times in the given commits, the return value will keep the one from the latest commit by file group ID.
Note: different with getFullPathToInfo(HoodieStorage, String), only the latest commit file for a file group is returned, this is an optimization for COPY_ON_WRITE table to eliminate legacy files for filesystem view.

Parameters:

basePath - The base path

Returns:

the file ID to file status mapping

toJsonString

public String toJsonString()
                    throws IOException

Throws:: IOException

fromJsonString

public static <T> T fromJsonString(String jsonStr,
                                   Class<T> clazz)
                            throws Exception

Throws:: Exception

getFileSliceForFileGroupFromDeltaCommit

public static Option<Pair<String,List<String>>> getFileSliceForFileGroupFromDeltaCommit(byte[] bytes,
                                                                                        HoodieFileGroupId fileGroupId)

parse the bytes of deltacommit, and get the base file and the log files belonging to this provided file group.

fetchTotalPartitionsWritten

public long fetchTotalPartitionsWritten()

fetchTotalFilesInsert
```
public long fetchTotalFilesInsert()
```

fetchTotalFilesUpdated
```
public long fetchTotalFilesUpdated()
```

fetchTotalUpdateRecordsWritten

public long fetchTotalUpdateRecordsWritten()

fetchTotalInsertRecordsWritten

public long fetchTotalInsertRecordsWritten()

fetchTotalRecordsWritten

public long fetchTotalRecordsWritten()

fetchTotalBytesWritten
```
public long fetchTotalBytesWritten()
```

fetchTotalWriteErrors
```
public long fetchTotalWriteErrors()
```

getTotalRecordsDeleted
```
public long getTotalRecordsDeleted()
```

getTotalLogRecordsCompacted

public Long getTotalLogRecordsCompacted()

getTotalLogFilesCompacted

public Long getTotalLogFilesCompacted()

getTotalCompactedRecordsUpdated

public Long getTotalCompactedRecordsUpdated()

getTotalCorruptLogBlocks

public Long getTotalCorruptLogBlocks()

getTotalRollbackLogBlocks

public Long getTotalRollbackLogBlocks()

getTotalLogFilesSize
```
public Long getTotalLogFilesSize()
```

getTotalScanTime
```
public Long getTotalScanTime()
```

getTotalCreateTime
```
public Long getTotalCreateTime()
```

getTotalUpsertTime
```
public Long getTotalUpsertTime()
```

getMinAndMaxEventTime

public Pair<Option<Long>,Option<Long>> getMinAndMaxEventTime()

getWritePartitionPaths

public HashSet<String> getWritePartitionPaths()

equals
```
public boolean equals(Object o)
```
Overrides:

equals in class Object

hashCode
```
public int hashCode()
```
Overrides:

hashCode in class Object

fromBytes

public static <T> T fromBytes(byte[] bytes,
                              Class<T> clazz)
                       throws IOException

Throws:: IOException

toString
```
public String toString()
```
Overrides:

toString in class Object

Class HoodieCommitMetadata

Field Summary

Constructor Summary

Method Summary

Methods inherited from class java.lang.Object

Field Detail

SCHEMA_KEY

partitionToWriteStats

compacted

extraMetadata

operationType

Constructor Detail

HoodieCommitMetadata

HoodieCommitMetadata

Method Detail

addWriteStat

addMetadata

getWriteStats

getExtraMetadata

getPartitionToWriteStats

getWriteStats

getMetadata

getCompacted

setCompacted

getFileIdAndRelativePaths

setOperationType

getOperationType

getFileIdAndFullPaths

getFullPathsByPartitionPath

getFileGroupIdAndFullPaths

getFullPathToInfo

getFileIdToInfo

toJsonString

fromJsonString

getFileSliceForFileGroupFromDeltaCommit

fetchTotalPartitionsWritten

fetchTotalFilesInsert

fetchTotalFilesUpdated

fetchTotalUpdateRecordsWritten

fetchTotalInsertRecordsWritten

fetchTotalRecordsWritten

fetchTotalBytesWritten

fetchTotalWriteErrors

getTotalRecordsDeleted

getTotalLogRecordsCompacted

getTotalLogFilesCompacted

getTotalCompactedRecordsUpdated

getTotalCorruptLogBlocks

getTotalRollbackLogBlocks

getTotalLogFilesSize

getTotalScanTime

getTotalCreateTime

getTotalUpsertTime

getMinAndMaxEventTime

getWritePartitionPaths

equals

hashCode

fromBytes

toString