io.delta.kernel.internal.actions

Class CommitInfo

Object

io.delta.kernel.internal.actions.CommitInfo

public class CommitInfo
extends Object

Delta log action representing a commit information action. According to the Delta protocol there isn't any specific schema for this action, but we use the following schema:

• inCommitTimestamp: Long - A monotonically increasing timestamp that represents the time since epoch in milliseconds when the commit write was started
• timestamp: Long - Milliseconds since epoch UTC of when this commit happened
• engineInfo: String - Engine that made this commit
• operation: String - Operation (e.g. insert, delete, merge etc.)
• operationParameters: Map(String, String) - each operation depending upon the type may add zero or more parameters about the operation. E.g. when creating a table `partitionBy` key with list of partition columns is added.
• isBlindAppend: Boolean - Is this commit a blind append?
• txnId: String - a unique transaction id of this commit

The Delta-Spark connector adds lot more fields to this action. We can add them as needed.

Field Summary

Fields

Modifier and Type	Field and Description
static StructType	FULL_SCHEMA

Constructor Summary

Constructors

Constructor and Description
CommitInfo(java.util.Optional<Long> inCommitTimestamp, long timestamp, String engineInfo, String operation, java.util.Map<String,String> operationParameters, boolean isBlindAppend, String txnId, java.util.Map<String,String> operationMetrics)

Method Summary

Modifier and Type	Method and Description
static CommitInfo	fromColumnVector(ColumnVector vector, int rowId)
static java.util.Optional<CommitInfo>	getCommitInfoOpt(Engine engine, Path logPath, long version) Get the persisted commit info (if available) for the given delta file.
String	getEngineInfo()
java.util.Optional<Long>	getInCommitTimestamp()
String	getOperation()
static long	getRequiredInCommitTimestamp(java.util.Optional<CommitInfo> commitInfoOpt, String version, Path dataPath) Returns the `inCommitTimestamp` of the given `commitInfoOpt` if it is defined.
long	getTimestamp()
void	setInCommitTimestamp(java.util.Optional<Long> inCommitTimestamp)
Row	toRow() Encode as a Row object with the schema FULL_SCHEMA.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

FULL_SCHEMA

public static StructType FULL_SCHEMA

Constructor Detail

CommitInfo

public CommitInfo(java.util.Optional<Long> inCommitTimestamp,
                  long timestamp,
                  String engineInfo,
                  String operation,
                  java.util.Map<String,String> operationParameters,
                  boolean isBlindAppend,
                  String txnId,
                  java.util.Map<String,String> operationMetrics)

Method Detail

fromColumnVector

public static CommitInfo fromColumnVector(ColumnVector vector,
                                          int rowId)

getInCommitTimestamp

public java.util.Optional<Long> getInCommitTimestamp()

setInCommitTimestamp

public void setInCommitTimestamp(java.util.Optional<Long> inCommitTimestamp)

getTimestamp

public long getTimestamp()

getEngineInfo

public String getEngineInfo()

getOperation

public String getOperation()

toRow

public Row toRow()

Encode as a Row object with the schema FULL_SCHEMA.

Returns:

Row object with the schema FULL_SCHEMA

getRequiredInCommitTimestamp

public static long getRequiredInCommitTimestamp(java.util.Optional<CommitInfo> commitInfoOpt,
                                                String version,
                                                Path dataPath)

Returns the `inCommitTimestamp` of the given `commitInfoOpt` if it is defined. Throws an exception if `commitInfoOpt` is empty or contains an empty `inCommitTimestamp`.

getCommitInfoOpt

public static java.util.Optional<CommitInfo> getCommitInfoOpt(Engine engine,
                                                              Path logPath,
                                                              long version)

Get the persisted commit info (if available) for the given delta file.