io.delta.kernel.internal

Class TransactionImpl

Object

io.delta.kernel.internal.TransactionImpl

All Implemented Interfaces:

Transaction

public class TransactionImpl
extends Object
implements Transaction

Field Summary

Fields

Modifier and Type	Field and Description
static int	DEFAULT_READ_VERSION
static int	DEFAULT_WRITE_VERSION

Constructor Summary

Constructors

Constructor and Description
TransactionImpl(boolean isNewTable, Path dataPath, Path logPath, SnapshotImpl readSnapshot, String engineInfo, Operation operation, Protocol protocol, Metadata metadata, java.util.Optional<SetTransaction> setTxnOpt, boolean shouldUpdateMetadata, boolean shouldUpdateProtocol, Clock clock)

Method Summary

Modifier and Type	Method and Description
void	addDomainMetadatas(java.util.List<DomainMetadata> domainMetadatas) Internal API to add domain metadata actions for this transaction.
TransactionCommitResult	commit(Engine engine, CloseableIterable<Row> dataActions) Commit the transaction including the data action rows generated by Transaction.generateAppendActions(io.delta.kernel.engine.Engine, io.delta.kernel.data.Row, io.delta.kernel.utils.CloseableIterator<io.delta.kernel.utils.DataFileStatus>, io.delta.kernel.DataWriteContext).
java.util.List<DomainMetadata>	getDomainMetadatas()
java.util.List<String>	getPartitionColumns(Engine engine) Get the list of logical names of the partition columns.
StructType	getSchema(Engine engine) Get the schema of the table.
java.util.Optional<SetTransaction>	getSetTxnOpt()
static java.util.List<Column>	getStatisticsColumns(Engine engine, Row transactionState) Get the part of the schema of the table that needs the statistics to be collected per file.
Row	getTransactionState(Engine engine) Get the state of the transaction.
boolean	isBlindAppend()

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.delta.kernel.Transaction

generateAppendActions, getWriteContext, transformLogicalData

Field Detail

DEFAULT_READ_VERSION

public static final int DEFAULT_READ_VERSION

See Also:

Constant Field Values

DEFAULT_WRITE_VERSION

public static final int DEFAULT_WRITE_VERSION

See Also:

Constant Field Values

Constructor Detail

TransactionImpl

public TransactionImpl(boolean isNewTable,
                       Path dataPath,
                       Path logPath,
                       SnapshotImpl readSnapshot,
                       String engineInfo,
                       Operation operation,
                       Protocol protocol,
                       Metadata metadata,
                       java.util.Optional<SetTransaction> setTxnOpt,
                       boolean shouldUpdateMetadata,
                       boolean shouldUpdateProtocol,
                       Clock clock)

Method Detail

getTransactionState

public Row getTransactionState(Engine engine)

Description copied from interface: Transaction

Get the state of the transaction. The state helps Kernel do the transformations to logical data according to the Delta protocol and table features enabled on the table. The engine should use this at the data writer task to transform the logical data that the engine wants to write to the table in to physical data that goes in data files using Transaction.transformLogicalData(Engine, Row, CloseableIterator, Map)

Specified by:

getTransactionState in interface Transaction

getPartitionColumns

public java.util.List<String> getPartitionColumns(Engine engine)

Description copied from interface: Transaction

Get the list of logical names of the partition columns. This helps the connector to do physical partitioning of the data before asking the Kernel to stage the data per partition.

Specified by:

getPartitionColumns in interface Transaction

getSchema

public StructType getSchema(Engine engine)

Description copied from interface: Transaction

Get the schema of the table. If the connector is adding any data to the table through this transaction, it should have the same schema as the table schema.

Specified by:

getSchema in interface Transaction

getSetTxnOpt

public java.util.Optional<SetTransaction> getSetTxnOpt()

addDomainMetadatas

public void addDomainMetadatas(java.util.List<DomainMetadata> domainMetadatas)

Internal API to add domain metadata actions for this transaction. Visible for testing.

Parameters:

domainMetadatas - List of domain metadata to be added to the transaction.

getDomainMetadatas

public java.util.List<DomainMetadata> getDomainMetadatas()

commit

public TransactionCommitResult commit(Engine engine,
                                      CloseableIterable<Row> dataActions)
                               throws ConcurrentWriteException

Description copied from interface: Transaction

Commit the transaction including the data action rows generated by Transaction.generateAppendActions(io.delta.kernel.engine.Engine, io.delta.kernel.data.Row, io.delta.kernel.utils.CloseableIterator<io.delta.kernel.utils.DataFileStatus>, io.delta.kernel.DataWriteContext).

Specified by:

commit in interface Transaction

Parameters:

engine - Engine instance.

dataActions - Iterable of data actions to commit. These data actions are generated by the Transaction.generateAppendActions(Engine, Row, CloseableIterator, DataWriteContext). The CloseableIterable allows the Kernel to access the list of actions multiple times (in case of retries to resolve the conflicts due to other writers to the table). Kernel provides a in-memory based implementation of CloseableIterable with utility API CloseableIterable.inMemoryIterable(CloseableIterator)

Returns:

TransactionCommitResult status of the successful transaction.

Throws:

ConcurrentWriteException - when the transaction has encountered a non-retryable conflicts or exceeded the maximum number of retries reached. The connector needs to rerun the query on top of the latest table state and retry the transaction.

isBlindAppend

public boolean isBlindAppend()

getStatisticsColumns

public static java.util.List<Column> getStatisticsColumns(Engine engine,
                                                          Row transactionState)

Get the part of the schema of the table that needs the statistics to be collected per file.

Parameters:

engine - Engine instance to use.

transactionState - State of the transaction

Returns: