io.delta.kernel.internal

Class SnapshotImpl

Object

io.delta.kernel.internal.SnapshotImpl

All Implemented Interfaces:

Snapshot

public class SnapshotImpl
extends Object
implements Snapshot

Implementation of Snapshot.

Constructor Summary

Constructors

Constructor and Description
SnapshotImpl(Path dataPath, LogSegment logSegment, LogReplay logReplay, Protocol protocol, Metadata metadata)

Method Summary

Modifier and Type	Method and Description
CreateCheckpointIterator	getCreateCheckpointIterator(Engine engine)
Path	getDataPath()
java.util.Map<String,DomainMetadata>	getDomainMetadataMap() Get the domain metadata map from the log replay, which lazily loads and replays a history of domain metadata actions, resolving them to produce the current state of the domain metadata.
java.util.Optional<Long>	getLatestTransactionVersion(Engine engine, String applicationId) Get the latest transaction version for given applicationId.
Path	getLogPath()
LogSegment	getLogSegment()
Metadata	getMetadata()
java.util.List<String>	getPartitionColumnNames(Engine engine) Get the names of the partition columns in the Delta table at this snapshot.
Protocol	getProtocol()
ScanBuilder	getScanBuilder(Engine engine) Create a scan builder to construct a Scan to read data from this snapshot.
StructType	getSchema(Engine engine) Get the schema of the table at this snapshot.
java.util.Optional<TableCommitCoordinatorClientHandler>	getTableCommitCoordinatorClientHandlerOpt(Engine engine) Returns the commit coordinator client handler based on the table metadata in this snapshot.
long	getTimestamp(Engine engine) Get the timestamp (in milliseconds since the Unix epoch) of the latest commit in this Snapshot.
long	getVersion(Engine engine) Get the version of this snapshot in the table.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SnapshotImpl

public SnapshotImpl(Path dataPath,
                    LogSegment logSegment,
                    LogReplay logReplay,
                    Protocol protocol,
                    Metadata metadata)

Method Detail

getVersion

public long getVersion(Engine engine)

Description copied from interface: Snapshot

Get the version of this snapshot in the table.

Specified by:

getVersion in interface Snapshot

Parameters:

engine - Engine instance to use in Delta Kernel.

Returns:

version of this snapshot in the Delta table

getTimestamp

public long getTimestamp(Engine engine)

Get the timestamp (in milliseconds since the Unix epoch) of the latest commit in this Snapshot. If the table does not yet exist (i.e. this Snapshot is being used to create the new table), this method returns -1. Note that this -1 value will never be exposed to users - either they get a valid snapshot for a table or they get an exception.

When InCommitTimestampTableFeature is enabled, the timestamp is retrieved from the CommitInfo of the latest commit in this Snapshot, which can result in an IO operation.

For non-ICT tables, this is the same as the file modification time of the latest commit in this Snapshot.

Specified by:

getTimestamp in interface Snapshot

Parameters:

engine - the engine to use for IO operations

Returns:

the timestamp of the latest commit

getSchema

public StructType getSchema(Engine engine)

Description copied from interface: Snapshot

Get the schema of the table at this snapshot.

Specified by:

getSchema in interface Snapshot

Parameters:

engine - Engine instance to use in Delta Kernel.

Returns:

Schema of the Delta table at this snapshot.

getScanBuilder

public ScanBuilder getScanBuilder(Engine engine)

Description copied from interface: Snapshot

Create a scan builder to construct a Scan to read data from this snapshot.

Specified by:

getScanBuilder in interface Snapshot

Parameters:

engine - Engine instance to use in Delta Kernel.

Returns:

an instance of ScanBuilder

getLogPath

public Path getLogPath()

getDataPath

public Path getDataPath()

getProtocol

public Protocol getProtocol()

getPartitionColumnNames

public java.util.List<String> getPartitionColumnNames(Engine engine)

Description copied from interface: Snapshot

Get the names of the partition columns in the Delta table at this snapshot.

The partition column names are returned in the order they are defined in the Delta table schema. If the table does not define any partition columns, this method returns an empty list.

Specified by:

getPartitionColumnNames in interface Snapshot

Parameters:

engine - Engine instance to use in Delta Kernel.

Returns:

a list of partition column names, or an empty list if the table is not partitioned.

getDomainMetadataMap

public java.util.Map<String,DomainMetadata> getDomainMetadataMap()

Get the domain metadata map from the log replay, which lazily loads and replays a history of domain metadata actions, resolving them to produce the current state of the domain metadata.

Returns:

A map where the keys are domain names and the values are DomainMetadata objects.

getMetadata

public Metadata getMetadata()

getLogSegment

public LogSegment getLogSegment()

getCreateCheckpointIterator

public CreateCheckpointIterator getCreateCheckpointIterator(Engine engine)

getLatestTransactionVersion

public java.util.Optional<Long> getLatestTransactionVersion(Engine engine,
                                                            String applicationId)

Get the latest transaction version for given applicationId. This information comes from the transactions identifiers stored in Delta transaction log. This API is not a public API. For now keep this internal to enable Flink upgrade to use Kernel.

Parameters:

applicationId - Identifier of the application that put transaction identifiers in Delta transaction log

Returns:

Last transaction version or Optional.empty() if no transaction identifier exists for this application.

getTableCommitCoordinatorClientHandlerOpt

public java.util.Optional<TableCommitCoordinatorClientHandler> getTableCommitCoordinatorClientHandlerOpt(Engine engine)

Returns the commit coordinator client handler based on the table metadata in this snapshot.

Parameters:

engine - the engine to use for IO operations

Returns:

the commit coordinator client handler for this snapshot or empty if the metadata is not configured to use the commit coordinator.