io.delta.kernel.internal.snapshot

Class SnapshotManager

Object

io.delta.kernel.internal.snapshot.SnapshotManager

public class SnapshotManager
extends Object

Constructor Summary

Constructors

Constructor and Description
SnapshotManager(Path logPath, Path tablePath)

Method Summary

Modifier and Type	Method and Description
Snapshot	buildLatestSnapshot(Engine engine) Construct the latest snapshot for given table.
void	checkpoint(Engine engine, long version)
java.util.Optional<LogSegment>	getLogSegmentForVersion(Engine engine, java.util.Optional<Long> startCheckpoint, java.util.Optional<Long> versionToLoad) Get a list of files that can be used to compute a Snapshot at version `versionToLoad`, If `versionToLoad` is not provided, will generate the list of files that are needed to load the latest version of the Delta table.
Snapshot	getSnapshotAt(Engine engine, long version) Construct the snapshot for the given table at the version provided.
Snapshot	getSnapshotForTimestamp(Engine engine, long millisSinceEpochUTC) Construct the snapshot for the given table at the provided timestamp.
static void	verifyDeltaVersions(java.util.List<Long> versions, java.util.Optional<Long> expectedStartVersion, java.util.Optional<Long> expectedEndVersion) - Verify the versions are contiguous.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SnapshotManager

public SnapshotManager(Path logPath,
                       Path tablePath)

Method Detail

verifyDeltaVersions

public static void verifyDeltaVersions(java.util.List<Long> versions,
                                       java.util.Optional<Long> expectedStartVersion,
                                       java.util.Optional<Long> expectedEndVersion)

- Verify the versions are contiguous. - Verify the versions start with `expectedStartVersion` if it's specified. - Verify the versions end with `expectedEndVersion` if it's specified.

buildLatestSnapshot

public Snapshot buildLatestSnapshot(Engine engine)
                             throws TableNotFoundException

Construct the latest snapshot for given table.

Parameters:

engine - Instance of Engine to use.

Returns:

Throws:

TableNotFoundException

getSnapshotAt

public Snapshot getSnapshotAt(Engine engine,
                              long version)
                       throws TableNotFoundException

Construct the snapshot for the given table at the version provided.

Parameters:

engine - Instance of Engine to use.

version - The snapshot version to construct

Returns:

a Snapshot of the table at version version

Throws:

TableNotFoundException

getSnapshotForTimestamp

public Snapshot getSnapshotForTimestamp(Engine engine,
                                        long millisSinceEpochUTC)
                                 throws TableNotFoundException

Construct the snapshot for the given table at the provided timestamp.

Parameters:

engine - Instance of Engine to use.

millisSinceEpochUTC - timestamp to fetch the snapshot for in milliseconds since the unix epoch

Returns:

a Snapshot of the table at the provided timestamp

Throws:

TableNotFoundException

checkpoint

public void checkpoint(Engine engine,
                       long version)
                throws TableNotFoundException,
                       java.io.IOException

Throws:

TableNotFoundException

java.io.IOException

getLogSegmentForVersion

public java.util.Optional<LogSegment> getLogSegmentForVersion(Engine engine,
                                                              java.util.Optional<Long> startCheckpoint,
                                                              java.util.Optional<Long> versionToLoad)

Get a list of files that can be used to compute a Snapshot at version `versionToLoad`, If `versionToLoad` is not provided, will generate the list of files that are needed to load the latest version of the Delta table. This method also performs checks to ensure that the delta files are contiguous.

Parameters:

startCheckpoint - A potential start version to perform the listing of the DeltaLog, typically that of a known checkpoint. If this version's not provided, we will start listing from version 0.

versionToLoad - A specific version to load. Typically used with time travel and the Delta streaming source. If not provided, we will try to load the latest version of the table.

Returns:

Some LogSegment to build a Snapshot if files do exist after the given startCheckpoint. None, if the delta log directory was missing or empty.