io.delta.kernel.internal.snapshot

Class SnapshotManager

Object

io.delta.kernel.internal.snapshot.SnapshotManager

public class SnapshotManager
extends Object

Constructor Summary

Constructors

Constructor and Description
SnapshotManager()

Method Summary

Modifier and Type	Method and Description
Snapshot	buildLatestSnapshot(TableClient tableClient, Path logPath, Path dataPath) Construct the latest snapshot for given table.
java.util.Optional<LogSegment>	getLogSegmentForVersion(Path logPath, TableClient tableClient, java.util.Optional<Long> startCheckpoint, java.util.Optional<Long> versionToLoad) Get a list of files that can be used to compute a Snapshot at version `versionToLoad`, If `versionToLoad` is not provided, will generate the list of files that are needed to load the latest version of the Delta table.
static void	verifyDeltaVersions(java.util.List<Long> versions, java.util.Optional<Long> expectedStartVersion, java.util.Optional<Long> expectedEndVersion) - Verify the versions are contiguous.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

SnapshotManager

public SnapshotManager()

Method Detail

verifyDeltaVersions

public static void verifyDeltaVersions(java.util.List<Long> versions,
                                       java.util.Optional<Long> expectedStartVersion,
                                       java.util.Optional<Long> expectedEndVersion)

- Verify the versions are contiguous. - Verify the versions start with `expectedStartVersion` if it's specified. - Verify the versions end with `expectedEndVersion` if it's specified.

buildLatestSnapshot

public Snapshot buildLatestSnapshot(TableClient tableClient,
                                    Path logPath,
                                    Path dataPath)
                             throws TableNotFoundException

Construct the latest snapshot for given table.

Parameters:

tableClient - Instance of TableClient to use.

logPath - Where the Delta log files are located.

dataPath - Where the Delta data files are located.

Returns:

Throws:

TableNotFoundException

getLogSegmentForVersion

public java.util.Optional<LogSegment> getLogSegmentForVersion(Path logPath,
                                                              TableClient tableClient,
                                                              java.util.Optional<Long> startCheckpoint,
                                                              java.util.Optional<Long> versionToLoad)

Get a list of files that can be used to compute a Snapshot at version `versionToLoad`, If `versionToLoad` is not provided, will generate the list of files that are needed to load the latest version of the Delta table. This method also performs checks to ensure that the delta files are contiguous.

Parameters:

startCheckpoint - A potential start version to perform the listing of the DeltaLog, typically that of a known checkpoint. If this version's not provided, we will start listing from version 0.

versionToLoad - A specific version to load. Typically used with time travel and the Delta streaming source. If not provided, we will try to load the latest version of the table.

Returns:

Some LogSegment to build a Snapshot if files do exist after the given startCheckpoint. None, if the directory was missing or empty.