trait SnapshotManagement extends AnyRef
Manages the creation, computation, and access of Snapshot's for Delta tables. Responsibilities include:
- Figuring out the set of files that are required to compute a specific version of a table
- Updating and exposing the latest snapshot of the Delta table in a thread-safe manner
- Self Type
- DeltaLog
- Alphabetic
- By Inheritance
- SnapshotManagement
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
def
createSnapshot(segment: LogSegment, minFileRetentionTimestamp: Long, timestamp: Long): Snapshot
- Attributes
- protected
-
val
currentSnapshot: Snapshot
- Attributes
- protected
- Annotations
- @volatile()
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( classOf[java.lang.Throwable] )
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
def
getLogSegmentForVersion(startCheckpoint: Option[Long], versionToLoad: Option[Long] = None): LogSegment
Get a list of files that can be used to compute a Snapshot at version
versionToLoad, IfversionToLoadis not provided, will generate the list of files that are needed to load the latest version of the Delta table.Get a list of files that can be used to compute a Snapshot at version
versionToLoad, IfversionToLoadis not provided, will generate the list of files that are needed to load the latest version of the Delta table. This method also performs checks to ensure that the delta files are contiguous.- startCheckpoint
A potential start version to perform the listing of the DeltaLog, typically that of a known checkpoint. If this version's not provided, we will start listing from version 0.
- versionToLoad
A specific version to load. Typically used with time travel and the Delta streaming source. If not provided, we will try to load the latest version of the table.
- returns
Some LogSegment to build a Snapshot if files do exist after the given startCheckpoint. None, if there are no new files after
startCheckpoint.
- Attributes
- protected
-
def
getLogSegmentFrom(startingCheckpoint: Option[CheckpointMetaData]): LogSegment
Get the LogSegment that will help in computing the Snapshot of the table at DeltaLog initialization.
Get the LogSegment that will help in computing the Snapshot of the table at DeltaLog initialization.
- startingCheckpoint
A checkpoint that we can start our listing from
- Attributes
- protected
-
def
getSnapshotAt(version: Long, commitTimestamp: Option[Long] = None, lastCheckpointHint: Option[CheckpointInstance] = None): Snapshot
Get the snapshot at
version. -
def
getSnapshotAtInit: Snapshot
Load the Snapshot for this Delta table at initialization.
Load the Snapshot for this Delta table at initialization. This method uses the
lastCheckpointfile as a hint on where to start listing the transaction log directory. If the _delta_log directory doesn't exist, this method will return anInitialSnapshot.- Attributes
- protected
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
val
lastUpdateTimestamp: Long
The timestamp when the last successful update action is finished.
The timestamp when the last successful update action is finished.
- Attributes
- protected
- Annotations
- @volatile()
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
-
def
snapshot: Snapshot
Returns the current snapshot.
Returns the current snapshot. Note this does not automatically
update(). -
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
def
update(stalenessAcceptable: Boolean = false): Snapshot
Update ActionLog by applying the new delta files if any.
Update ActionLog by applying the new delta files if any.
- stalenessAcceptable
Whether we can accept working with a stale version of the table. If the table has surpassed our staleness tolerance, we will update to the latest state of the table synchronously. If staleness is acceptable, and the table hasn't passed the staleness tolerance, we will kick off a job in the background to update the table state, and can return a stale snapshot in the meantime.
-
def
updateInternal(isAsync: Boolean): Snapshot
Queries the store for new delta files and applies them to the current state.
Queries the store for new delta files and applies them to the current state. Note: the caller should hold
deltaLogLockbefore calling this method.- Attributes
- protected
-
def
verifyDeltaVersions(versions: Array[Long]): Unit
Verify the versions are contiguous.
Verify the versions are contiguous.
- Attributes
- protected
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()