io.delta.kernel.internal.replay

Class ActionsIterator

Object

io.delta.kernel.internal.replay.ActionsIterator

All Implemented Interfaces:

CloseableIterator<ActionWrapper>, java.io.Closeable, AutoCloseable, java.util.Iterator<ActionWrapper>

public class ActionsIterator
extends Object
implements CloseableIterator<ActionWrapper>

This class takes as input a list of delta files (.json, .checkpoint.parquet) and produces an iterator of (ColumnarBatch, isFromCheckpoint) tuples, where the schema of the ColumnarBatch semantically represents actions (or, a subset of action fields) parsed from the Delta Log.

Users must pass in a `readSchema` to select which actions and sub-fields they want to consume.

Constructor Summary

Constructors

Constructor and Description
ActionsIterator(Engine engine, java.util.List<FileStatus> files, StructType readSchema, java.util.Optional<Predicate> checkpointPredicate)

Method Summary

Modifier and Type	Method and Description
void	close()
ColumnarBatch	extractSidecarsFromBatch(FileStatus checkpointFileStatus, long checkpointVersion, ColumnarBatch columnarBatch) Reads SidecarFile actions from ColumnarBatch, removing sidecar actions from the ColumnarBatch.
boolean	hasNext() Returns true if the iteration has more elements.
ActionWrapper	next() Returns the next element in the iteration.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.delta.kernel.utils.CloseableIterator

combine, filter, map

Methods inherited from interface java.util.Iterator

forEachRemaining, remove

Constructor Detail

ActionsIterator

public ActionsIterator(Engine engine,
                       java.util.List<FileStatus> files,
                       StructType readSchema,
                       java.util.Optional<Predicate> checkpointPredicate)

Method Detail

hasNext

public boolean hasNext()

Description copied from interface: CloseableIterator

Returns true if the iteration has more elements. (In other words, returns true if next would return an element rather than throwing an exception.)

Specified by:

hasNext in interface CloseableIterator<ActionWrapper>

Specified by:

hasNext in interface java.util.Iterator<ActionWrapper>

Returns:

true if the iteration has more elements

next

public ActionWrapper next()

Description copied from interface: CloseableIterator

Returns the next element in the iteration.

Specified by:

next in interface CloseableIterator<ActionWrapper>

Specified by:

next in interface java.util.Iterator<ActionWrapper>

Returns:

a tuple of (ColumnarBatch, isFromCheckpoint), where ColumnarBatch conforms to the instance readSchema.

close

public void close()
           throws java.io.IOException

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface AutoCloseable

Throws:

java.io.IOException

extractSidecarsFromBatch

public ColumnarBatch extractSidecarsFromBatch(FileStatus checkpointFileStatus,
                                              long checkpointVersion,
                                              ColumnarBatch columnarBatch)

Reads SidecarFile actions from ColumnarBatch, removing sidecar actions from the ColumnarBatch. Returns a list of SidecarFile actions found.