io.delta.kernel.internal.replay

Class ActiveAddFilesIterator

Object

io.delta.kernel.internal.replay.ActiveAddFilesIterator

All Implemented Interfaces:

CloseableIterator<FilteredColumnarBatch>, java.io.Closeable, AutoCloseable, java.util.Iterator<FilteredColumnarBatch>

public class ActiveAddFilesIterator
extends Object
implements CloseableIterator<FilteredColumnarBatch>

This class takes an iterator of (ColumnarBatch, isFromCheckpoint), where the columnar data inside the columnar batch represents has top level columns "add" and "remove", and produces an iterator of FilteredColumnarBatch with only the "add" column and with a selection vector indicating which AddFiles are still active in the table (have not been tombstoned).

Method Summary

Modifier and Type	Method and Description
void	close()
static DeletionVectorDescriptor	getAddFileDV(ColumnVector addFileVector, int rowId)
static String	getAddFilePath(ColumnVector addFileVector, int rowId)
LogReplayMetrics	getMetrics() Returns the metrics for the log replay.
static DeletionVectorDescriptor	getRemoveFileDV(ColumnVector removeFileVector, int rowId)
static String	getRemoveFilePath(ColumnVector removeFileVector, int rowId)
boolean	hasNext() Returns true if the iteration has more elements.
FilteredColumnarBatch	next() Returns the next element in the iteration.

Methods inherited from class Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.delta.kernel.utils.CloseableIterator

combine, filter, map

Methods inherited from interface java.util.Iterator

forEachRemaining, remove

Method Detail

hasNext

public boolean hasNext()

Description copied from interface: CloseableIterator

Returns true if the iteration has more elements. (In other words, returns true if next would return an element rather than throwing an exception.)

Specified by:

hasNext in interface CloseableIterator<FilteredColumnarBatch>

Specified by:

hasNext in interface java.util.Iterator<FilteredColumnarBatch>

Returns:

true if the iteration has more elements

next

public FilteredColumnarBatch next()

Description copied from interface: CloseableIterator

Returns the next element in the iteration.

Specified by:

next in interface CloseableIterator<FilteredColumnarBatch>

Specified by:

next in interface java.util.Iterator<FilteredColumnarBatch>

Returns:

the next element in the iteration

close

public void close()
           throws java.io.IOException

Specified by:

close in interface java.io.Closeable

Specified by:

close in interface AutoCloseable

Throws:

java.io.IOException

getAddFilePath

public static String getAddFilePath(ColumnVector addFileVector,
                                    int rowId)

getAddFileDV

public static DeletionVectorDescriptor getAddFileDV(ColumnVector addFileVector,
                                                    int rowId)

getRemoveFilePath

public static String getRemoveFilePath(ColumnVector removeFileVector,
                                       int rowId)

getRemoveFileDV

public static DeletionVectorDescriptor getRemoveFileDV(ColumnVector removeFileVector,
                                                       int rowId)

getMetrics

public LogReplayMetrics getMetrics()

Returns the metrics for the log replay. Currently used in tests only. Caution: The metrics should be fetched only after the iterator is closed, to avoid reading incomplete metrics.