io.delta.kernel.data

Interface ColumnarBatch

@Evolving
public interface ColumnarBatch

Represents zero or more rows of records with same schema type.

Since:

3.0.0

Method Summary

Modifier and Type	Method and Description
ColumnVector	getColumnVector(int ordinal) Return the ColumnVector for the given ordinal in the columnar batch.
default CloseableIterator<Row>	getRows()
StructType	getSchema()
int	getSize()
default ColumnarBatch	slice(int start, int end) Return a slice of the current batch.
default ColumnarBatch	withDeletedColumnAt(int ordinal) Return a copy of this ColumnarBatch with the column at given ordinal removed.
default ColumnarBatch	withNewColumn(int ordinal, StructField columnSchema, ColumnVector columnVector) Return a copy of the ColumnarBatch with given new column vector inserted at the given columnVector at given ordinal.
default ColumnarBatch	withNewSchema(StructType newSchema) Generate a copy of this ColumnarBatch with the given newSchema.

Method Detail

getSchema

StructType getSchema()

Returns:

the schema of the data in this batch.

getColumnVector

ColumnVector getColumnVector(int ordinal)

Return the ColumnVector for the given ordinal in the columnar batch. If the ordinal is not valid throws error.

Parameters:

ordinal - the ordinal of the column to retrieve

Returns:

the ColumnVector for the given ordinal in the columnar batch

getSize

int getSize()

Returns:

the number of rows/records in the columnar batch

withNewColumn

default ColumnarBatch withNewColumn(int ordinal,
                                    StructField columnSchema,
                                    ColumnVector columnVector)

Return a copy of the ColumnarBatch with given new column vector inserted at the given columnVector at given ordinal. Shift the existing ColumnVectors located at from ordinal to the end by one position. The schema of the new ColumnarBatch will also be changed to reflect the newly inserted vector.

Parameters:

ordinal -

columnSchema - Column name and schema details of the new column vector.

columnVector -

Returns:

ColumnarBatch with new vector inserted.

Throws:

IllegalArgumentException - If the ordinal is not valid (ie less than zero or greater than the current number of vectors).

withDeletedColumnAt

default ColumnarBatch withDeletedColumnAt(int ordinal)

Return a copy of this ColumnarBatch with the column at given ordinal removed. All columns after the ordinal will be shifted to left by one position.

Parameters:

ordinal - Column ordinal to delete.

Returns:

ColumnarBatch with a column vector deleted.

withNewSchema

default ColumnarBatch withNewSchema(StructType newSchema)

Generate a copy of this ColumnarBatch with the given newSchema. The data types of elements in the given new schema and existing schema should be the same. Rest of the details such as name of the column or column metadata could be different.

Parameters:

newSchema -

Returns:

ColumnarBatch with given new schema.

slice

default ColumnarBatch slice(int start,
                            int end)

Return a slice of the current batch.

Parameters:

start - Starting record index to include in the returned columnar batch

end - Ending record index (exclusive) to include in the returned columnar batch

Returns:

a columnar batch containing the records between [start, end)

getRows

default CloseableIterator<Row> getRows()

Returns:

iterator of Rows in this batch