Type Members

1.  case class CachedColumnarBatch(numRows: Int, sizeInBytes: Long, bytes: Array[Byte]) extends CachedBatch with Product with Serializable
2.  class ColumnarCachedBatchSerializer extends CachedBatchSerializer with SQLConfHelper

   Feature: 1.

   Feature: 1. This serializer supports column pruning 2. TODO: support push down filter 3. Super TODO: support store offheap object directly

   The data transformation pipeline:

   • Serializer ColumnarBatch -> CachedColumnarBatch -> serialize to byte[]
   • Deserializer CachedColumnarBatch -> ColumnarBatch -> deserialize to byte[] to create Velox ColumnarBatch
   • Serializer InternalRow -> CachedColumnarBatch (support RowToColumnar) -> Convert InternalRow to ColumnarBatch -> Serializer ColumnarBatch -> CachedColumnarBatch
   • Serializer InternalRow -> DefaultCachedBatch (unsupport RowToColumnar) -> Convert InternalRow to DefaultCachedBatch using vanilla Spark serializer
   • Deserializer CachedColumnarBatch -> InternalRow (support ColumnarToRow) -> Deserializer CachedColumnarBatch -> ColumnarBatch -> Convert ColumnarBatch to InternalRow
   • Deserializer DefaultCachedBatch -> InternalRow (unsupport ColumnarToRow) -> Convert DefaultCachedBatch to InternalRow using vanilla Spark serializer