Packages

c

org.apache.spark.sql.execution.datasources.orc

OrcColumnarBatchReader

class OrcColumnarBatchReader extends RecordReader[Void, ColumnarBatch]

To support vectorization in WholeStageCodeGen, this reader returns ColumnarBatch. After creating, initialize and initBatch should be called sequentially.

Linear Supertypes
RecordReader[Void, ColumnarBatch], Closeable, AutoCloseable, AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. OrcColumnarBatchReader
  2. RecordReader
  3. Closeable
  4. AutoCloseable
  5. AnyRef
  6. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. Protected

Instance Constructors

  1. new OrcColumnarBatchReader(capacity: Int)

Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##: Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  5. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.CloneNotSupportedException]) @native()
  6. def close(): Unit
    Definition Classes
    OrcColumnarBatchReader → RecordReader → Closeable → AutoCloseable
    Annotations
    @Override()
  7. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  8. def equals(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef → Any
  9. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.Throwable])
  10. final def getClass(): Class[_ <: AnyRef]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  11. def getCurrentKey(): Void
    Definition Classes
    OrcColumnarBatchReader → RecordReader
    Annotations
    @Override()
  12. def getCurrentValue(): ColumnarBatch
    Definition Classes
    OrcColumnarBatchReader → RecordReader
    Annotations
    @Override()
  13. def getProgress(): Float
    Definition Classes
    OrcColumnarBatchReader → RecordReader
    Annotations
    @Override()
  14. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  15. def initBatch(orcSchema: TypeDescription, requiredFields: Array[StructField], requestedDataColIds: Array[Int], requestedPartitionColIds: Array[Int], partitionValues: InternalRow): Unit

    Initialize columnar batch by setting required schema and partition information.

    Initialize columnar batch by setting required schema and partition information. With this information, this creates ColumnarBatch with the full schema.

    orcSchema

    Schema from ORC file reader.

    requiredFields

    All the fields that are required to return, including partition fields.

    requestedDataColIds

    Requested column ids from orcSchema. -1 if not existed.

    requestedPartitionColIds

    Requested column ids from partition schema. -1 if not existed.

    partitionValues

    Values of partition columns.

  16. def initialize(inputSplit: InputSplit, taskAttemptContext: TaskAttemptContext): Unit

    Initialize ORC file reader and batch record reader.

    Initialize ORC file reader and batch record reader. Please note that initBatch is needed to be called after this.

    Definition Classes
    OrcColumnarBatchReader → RecordReader
    Annotations
    @Override()
  17. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  18. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  19. def nextKeyValue(): Boolean
    Definition Classes
    OrcColumnarBatchReader → RecordReader
    Annotations
    @Override()
  20. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  21. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  22. final def synchronized[T0](arg0: => T0): T0
    Definition Classes
    AnyRef
  23. def toString(): String
    Definition Classes
    AnyRef → Any
  24. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  25. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException])
  26. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws(classOf[java.lang.InterruptedException]) @native()

Inherited from RecordReader[Void, ColumnarBatch]

Inherited from Closeable

Inherited from AutoCloseable

Inherited from AnyRef

Inherited from Any

Ungrouped