Packages

c

org.apache.spark.sql.execution.vectorized

WritableColumnVector

abstract class WritableColumnVector extends ColumnVector

This class adds write APIs to ColumnVector. It supports all the types and contains put APIs as well as their batched versions. The batched versions are preferable whenever possible.

Capacity: The data stored is dense but the arrays are not fixed capacity. It is the responsibility of the caller to call reserve() to ensure there is enough room before adding elements. This means that the put() APIs do not check as in common cases (i.e. flat schemas), the lengths are known up front.

A WritableColumnVector should be considered immutable once originally created. In other words, it is not valid to call put APIs after reads until reset() is called.

WritableColumnVector are intended to be reused.

Linear Supertypes
ColumnVector, AutoCloseable, AnyRef, Any
Known Subclasses
OffHeapColumnVector, OnHeapColumnVector
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. WritableColumnVector
  2. ColumnVector
  3. AutoCloseable
  4. AnyRef
  5. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Instance Constructors

  1. new WritableColumnVector(capacity: Int, type: DataType)
    Attributes
    protected[vectorized]

Abstract Value Members

  1. abstract def getArrayLength(rowId: Int): Int
  2. abstract def getArrayOffset(rowId: Int): Int
  3. abstract def getBoolean(arg0: Int): Boolean
    Definition Classes
    ColumnVector
  4. abstract def getByte(arg0: Int): Byte
    Definition Classes
    ColumnVector
  5. abstract def getBytesAsUTF8String(rowId: Int, count: Int): UTF8String

    Gets the values of bytes from [rowId, rowId + count), as a UTF8String.

    Gets the values of bytes from [rowId, rowId + count), as a UTF8String. This method is similar to int), but can save data copy as UTF8String is used as a pointer.

    Attributes
    protected[vectorized]
  6. abstract def getDictId(rowId: Int): Int

    Returns the dictionary Id for rowId.

    Returns the dictionary Id for rowId.

    This should only be called when this WritableColumnVector represents dictionaryIds. We have this separate method for dictionaryIds as per SPARK-16928.

  7. abstract def getDouble(arg0: Int): Double
    Definition Classes
    ColumnVector
  8. abstract def getFloat(arg0: Int): Float
    Definition Classes
    ColumnVector
  9. abstract def getInt(arg0: Int): Int
    Definition Classes
    ColumnVector
  10. abstract def getLong(arg0: Int): Long
    Definition Classes
    ColumnVector
  11. abstract def getShort(arg0: Int): Short
    Definition Classes
    ColumnVector
  12. abstract def isNullAt(arg0: Int): Boolean
    Definition Classes
    ColumnVector
  13. abstract def putArray(rowId: Int, offset: Int, length: Int): Unit

    Puts a byte array that already exists in this column.

  14. abstract def putBoolean(rowId: Int, value: Boolean): Unit

    Sets value to the value at rowId.

  15. abstract def putBooleans(rowId: Int, count: Int, value: Boolean): Unit

    Sets value to [rowId, rowId + count).

  16. abstract def putByte(rowId: Int, value: Byte): Unit

    Sets value to the value at rowId.

  17. abstract def putByteArray(rowId: Int, value: Array[Byte], offset: Int, count: Int): Int

    Sets values from [value + offset, value + offset + count) to the values at rowId.

  18. abstract def putBytes(rowId: Int, count: Int, src: Array[Byte], srcIndex: Int): Unit

    Sets values from [src[srcIndex], src[srcIndex + count]) to [rowId, rowId + count)

  19. abstract def putBytes(rowId: Int, count: Int, value: Byte): Unit

    Sets value to [rowId, rowId + count).

  20. abstract def putDouble(rowId: Int, value: Double): Unit

    Sets value to the value at rowId.

  21. abstract def putDoubles(rowId: Int, count: Int, src: Array[Byte], srcIndex: Int): Unit

    Sets values from [src[srcIndex], src[srcIndex + count * 8]) to [rowId, rowId + count) The data in src must be ieee formatted doubles in platform native endian.

  22. abstract def putDoubles(rowId: Int, count: Int, src: Array[Double], srcIndex: Int): Unit

    Sets values from [src[srcIndex], src[srcIndex + count]) to [rowId, rowId + count)

  23. abstract def putDoubles(rowId: Int, count: Int, value: Double): Unit

    Sets value to [rowId, rowId + count).

  24. abstract def putDoublesLittleEndian(rowId: Int, count: Int, src: Array[Byte], srcIndex: Int): Unit

    Sets values from [src[srcIndex], src[srcIndex + count * 8]) to [rowId, rowId + count) The data in src must be ieee formatted doubles in little endian.

  25. abstract def putFloat(rowId: Int, value: Float): Unit

    Sets value to the value at rowId.

  26. abstract def putFloats(rowId: Int, count: Int, src: Array[Byte], srcIndex: Int): Unit

    Sets values from [src[srcIndex], src[srcIndex + count * 4]) to [rowId, rowId + count) The data in src must be ieee formatted floats in platform native endian.

  27. abstract def putFloats(rowId: Int, count: Int, src: Array[Float], srcIndex: Int): Unit

    Sets values from [src[srcIndex], src[srcIndex + count]) to [rowId, rowId + count)

  28. abstract def putFloats(rowId: Int, count: Int, value: Float): Unit

    Sets value to [rowId, rowId + count).

  29. abstract def putFloatsLittleEndian(rowId: Int, count: Int, src: Array[Byte], srcIndex: Int): Unit

    Sets values from [src[srcIndex], src[srcIndex + count * 4]) to [rowId, rowId + count) The data in src must be ieee formatted floats in little endian.

  30. abstract def putInt(rowId: Int, value: Int): Unit

    Sets value to the value at rowId.

  31. abstract def putInts(rowId: Int, count: Int, src: Array[Byte], srcIndex: Int): Unit

    Sets values from [src[srcIndex], src[srcIndex + count * 4]) to [rowId, rowId + count) The data in src must be 4-byte platform native endian ints.

  32. abstract def putInts(rowId: Int, count: Int, src: Array[Int], srcIndex: Int): Unit

    Sets values from [src[srcIndex], src[srcIndex + count]) to [rowId, rowId + count)

  33. abstract def putInts(rowId: Int, count: Int, value: Int): Unit

    Sets value to [rowId, rowId + count).

  34. abstract def putIntsLittleEndian(rowId: Int, count: Int, src: Array[Byte], srcIndex: Int): Unit

    Sets values from [src[srcIndex], src[srcIndex + count * 4]) to [rowId, rowId + count) The data in src must be 4-byte little endian ints.

  35. abstract def putLong(rowId: Int, value: Long): Unit

    Sets value to the value at rowId.

  36. abstract def putLongs(rowId: Int, count: Int, src: Array[Byte], srcIndex: Int): Unit

    Sets values from [src[srcIndex], src[srcIndex + count * 8]) to [rowId, rowId + count) The data in src must be 8-byte platform native endian longs.

  37. abstract def putLongs(rowId: Int, count: Int, src: Array[Long], srcIndex: Int): Unit

    Sets values from [src[srcIndex], src[srcIndex + count]) to [rowId, rowId + count)

  38. abstract def putLongs(rowId: Int, count: Int, value: Long): Unit

    Sets value to [rowId, rowId + count).

  39. abstract def putLongsLittleEndian(rowId: Int, count: Int, src: Array[Byte], srcIndex: Int): Unit

    Sets values from [src + srcIndex, src + srcIndex + count * 8) to [rowId, rowId + count) The data in src must be 8-byte little endian longs.

  40. abstract def putNotNull(rowId: Int): Unit

    Sets null/not null to the value at rowId.

  41. abstract def putNotNulls(rowId: Int, count: Int): Unit
  42. abstract def putNull(rowId: Int): Unit
  43. abstract def putNulls(rowId: Int, count: Int): Unit

    Sets null/not null to the values at [rowId, rowId + count).

  44. abstract def putShort(rowId: Int, value: Short): Unit

    Sets value to the value at rowId.

  45. abstract def putShorts(rowId: Int, count: Int, src: Array[Byte], srcIndex: Int): Unit

    Sets values from [src[srcIndex], src[srcIndex + count * 2]) to [rowId, rowId + count) The data in src must be 2-byte platform native endian shorts.

  46. abstract def putShorts(rowId: Int, count: Int, src: Array[Short], srcIndex: Int): Unit

    Sets values from [src[srcIndex], src[srcIndex + count]) to [rowId, rowId + count)

  47. abstract def putShorts(rowId: Int, count: Int, value: Short): Unit

    Sets value to [rowId, rowId + count).

  48. abstract def reserveInternal(capacity: Int): Unit

    Ensures that there is enough storage to store capacity elements.

    Ensures that there is enough storage to store capacity elements. That is, the put() APIs must work for all rowIds < capacity.

    Attributes
    protected[vectorized]
  49. abstract def reserveNewColumn(capacity: Int, type: DataType): WritableColumnVector

    Reserve a new column.

    Reserve a new column.

    Attributes
    protected[vectorized]

Concrete Value Members

  1. final def !=(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  2. final def ##(): Int
    Definition Classes
    AnyRef → Any
  3. final def ==(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  4. final def appendArray(length: Int): Int
  5. final def appendBoolean(v: Boolean): Int
  6. final def appendBooleans(count: Int, v: Boolean): Int
  7. final def appendByte(v: Byte): Int
  8. final def appendByteArray(value: Array[Byte], offset: Int, length: Int): Int
  9. final def appendBytes(length: Int, src: Array[Byte], offset: Int): Int
  10. final def appendBytes(count: Int, v: Byte): Int
  11. final def appendDouble(v: Double): Int
  12. final def appendDoubles(length: Int, src: Array[Double], offset: Int): Int
  13. final def appendDoubles(count: Int, v: Double): Int
  14. final def appendFloat(v: Float): Int
  15. final def appendFloats(length: Int, src: Array[Float], offset: Int): Int
  16. final def appendFloats(count: Int, v: Float): Int
  17. final def appendInt(v: Int): Int
  18. final def appendInts(length: Int, src: Array[Int], offset: Int): Int
  19. final def appendInts(count: Int, v: Int): Int
  20. final def appendLong(v: Long): Int
  21. final def appendLongs(length: Int, src: Array[Long], offset: Int): Int
  22. final def appendLongs(count: Int, v: Long): Int
  23. final def appendNotNull(): Int
  24. final def appendNotNulls(count: Int): Int
  25. final def appendNull(): Int

    Append APIs.

    Append APIs. These APIs all behave similarly and will append data to the current vector. It is not valid to mix the put and append APIs. The append APIs are slower and should only be used if the sizes are not known up front. In all these cases, the return value is the rowId for the first appended element.

  26. final def appendNulls(count: Int): Int
  27. final def appendShort(v: Short): Int
  28. final def appendShorts(length: Int, src: Array[Short], offset: Int): Int
  29. final def appendShorts(count: Int, v: Short): Int
  30. final def appendStruct(isNull: Boolean): Int

    Appends a NULL struct.

    Appends a NULL struct. This *has* to be used for structs instead of appendNull() as this recursively appends a NULL to its children. We don't have this logic as the general appendNull implementation to optimize the more common non-struct case.

  31. def arrayData(): WritableColumnVector
  32. final def asInstanceOf[T0]: T0
    Definition Classes
    Any
  33. def clone(): AnyRef
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()
  34. def close(): Unit
    Definition Classes
    WritableColumnVector → ColumnVector → AutoCloseable
    Annotations
    @Override()
  35. final def dataType(): DataType
    Definition Classes
    ColumnVector
  36. final def eq(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  37. def equals(arg0: Any): Boolean
    Definition Classes
    AnyRef → Any
  38. def finalize(): Unit
    Attributes
    protected[lang]
    Definition Classes
    AnyRef
    Annotations
    @throws( classOf[java.lang.Throwable] )
  39. final def getArray(rowId: Int): ColumnarArray
    Definition Classes
    WritableColumnVector → ColumnVector
    Annotations
    @Override()
  40. def getBinary(rowId: Int): Array[Byte]
    Definition Classes
    WritableColumnVector → ColumnVector
    Annotations
    @Override()
  41. def getBooleans(arg0: Int, arg1: Int): Array[Boolean]
    Definition Classes
    ColumnVector
  42. def getBytes(arg0: Int, arg1: Int): Array[Byte]
    Definition Classes
    ColumnVector
  43. def getChild(ordinal: Int): WritableColumnVector
    Definition Classes
    WritableColumnVector → ColumnVector
    Annotations
    @Override()
  44. final def getClass(): Class[_]
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  45. def getDecimal(rowId: Int, precision: Int, scale: Int): Decimal
    Definition Classes
    WritableColumnVector → ColumnVector
    Annotations
    @Override()
  46. def getDictionaryIds(): WritableColumnVector

    Returns the underlying integer column for ids of dictionary.

  47. def getDoubles(arg0: Int, arg1: Int): Array[Double]
    Definition Classes
    ColumnVector
  48. final def getElementsAppended(): Int

    Returns the elements appended.

  49. def getFloats(arg0: Int, arg1: Int): Array[Float]
    Definition Classes
    ColumnVector
  50. final def getInterval(arg0: Int): CalendarInterval
    Definition Classes
    ColumnVector
  51. def getInts(arg0: Int, arg1: Int): Array[Int]
    Definition Classes
    ColumnVector
  52. def getLongs(arg0: Int, arg1: Int): Array[Long]
    Definition Classes
    ColumnVector
  53. final def getMap(rowId: Int): ColumnarMap
    Definition Classes
    WritableColumnVector → ColumnVector
    Annotations
    @Override()
  54. def getShorts(arg0: Int, arg1: Int): Array[Short]
    Definition Classes
    ColumnVector
  55. final def getStruct(arg0: Int): ColumnarRow
    Definition Classes
    ColumnVector
  56. def getUTF8String(rowId: Int): UTF8String
    Definition Classes
    WritableColumnVector → ColumnVector
    Annotations
    @Override()
  57. def hasDictionary(): Boolean

    Returns true if this column has a dictionary.

  58. def hasNull(): Boolean
    Definition Classes
    WritableColumnVector → ColumnVector
    Annotations
    @Override()
  59. def hashCode(): Int
    Definition Classes
    AnyRef → Any
    Annotations
    @native()
  60. def isArray(): Boolean
    Attributes
    protected[vectorized]
  61. final def isInstanceOf[T0]: Boolean
    Definition Classes
    Any
  62. final def ne(arg0: AnyRef): Boolean
    Definition Classes
    AnyRef
  63. final def notify(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  64. final def notifyAll(): Unit
    Definition Classes
    AnyRef
    Annotations
    @native()
  65. def numNulls(): Int
    Definition Classes
    WritableColumnVector → ColumnVector
    Annotations
    @Override()
  66. final def putByteArray(rowId: Int, value: Array[Byte]): Int
  67. def putDecimal(rowId: Int, value: Decimal, precision: Int): Unit
  68. def putInterval(rowId: Int, value: CalendarInterval): Unit
  69. def reserve(requiredCapacity: Int): Unit
  70. def reserveDictionaryIds(capacity: Int): WritableColumnVector

    Reserve a integer column for ids of dictionary.

  71. def reset(): Unit

    Resets this column for writing.

    Resets this column for writing. The currently stored values are no longer accessible.

  72. def setDictionary(dictionary: Dictionary): Unit

    Update the dictionary.

  73. final def setIsConstant(): Unit

    Marks this column as being constant.

  74. final def synchronized[T0](arg0: ⇒ T0): T0
    Definition Classes
    AnyRef
  75. def toString(): String
    Definition Classes
    AnyRef → Any
  76. final def wait(): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  77. final def wait(arg0: Long, arg1: Int): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... )
  78. final def wait(arg0: Long): Unit
    Definition Classes
    AnyRef
    Annotations
    @throws( ... ) @native()

Inherited from ColumnVector

Inherited from AutoCloseable

Inherited from AnyRef

Inherited from Any

Ungrouped