RocksDbTableDescriptor (samza-kv-rocksdb

java.lang.Object
- org.apache.samza.table.descriptors.BaseTableDescriptor<K,V,D>
- - org.apache.samza.table.descriptors.LocalTableDescriptor<K,V,RocksDbTableDescriptor<K,V>>
  - - org.apache.samza.storage.kv.descriptors.RocksDbTableDescriptor<K,V>

Type Parameters:

K - the type of the key

V - the type of the value

All Implemented Interfaces:

org.apache.samza.table.descriptors.TableDescriptor<K,V,RocksDbTableDescriptor<K,V>>
```
public class RocksDbTableDescriptor<K,V>
extends org.apache.samza.table.descriptors.LocalTableDescriptor<K,V,RocksDbTableDescriptor<K,V>>
```
Table descriptor for RocksDb backed tables

Field Summary

Fields
Modifier and Type	Field and Description
`static java.lang.String`	`CONTAINER_CACHE_SIZE_BYTES`
`static java.lang.String`	`CONTAINER_WRITE_BUFFER_SIZE_BYTES`
`static java.lang.String`	`OBJECT_CACHE_SIZE`
`static java.lang.String`	`ROCKSDB_BLOCK_SIZE_BYTES`
`static java.lang.String`	`ROCKSDB_COMPACTION_STYLE`
`static java.lang.String`	`ROCKSDB_COMPRESSION`
`static java.lang.String`	`ROCKSDB_KEEP_LOG_FILE_NUM`
`static java.lang.String`	`ROCKSDB_MAX_LOG_FILE_SIZE_BYTES`
`static java.lang.String`	`ROCKSDB_NUM_WRITE_BUFFERS`
`static java.lang.String`	`ROCKSDB_TTL_MS`
`static java.lang.String`	`WRITE_BATCH_SIZE`

Fields inherited from class org.apache.samza.table.descriptors.LocalTableDescriptor
changelogReplicationFactor, changelogStream, enableChangelog, serde, sideInputs, sideInputsProcessor, SYSTEM_STREAM_NAME_PATTERN

Fields inherited from class org.apache.samza.table.descriptors.BaseTableDescriptor
config, tableId

Constructor Summary

Constructors
Constructor and Description
`RocksDbTableDescriptor(java.lang.String tableId, org.apache.samza.serializers.KVSerde<K,V> serde)` Constructs a table descriptor instance

Method Summary

All Methods Instance Methods Concrete Methods
Modifier and Type	Method and Description
`java.lang.String`	`getProviderFactoryClassName()`
`java.util.Map<java.lang.String,java.lang.String>`	`toConfig(org.apache.samza.config.Config jobConfig)`
`RocksDbTableDescriptor<K,V>`	`withBlockSize(int blockSize)` If compression is enabled, RocksDB groups approximately this many uncompressed bytes into one compressed block.
`RocksDbTableDescriptor<K,V>`	`withCacheSize(int cacheSize)` The size of RocksDB's block cache in bytes, per container.
`RocksDbTableDescriptor<K,V>`	`withCompactionStyle(java.lang.String compactionStyle)` This property controls the compaction style that RocksDB will employ when compacting its levels.
`RocksDbTableDescriptor<K,V>`	`withCompressionType(java.lang.String compressionType)` Controls whether RocksDB should compress data on disk and in the block cache.
`RocksDbTableDescriptor<K,V>`	`withMaxLogFileSize(int maxLogFileSize)` The maximum size in bytes of the RocksDB LOG file before it is rotated.
`RocksDbTableDescriptor<K,V>`	`withNumLogFilesToKeep(int numLogFilesToKeep)` The number of RocksDB LOG files (including rotated LOG.old.* files) to keep.
`RocksDbTableDescriptor<K,V>`	`withNumWriteBuffers(int numWriteBuffers)` Configures the number of write buffers that a RocksDB store uses.
`RocksDbTableDescriptor`	`withObjectCacheSize(int objectCacheSize)` Samza maintains an additional cache in front of RocksDB for frequently-accessed objects.
`RocksDbTableDescriptor<K,V>`	`withTtl(long ttl)` The time-to-live of the store.
`RocksDbTableDescriptor`	`withWriteBatchSize(int writeBatchSize)` For better write performance, the storage engine buffers writes and applies them to the underlying store in a batch.
`RocksDbTableDescriptor<K,V>`	`withWriteBufferSize(int writeBufferSize)` The amount of memory (in bytes) that RocksDB uses for buffering writes before they are written to disk, per container.

Methods inherited from class org.apache.samza.table.descriptors.LocalTableDescriptor
addStoreConfig, getSerde, getSideInputs, validate, withChangelogEnabled, withChangelogReplicationFactor, withChangelogStream, withSideInputs, withSideInputsProcessor

Methods inherited from class org.apache.samza.table.descriptors.BaseTableDescriptor
addTableConfig, getTableId, withConfig

Methods inherited from class java.lang.Object
clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

- Field Detail
  - WRITE_BATCH_SIZE
```
public static final java.lang.String WRITE_BATCH_SIZE
```
    See Also:
    
    Constant Field Values
  - OBJECT_CACHE_SIZE
```
public static final java.lang.String OBJECT_CACHE_SIZE
```
    See Also:
    
    Constant Field Values
  - CONTAINER_CACHE_SIZE_BYTES
```
public static final java.lang.String CONTAINER_CACHE_SIZE_BYTES
```
    See Also:
    
    Constant Field Values
  - CONTAINER_WRITE_BUFFER_SIZE_BYTES
```
public static final java.lang.String CONTAINER_WRITE_BUFFER_SIZE_BYTES
```
    See Also:
    
    Constant Field Values
  - ROCKSDB_COMPRESSION
```
public static final java.lang.String ROCKSDB_COMPRESSION
```
    See Also:
    
    Constant Field Values
  - ROCKSDB_BLOCK_SIZE_BYTES
```
public static final java.lang.String ROCKSDB_BLOCK_SIZE_BYTES
```
    See Also:
    
    Constant Field Values
  - ROCKSDB_TTL_MS
```
public static final java.lang.String ROCKSDB_TTL_MS
```
    See Also:
    
    Constant Field Values
  - ROCKSDB_COMPACTION_STYLE
```
public static final java.lang.String ROCKSDB_COMPACTION_STYLE
```
    See Also:
    
    Constant Field Values
  - ROCKSDB_NUM_WRITE_BUFFERS
```
public static final java.lang.String ROCKSDB_NUM_WRITE_BUFFERS
```
    See Also:
    
    Constant Field Values
  - ROCKSDB_MAX_LOG_FILE_SIZE_BYTES
```
public static final java.lang.String ROCKSDB_MAX_LOG_FILE_SIZE_BYTES
```
    See Also:
    
    Constant Field Values
  - ROCKSDB_KEEP_LOG_FILE_NUM
```
public static final java.lang.String ROCKSDB_KEEP_LOG_FILE_NUM
```
    See Also:
    
    Constant Field Values
- Constructor Detail
  - RocksDbTableDescriptor
```
public RocksDbTableDescriptor(java.lang.String tableId,
                              org.apache.samza.serializers.KVSerde<K,V> serde)
```
    Constructs a table descriptor instance
    
    Parameters:
    
    tableId - Id of the table, it must conform to pattern [\\d\\w-_]+
    
    serde - the serde for key and value
- Method Detail
  - withWriteBatchSize
```
public RocksDbTableDescriptor withWriteBatchSize(int writeBatchSize)
```
    For better write performance, the storage engine buffers writes and applies them to the underlying store in a batch. If the same key is written multiple times in quick succession, this buffer also deduplicates writes to the same key. This property is set to the number of key/value pairs that should be kept in this in-memory buffer, per task instance. The number cannot be greater than withObjectCacheSize(int).
    Default value is 500.
    Refer to stores.store-name.write.batch.size in Samza configuration guide
    
    Parameters:
    
    writeBatchSize - write batch size
    
    Returns:
    
    this table descriptor instance
  - withObjectCacheSize
```
public RocksDbTableDescriptor withObjectCacheSize(int objectCacheSize)
```
    Samza maintains an additional cache in front of RocksDB for frequently-accessed objects. This cache contains deserialized objects (avoiding the deserialization overhead on cache hits), in contrast to the RocksDB block cache (withCacheSize(int)), which caches serialized objects. This property determines the number of objects to keep in Samza's cache, per task instance. This same cache is also used for write buffering (see withWriteBatchSize(int)). A value of 0 disables all caching and batching.
    Default value is 1,000.
    Refer to stores.store-name.object.cache.size in Samza configuration guide
    
    Parameters:
    
    objectCacheSize - the object cache size
    
    Returns:
    
    this table descriptor instance
  - withCacheSize
```
public RocksDbTableDescriptor<K,V> withCacheSize(int cacheSize)
```
    The size of RocksDB's block cache in bytes, per container. If there are several task instances within one container, each is given a proportional share of this cache. Note that this is an off-heap memory allocation, so the container's total memory use is the maximum JVM heap size plus the size of this cache.
    Default value is 104,857,600.
    Refer to stores.store-name.container.cache.size.bytes in Samza configuration guide
    
    Parameters:
    
    cacheSize - the cache size in bytes
    
    Returns:
    
    this table descriptor instance
  - withWriteBufferSize
```
public RocksDbTableDescriptor<K,V> withWriteBufferSize(int writeBufferSize)
```
    The amount of memory (in bytes) that RocksDB uses for buffering writes before they are written to disk, per container. If there are several task instances within one container, each is given a proportional share of this buffer. This setting also determines the size of RocksDB's segment files.
    Default value is 33,554,432.
    Refer to stores.store-name.container.write.buffer.size.bytes in Samza configuration guide
    
    Parameters:
    
    writeBufferSize - the write buffer size in bytes
    
    Returns:
    
    this table descriptor instance
  - withCompressionType
```
public RocksDbTableDescriptor<K,V> withCompressionType(java.lang.String compressionType)
```
    Controls whether RocksDB should compress data on disk and in the block cache. The following values are valid:
    - snappy Compress data using the Snappy codec.
    - bzip2 Compress data using the bzip2 codec.
    - zlib Compress data using the zlib codec.
    - lz4 Compress data using the lz4 codec.
    - lz4hc Compress data using the lz4hc (high compression) codec.
    - none Do not compress data.
    Default value is snappy.
    Refer to stores.store-name.rocksdb.compression in Samza configuration guide
    Parameters:
    
    compressionType - the compression type
    
    Returns:
    
    this table descriptor instance
  - withBlockSize
```
public RocksDbTableDescriptor<K,V> withBlockSize(int blockSize)
```
    If compression is enabled, RocksDB groups approximately this many uncompressed bytes into one compressed block. You probably don't need to change this property.
    Default value is 4,096.
    Refer to stores.store-name.rocksdb.block.size.bytes in Samza configuration guide
    
    Parameters:
    
    blockSize - the block size in bytes
    
    Returns:
    
    this table descriptor instance
  - withTtl
```
public RocksDbTableDescriptor<K,V> withTtl(long ttl)
```
    The time-to-live of the store. Please note it's not a strict TTL limit (removed only after compaction). Please use caution opening a database with and without TTL, as it might corrupt the database. Please make sure to read the constraints before using.
    Refer to stores.store-name.rocksdb.ttl.ms in Samza configuration guide
    
    Parameters:
    
    ttl - the time to live in milliseconds
    
    Returns:
    
    this table descriptor instance
  - withCompactionStyle
```
public RocksDbTableDescriptor<K,V> withCompactionStyle(java.lang.String compactionStyle)
```
    This property controls the compaction style that RocksDB will employ when compacting its levels. The following values are valid:
    - universal Use universal compaction.
    - fifo Use FIFO compaction.
    - level Use RocksDB's standard leveled compaction.
    Default value is universal.
    Refer to stores.store-name.rocksdb.compaction.style in Samza configuration guide
    Parameters:
    
    compactionStyle - the compaction style
    
    Returns:
    
    this table descriptor instance
  - withNumWriteBuffers
```
public RocksDbTableDescriptor<K,V> withNumWriteBuffers(int numWriteBuffers)
```
    Configures the number of write buffers that a RocksDB store uses. This allows RocksDB to continue taking writes to other buffers even while a given write buffer is being flushed to disk.
    Default value is 3.
    Refer to stores.store-name.rocksdb.num.write.buffers in Samza configuration guide
    
    Parameters:
    
    numWriteBuffers - the number of write buffers
    
    Returns:
    
    this table descriptor instance
  - withMaxLogFileSize
```
public RocksDbTableDescriptor<K,V> withMaxLogFileSize(int maxLogFileSize)
```
    The maximum size in bytes of the RocksDB LOG file before it is rotated.
    Default value is 67,108,864.
    Refer to stores.store-name.rocksdb.max.log.file.size.bytes in Samza configuration guide
    
    Parameters:
    
    maxLogFileSize - the maximal log file size in bytes
    
    Returns:
    
    this table descriptor instance
  - withNumLogFilesToKeep
```
public RocksDbTableDescriptor<K,V> withNumLogFilesToKeep(int numLogFilesToKeep)
```
    The number of RocksDB LOG files (including rotated LOG.old.* files) to keep.
    Default value is 2.
    Refer to stores.store-name.rocksdb.keep.log.file.num in Samza configuration guide
    
    Parameters:
    
    numLogFilesToKeep - the number of log files to keep
    
    Returns:
    
    this table descriptor instance
  - getProviderFactoryClassName
```
public java.lang.String getProviderFactoryClassName()
```
    Specified by:
    
    getProviderFactoryClassName in class org.apache.samza.table.descriptors.BaseTableDescriptor<K,V,RocksDbTableDescriptor<K,V>>
  - toConfig
```
public java.util.Map<java.lang.String,java.lang.String> toConfig(org.apache.samza.config.Config jobConfig)
```
    Specified by:
    
    toConfig in interface org.apache.samza.table.descriptors.TableDescriptor<K,V,RocksDbTableDescriptor<K,V>>
    
    Overrides:
    
    toConfig in class org.apache.samza.table.descriptors.LocalTableDescriptor<K,V,RocksDbTableDescriptor<K,V>>

Class RocksDbTableDescriptor<K,V>

Field Summary

Fields inherited from class org.apache.samza.table.descriptors.LocalTableDescriptor

Fields inherited from class org.apache.samza.table.descriptors.BaseTableDescriptor

Constructor Summary

Method Summary

Methods inherited from class org.apache.samza.table.descriptors.LocalTableDescriptor

Methods inherited from class org.apache.samza.table.descriptors.BaseTableDescriptor

Methods inherited from class java.lang.Object

Field Detail

WRITE_BATCH_SIZE

OBJECT_CACHE_SIZE

CONTAINER_CACHE_SIZE_BYTES

CONTAINER_WRITE_BUFFER_SIZE_BYTES

ROCKSDB_COMPRESSION

ROCKSDB_BLOCK_SIZE_BYTES

ROCKSDB_TTL_MS

ROCKSDB_COMPACTION_STYLE

ROCKSDB_NUM_WRITE_BUFFERS

ROCKSDB_MAX_LOG_FILE_SIZE_BYTES

ROCKSDB_KEEP_LOG_FILE_NUM

Constructor Detail

RocksDbTableDescriptor

Method Detail

withWriteBatchSize

withObjectCacheSize

withCacheSize

withWriteBufferSize

withCompressionType

withBlockSize

withTtl

withCompactionStyle

withNumWriteBuffers

withMaxLogFileSize

withNumLogFilesToKeep

getProviderFactoryClassName

toConfig