package accumulo

Linear Supertypes
AnyRef, Any
Ordering
  1. Alphabetic
  2. By Inheritance
Inherited
  1. accumulo
  2. AnyRef
  3. Any
  1. Hide All
  2. Show All
Visibility
  1. Public
  2. All

Type Members

  1. class AccumuloAttributeStore extends DiscreteLayerAttributeStore
  2. class AccumuloCollectionLayerReader extends CollectionLayerReader[LayerId]
  3. trait AccumuloEncoder[T] extends AnyRef
  4. trait AccumuloInstance extends Serializable
  5. class AccumuloLayerCopier extends LayerCopier[LayerId]
  6. class AccumuloLayerDeleter extends LayerDeleter[LayerId]
  7. case class AccumuloLayerHeader(keyClass: String, valueClass: String, tileTable: String, layerType: LayerType = AvroLayerType) extends LayerHeader with Product with Serializable
  8. class AccumuloLayerManager extends LayerManager[LayerId]
  9. class AccumuloLayerProvider extends AttributeStoreProvider with LayerReaderProvider with LayerWriterProvider with ValueReaderProvider with CollectionLayerReaderProvider

    Provides AccumuloAttributeStore instance for URI with accumulo scheme.

    Provides AccumuloAttributeStore instance for URI with accumulo scheme. ex: accumulo://[user[:password]@]zookeeper/instance-name[?attributes=table1[&layers=table2]]

    Attributes table name is optional, not provided default value will be used. Layers table name is required to instantiate a LayerWriter

  10. class AccumuloLayerReader extends FilteringLayerReader[LayerId]
  11. class AccumuloLayerReindexer extends LayerReindexer[LayerId]
  12. class AccumuloLayerWriter extends LayerWriter[LayerId]
  13. class AccumuloValueReader extends OverzoomingValueReader
  14. sealed trait AccumuloWriteStrategy extends AnyRef
  15. case class BaseAccumuloInstance(instanceName: String, zookeeper: String, user: String, tokenBytes: (String, Array[Byte])) extends AccumuloInstance with Product with Serializable
  16. case class HdfsWriteStrategy(ingestPath: Path) extends AccumuloWriteStrategy with Product with Serializable

    This strategy will perfom Accumulo bulk ingest.

    This strategy will perfom Accumulo bulk ingest. Bulk ingest requires that sorted records be written to the filesystem, preferbly HDFS, before Accumulo is able to ingest them. After the ingest is finished the nodes will likely go through a period of high load as they perform major compactions.

    Note: Giving relative URLs will cause HDFS to use the fs.defaultFS property in core-site.xml. If not specified this will default to local ('file:/') system, this is undesriable.

    ingestPath

    Path where spark will write RDD records for ingest

  17. case class SocketWriteStrategy(config: BatchWriterConfig = ..., threads: Int = ...) extends AccumuloWriteStrategy with Product with Serializable

    This strategy will create one BatchWriter per partition and attempt to stream the records to the target tablets.

    This strategy will create one BatchWriter per partition and attempt to stream the records to the target tablets. In order to gain some parallism this strategy will create a number of splits in the target table equal to the number of tservers in the cluster. This is suitable for smaller ingests, or where HdfsWriteStrategy is otherwise not possible.

    This strategy will not create splits before starting to write. If you wish to do that use AccumuloUtils.getSplits first.

    There is a problem in Accumulo 1.6 (fixed in 1.7) where the split creation does not wait for the resulting empty tablets to distribute through the cluster before returning. This will create a warm-up period where the pressure the ingest writers on that node will delay tablet re-balancing.

    The speed of the ingest can be improved by setting tserver.wal.sync.method=hflush in accumulo shell. Note: this introduces higher chance of data loss due to sudden node failure.

    BatchWriter is notified of the tablet migrations and will follow them around the cluster.

    config

    Configuration for the BatchWriters

  18. implicit class connectorWriter extends AnyRef
  19. implicit class scannerIterator extends Iterator[(Key, Value)]

Value Members

  1. def columnFamily(id: LayerId): String
  2. implicit def stringToText(s: String): Text
  3. object AccumuloAttributeStore extends Serializable
  4. object AccumuloCollectionLayerReader
  5. object AccumuloCollectionReader
  6. object AccumuloInstance extends Serializable
  7. object AccumuloKeyEncoder
  8. object AccumuloLayerCopier
  9. object AccumuloLayerDeleter
  10. object AccumuloLayerHeader extends Serializable
  11. object AccumuloLayerMover
  12. object AccumuloLayerReader
  13. object AccumuloLayerReindexer
  14. object AccumuloLayerWriter
  15. object AccumuloRDDReader
  16. object AccumuloRDDWriter
  17. object AccumuloUtils
  18. object AccumuloValueReader
  19. object AccumuloWriteStrategy
  20. object HdfsWriteStrategy extends Serializable

Inherited from AnyRef

Inherited from Any

Ungrouped