public class Storage extends Object
Log factory.
This class provides a factory for Log objects. Storage objects are immutable and
can be created only via the Storage.Builder. To create a new
Storage.Builder, use the static builder() factory method:
Storage storage = Storage.builder()
.withDirectory(new File("logs"))
.withStorageLevel(StorageLevel.DISK)
.build();
Users can also configure a number of options related to how logs are constructed and managed.
Most notable of the configuration options is the number of compactionThreads(), which specifies the
number of background threads to use to compact log segments. The parallelism of the log
compaction algorithm will be limited by the number of compactionThreads().Log| Modifier and Type | Class and Description |
|---|---|
static class |
Storage.Builder
Builds a
Storage configuration. |
| Constructor and Description |
|---|
Storage() |
Storage(File directory) |
Storage(File directory,
StorageLevel storageLevel) |
Storage(StorageLevel storageLevel) |
Storage(String directory) |
Storage(String directory,
StorageLevel storageLevel) |
| Modifier and Type | Method and Description |
|---|---|
static Storage.Builder |
builder()
Returns a new storage builder.
|
int |
compactionThreads()
Returns the number of log compaction threads.
|
double |
compactionThreshold()
Returns the compaction threshold.
|
void |
deleteLog(String name)
Deletes a
Log from disk. |
void |
deleteMetaStore(String name)
Deletes a
MetaStore from disk. |
void |
deleteSnapshotStore(String name)
Deletes a
SnapshotStore from disk. |
File |
directory()
Returns the storage directory.
|
int |
entryBufferSize()
Returns the entry buffer size.
|
boolean |
flushOnCommit()
Returns whether to flush buffers to disk when entries are committed.
|
StorageLevel |
level()
Returns the storage level.
|
Duration |
majorCompactionInterval()
Returns the major compaction interval.
|
int |
maxEntriesPerSegment()
Returns the maximum number of entries per segment.
|
int |
maxSegmentSize()
Returns the maximum log segment size.
|
Duration |
minorCompactionInterval()
Returns the minor compaction interval.
|
Log |
openLog(String name)
Opens a new
Log, recovering the log from disk if it exists. |
MetaStore |
openMetaStore(String name)
Opens a new
MetaStore, recovering metadata from disk if it exists. |
SnapshotStore |
openSnapshotStore(String name)
Opens a new
SnapshotStore, recovering snapshots from disk if they exist. |
boolean |
retainStaleSnapshots()
Returns a boolean value indicating whether to retain stale snapshots on disk.
|
String |
toString() |
public Storage()
public Storage(StorageLevel storageLevel)
public Storage(String directory)
NullPointerException - if directory is nullpublic Storage(File directory)
NullPointerException - if directory is nullpublic Storage(String directory, StorageLevel storageLevel)
NullPointerException - if directory is nullpublic Storage(File directory, StorageLevel storageLevel)
NullPointerException - if directory is nullpublic static Storage.Builder builder()
public File directory()
The storage directory is the directory to which all Logs write Segment files. Segment files
for multiple logs may be stored in the storage directory, and files for each log instance will be identified
by the name provided when the log is opened.
public StorageLevel level()
The storage level dictates how entries within individual log Segments should be stored.
public int maxSegmentSize()
The maximum segment size dictates the maximum size any Segment in a Log may consume
in bytes.
public int maxEntriesPerSegment()
The maximum entries per segment dictates the maximum number of entries
that are allowed to be stored in any Segment in a Log.
public int entryBufferSize()
The entry buffer size dictates the number of entries that will be held in memory for read operations at the tail of the log.
public boolean flushOnCommit()
public boolean retainStaleSnapshots()
If this option is enabled, snapshots will be retained on disk even after they no longer contribute to the state of the system (there's a more recent snapshot). Users may want to disable this option for backup purposes.
public int compactionThreads()
The compaction thread count dictates the parallelism with which the log
Compactor can rewrite segments in the log.
public Duration minorCompactionInterval()
The minor compaction interval dictates the interval at which the
MinorCompactionManager should evaluate Segments
in the log for minor compaction.
public Duration majorCompactionInterval()
The major compaction interval dictates the interval at which the
MajorCompactionManager should evaluate Segments
in the log for major compaction.
public double compactionThreshold()
The compaction threshold is used during minor compaction
to determine the set of segments to compact.
public MetaStore openMetaStore(String name)
MetaStore, recovering metadata from disk if it exists.
The meta store will be loaded using based on the configured StorageLevel. If the storage level is persistent
then the meta store will be loaded from disk, otherwise a new meta store will be created.
name - The metastore name.public void deleteMetaStore(String name)
MetaStore from disk.
The meta store will be deleted by simply reading meta file names from disk and deleting metadata
files directly. Deleting the meta store does not involve reading any metadata files into memory.
name - The metastore name.public SnapshotStore openSnapshotStore(String name)
SnapshotStore, recovering snapshots from disk if they exist.
The snapshot store will be loaded using based on the configured StorageLevel. If the storage level is persistent
then the snapshot store will be loaded from disk, otherwise a new snapshot store will be created.
name - The snapshot store name.public void deleteSnapshotStore(String name)
SnapshotStore from disk.
The snapshot store will be deleted by simply reading snapshot file names from disk and deleting snapshot
files directly. Deleting the snapshot store does not involve reading any snapshot files into memory.
name - The snapshot store name.public Log openLog(String name)
Log, recovering the log from disk if it exists.
When a log is opened, the log will attempt to load Segments from the storage directory()
according to the provided log name. If segments for the given log name are present on disk, segments
will be loaded and indexes will be rebuilt from disk. If no segments are found, an empty log will be created.
When log files are loaded from disk, the file names are expected to be based on the provided log name.
name - The log name.public void deleteLog(String name)
Log from disk.
The log will be deleted by simply reading log file names from disk and deleting log files directly.
Deleting log files does not involve rebuilding indexes or reading any logs into memory.
name - The log name.Copyright © 2013–2016. All rights reserved.