one.microstream.storage.types

Interface StorageDataFileEvaluator

All Superinterfaces:

StorageDataFileDissolvingEvaluator

All Known Implementing Classes:

StorageDataFileEvaluator.Default

public interface StorageDataFileEvaluator
extends StorageDataFileDissolvingEvaluator

Function type that evaluates if a storage file needs to be dissolved and its remaining data content be transferred to a new file or if the current head storage file needs to retire and be replaced by a new one.

Note that any implementation of this type must be safe enough to never throw an exception as this would doom the storage thread that executes it. Catching any exception would not prevent the problem for the channel thread as the function has to work in order for the channel to work properly. It is therefore strongly suggested that implementations only use "exception free" logic (like simple arithmetic) or handle any possible exception internally.

Nested Class Summary

Nested Classes

Modifier and Type	Interface and Description
static class	StorageDataFileEvaluator.Default
static interface	StorageDataFileEvaluator.Defaults
static interface	StorageDataFileEvaluator.Validation

Method Summary

Modifier and Type	Method and Description
int	fileMaximumSize()
int	fileMinimumSize()
boolean	needsDissolving(StorageLiveDataFile storageFile)
boolean	needsRetirement(long fileTotalLength)
static StorageDataFileEvaluator	New() Pseudo-constructor method to create a new StorageDataFileEvaluator instance using default values specified by StorageDataFileEvaluator.Defaults.
static StorageDataFileEvaluator	New(double minimumUseRatio) Pseudo-constructor method to create a new StorageDataFileEvaluator instance using the passed value and default values specified by StorageDataFileEvaluator.Defaults.
static StorageDataFileEvaluator	New(int fileMinimumSize, int fileMaximumSize) Pseudo-constructor method to create a new StorageDataFileEvaluator instance using the passed values and default values specified by StorageDataFileEvaluator.Defaults.
static StorageDataFileEvaluator	New(int fileMinimumSize, int fileMaximumSize, double minimumUseRatio) Pseudo-constructor method to create a new StorageDataFileEvaluator instance using the passed values.
static StorageDataFileEvaluator	New(int fileMinimumSize, int fileMaximumSize, double minimumUseRatio, boolean cleanUpHeadFile) Pseudo-constructor method to create a new StorageDataFileEvaluator instance using the passed values and default values specified by StorageDataFileEvaluator.Defaults.

Method Detail

needsDissolving

boolean needsDissolving(StorageLiveDataFile storageFile)

Specified by:

needsDissolving in interface StorageDataFileDissolvingEvaluator

needsRetirement

boolean needsRetirement(long fileTotalLength)

fileMinimumSize

int fileMinimumSize()

fileMaximumSize

int fileMaximumSize()

New

static StorageDataFileEvaluator New()

Pseudo-constructor method to create a new StorageDataFileEvaluator instance using default values specified by StorageDataFileEvaluator.Defaults.

For explanations and customizing values, see New(int, int, double).

Returns:

a new StorageDataFileEvaluator instance.

See Also:

New(int, int), New(double), New(int, int, double), StorageDataFileEvaluator.Defaults

New

static StorageDataFileEvaluator New(double minimumUseRatio)

Pseudo-constructor method to create a new StorageDataFileEvaluator instance using the passed value and default values specified by StorageDataFileEvaluator.Defaults.

For explanations and customizing values, see New(int, int, double).

Parameters:

minimumUseRatio - the ratio (value in ]0.0;1.0]) of non-gap data contained in a storage file to prevent the file from being dissolved. "Gap" data is anything that is not the latest version of an entity's data, inluding older versions of an entity and "comment" bytes (a sequence of bytes beginning with its length as a negative value length header).
The closer this value is to 1.0 (100%), the less disk space is occupied by storage files, but the more file dissolving (data transfers to new files) is required and vice versa.

Returns:

a new StorageDataFileEvaluator instance.

See Also:

New(), New(int, int), New(int, int, double), StorageDataFileEvaluator.Defaults

New

static StorageDataFileEvaluator New(int fileMinimumSize,
                                    int fileMaximumSize)

Pseudo-constructor method to create a new StorageDataFileEvaluator instance using the passed values and default values specified by StorageDataFileEvaluator.Defaults.

For explanations and customizing values, see New(int, int, double).

Parameters:

fileMinimumSize - the minimum file size in bytes that a single storage file must have. Smaller files will be dissolved.

fileMaximumSize - the maximum file size in bytes that a single storage file may have. Larger files will be dissolved.
Note that a file can exceed this limit if it contains a single entity that already exceeds the limit. E.g. an int array with 10 million elements would be about 40 MB in size and would exceed a file size limit of, for example, 30 MB.

Returns:

a new StorageDataFileEvaluator instance.

See Also:

New(), New(double), New(int, int, double), StorageDataFileEvaluator.Defaults

New

static StorageDataFileEvaluator New(int fileMinimumSize,
                                    int fileMaximumSize,
                                    double minimumUseRatio)

Pseudo-constructor method to create a new StorageDataFileEvaluator instance using the passed values.

A StorageDataFileEvaluator is used to evaluate storage data files regarding if they should be desolved, meaning copy all their still needed data to the current head file and then delete the file.
This technique is used to optimize (or "clean up") the occupied storage space: No longer needed data, e.g. of entities that became unreachable or prior versions of a still reachable entity, is considered a logical "gap", that occupies storage space unnecessarily. These "gaps" have to be removed in order to only occupy as much storage space as required.

The parameters defined here give an opportunity to configure how "aggressive" this clean up shall be performaned. The trade-off is between acceptale gap sizes and the required performance and disk-writes to clean them up.
Without specifying them, default values will be used, defined in StorageDataFileEvaluator.Defaults.

Parameters:

fileMinimumSize - the minimum file size in bytes that a single storage file must have. Smaller files will be dissolved.

fileMaximumSize - the maximum file size in bytes that a single storage file may have. Larger files will be dissolved.
Note that a file can exceed this limit if it contains a single entity that already exceeds the limit. E.g. an int array with 10 million elements would be about 40 MB in size and would exceed a file size limit of, for example, 30 MB.

minimumUseRatio - the ratio (value in ]0.0;1.0]) of non-gap data contained in a storage file to prevent the file from being dissolved. "Gap" data is anything that is not the latest version of an entity's data, inluding older versions of an entity and "comment" bytes (a sequence of bytes beginning with its length as a negative value length header).
The closer this value is to 1.0 (100%), the less disk space is occupied by storage files, but the more file dissolving (data transfers to new files) is required and vice versa.

Returns:

a new StorageDataFileEvaluator instance.

See Also:

New(), New(double), New(int, int), StorageDataFileEvaluator.Defaults

New

static StorageDataFileEvaluator New(int fileMinimumSize,
                                    int fileMaximumSize,
                                    double minimumUseRatio,
                                    boolean cleanUpHeadFile)

Pseudo-constructor method to create a new StorageDataFileEvaluator instance using the passed values and default values specified by StorageDataFileEvaluator.Defaults.

For explanations and customizing values, see New(int, int, double).

Parameters:

fileMinimumSize - the minimum file size in bytes that a single storage file must have. Smaller files will be dissolved.

fileMaximumSize - the maximum file size in bytes that a single storage file may have. Larger files will be dissolved.
Note that a file can exceed this limit if it contains a single entity that already exceeds the limit. E.g. an int array with 10 million elements would be about 40 MB in size and would exceed a file size limit of, for example, 30 MB.

minimumUseRatio - the ratio (value in ]0.0;1.0]) of non-gap data contained in a storage file to prevent the file from being dissolved. "Gap" data is anything that is not the latest version of an entity's data, inluding older versions of an entity and "comment" bytes (a sequence of bytes beginning with its length as a negative value length header).
The closer this value is to 1.0 (100%), the less disk space is occupied by storage files, but the more file dissolving (data transfers to new files) is required and vice versa.

cleanUpHeadFile - a flag defining wether the current head file (the only file actively written to) shall be subjected to file cleanups as well.

Returns:

a new StorageDataFileEvaluator instance.

See Also:

New(), New(double), New(int, int), New(int, int, double), StorageDataFileEvaluator.Defaults