com.datadoghq.sketch.ddsketch

Class DDSketch

java.lang.Object

com.datadoghq.sketch.ddsketch.DDSketch

All Implemented Interfaces:

QuantileSketch<DDSketch>, java.util.function.DoubleConsumer

public class DDSketch
extends java.lang.Object
implements QuantileSketch<DDSketch>

A QuantileSketch with relative-error guarantees. This sketch computes quantile values with an approximation error that is relative to the actual quantile value. It works with both positive and negative input values.

For instance, using DDSketch with a relative accuracy guarantee set to 1%, if the expected quantile value is 100, the computed quantile value is guaranteed to be between 99 and 101. If the expected quantile value is 1000, the computed quantile value is guaranteed to be between 990 and 1010.

DDSketch works by mapping floating-point input values to bins and counting the number of values for each bin. The mapping to bins is handled by IndexMapping, while the underlying structure that keeps track of bin counts is Store. The IndexMapping and the Store supplier are provided when constructing the sketch and can be adjusted based on use-case requirements. See DDSketches for preset sketches and some explanation about the involved tradeoffs.

Note that negative values are inverted before being mapped to the store. That means that if you use a store that collapses the lowest (resp., the highest) indexes, it will affect the input values that are the closest to (resp., the farthest away from) zero.

Note that this implementation is not thread-safe.

Constructor Summary

Constructors

Constructor and Description
DDSketch(double relativeAccuracy) Constructs a simple instance of DDSketch with the provided relative accuracy guarantee.
DDSketch(IndexMapping indexMapping, java.util.function.Supplier<Store> storeSupplier) Constructs an initially empty quantile sketch using the specified IndexMapping and Store supplier.
DDSketch(IndexMapping indexMapping, java.util.function.Supplier<Store> negativeValueStoreSupplier, java.util.function.Supplier<Store> positiveValueStoreSupplier) Constructs an initially empty quantile sketch using the specified IndexMapping and Store suppliers.
DDSketch(IndexMapping indexMapping, java.util.function.Supplier<Store> negativeValueStoreSupplier, java.util.function.Supplier<Store> positiveValueStoreSupplier, double minIndexedValue) Constructs an initially empty quantile sketch using the specified IndexMapping and Store supplier.

Method Summary

Modifier and Type	Method and Description
void	accept(double value) Adds a value to the sketch.
void	accept(double value, double count) Adds a value to the sketch with a floating-point count.
void	accept(double value, long count) Adds a value to the sketch as many times as specified by count.
static DDSketch	balanced(double relativeAccuracy) Deprecated. Use DDSketches.unboundedDense(double).
static DDSketch	balancedCollapsingHighest(double relativeAccuracy, int maxNumBins) Deprecated. Use DDSketches.collapsingHighestDense(double, int).
static DDSketch	balancedCollapsingLowest(double relativeAccuracy, int maxNumBins) Deprecated. Use DDSketches.collapsingLowestDense(double, int).
void	clear() Sets all counts in the sketch to zero.
DDSketch	copy()
static DDSketch	fast(double relativeAccuracy) Deprecated. Use new DDSketch(new BitwiseLinearlyInterpolatedMapping(relativeAccuracy), UnboundedSizeDenseStore::new).
static DDSketch	fastCollapsingHighest(double relativeAccuracy, int maxNumBins) Deprecated. Use new DDSketch(new BitwiseLinearlyInterpolatedMapping(relativeAccuracy), () -> new CollapsingHighestDenseStore(maxNumBins)).
static DDSketch	fastCollapsingLowest(double relativeAccuracy, int maxNumBins) Deprecated. Use new DDSketch(new BitwiseLinearlyInterpolatedMapping(relativeAccuracy), () -> new CollapsingLowestDenseStore(maxNumBins).
static DDSketch	fromProto(java.util.function.Supplier<? extends Store> storeSupplier, com.datadoghq.sketch.ddsketch.proto.DDSketch proto) Builds a new instance of DDSketch based on the provided protobuf representation.
double	getCount()
IndexMapping	getIndexMapping()
double	getMaxValue()
double	getMinValue()
Store	getNegativeValueStore()
Store	getPositiveValueStore()
double	getValueAtQuantile(double quantile)
double[]	getValuesAtQuantiles(double[] quantiles)
boolean	isEmpty()
static DDSketch	memoryOptimal(double relativeAccuracy) Deprecated. Use DDSketches.logarithmicCollapsingLowestDense(double, int).
static DDSketch	memoryOptimalCollapsingHighest(double relativeAccuracy, int maxNumBins) Deprecated. Use DDSketches.logarithmicCollapsingHighestDense(double, int).
static DDSketch	memoryOptimalCollapsingLowest(double relativeAccuracy, int maxNumBins) Deprecated. Use DDSketches.logarithmicCollapsingLowestDense(double, int).
void	mergeWith(DDSketch other) Merges the other sketch into this one.
com.datadoghq.sketch.ddsketch.proto.DDSketch	toProto() Generates a protobuf representation of this DDSketch.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface java.util.function.DoubleConsumer

andThen

Constructor Detail

DDSketch

public DDSketch(IndexMapping indexMapping,
                java.util.function.Supplier<Store> storeSupplier)

Constructs an initially empty quantile sketch using the specified IndexMapping and Store supplier.

Parameters:

indexMapping - the mapping between floating-point values and integer indices to be used by the sketch

storeSupplier - the store constructor for keeping track of added values

See Also:

DDSketches

DDSketch

public DDSketch(IndexMapping indexMapping,
                java.util.function.Supplier<Store> negativeValueStoreSupplier,
                java.util.function.Supplier<Store> positiveValueStoreSupplier)

Constructs an initially empty quantile sketch using the specified IndexMapping and Store suppliers.

Parameters:

indexMapping - the mapping between floating-point values and integer indices to be used by the sketch

negativeValueStoreSupplier - the store constructor for keeping track of added negative values

positiveValueStoreSupplier - the store constructor for keeping track of added positive values

See Also:

DDSketches

DDSketch

public DDSketch(IndexMapping indexMapping,
                java.util.function.Supplier<Store> negativeValueStoreSupplier,
                java.util.function.Supplier<Store> positiveValueStoreSupplier,
                double minIndexedValue)

Constructs an initially empty quantile sketch using the specified IndexMapping and Store supplier.

Parameters:

indexMapping - the mapping between floating-point values and integer indices to be used by the sketch

negativeValueStoreSupplier - the store constructor for keeping track of added negative values

positiveValueStoreSupplier - the store constructor for keeping track of added positive values

minIndexedValue - the least value that should be distinguished from zero

See Also:

DDSketches

DDSketch

public DDSketch(double relativeAccuracy)

Constructs a simple instance of DDSketch with the provided relative accuracy guarantee.

This is a convenience constructor, mostly for testing purposes. For preset sketches that are well-suited to a production setting and to situations where performance and memory usage constraints are involved, see DDSketches.

Parameters:

relativeAccuracy - the relative accuracy guarantee of the constructed sketch

See Also:

DDSketches

Method Detail

getIndexMapping

public IndexMapping getIndexMapping()

getNegativeValueStore

public Store getNegativeValueStore()

getPositiveValueStore

public Store getPositiveValueStore()

accept

public void accept(double value)

Adds a value to the sketch.

Specified by:

accept in interface QuantileSketch<DDSketch>

Specified by:

accept in interface java.util.function.DoubleConsumer

Parameters:

value - the value to be added

Throws:

java.lang.IllegalArgumentException - if the value is outside the range that is tracked by the sketch

accept

public void accept(double value,
                   long count)

Adds a value to the sketch as many times as specified by count.

Specified by:

accept in interface QuantileSketch<DDSketch>

Parameters:

value - the value to be added

count - the number of times the value is to be added

Throws:

java.lang.IllegalArgumentException - if the value is outside the range that is tracked by the sketch

accept

public void accept(double value,
                   double count)

Adds a value to the sketch with a floating-point count.

Parameters:

value - the value to be added

count - the weight associated with the value to be added

Throws:

java.lang.IllegalArgumentException - if count is negative

mergeWith

public void mergeWith(DDSketch other)

Merges the other sketch into this one. After this operation, this sketch encodes the values that were added to both this and the other sketches.

Specified by:

mergeWith in interface QuantileSketch<DDSketch>

Parameters:

other - the sketch to be merged into this one

Throws:

java.lang.IllegalArgumentException - if the other sketch does not use the same index mapping

copy

public DDSketch copy()

Specified by:

copy in interface QuantileSketch<DDSketch>

Returns:

a (deep) copy of this sketch

isEmpty

public boolean isEmpty()

Specified by:

isEmpty in interface QuantileSketch<DDSketch>

Returns:

iff no value has been added to this sketch

clear

public void clear()

Description copied from interface: QuantileSketch

Sets all counts in the sketch to zero. The sketch behaves as if it were empty after this call, but any allocated memory is not released.

Specified by:

clear in interface QuantileSketch<DDSketch>

getCount

public double getCount()

Specified by:

getCount in interface QuantileSketch<DDSketch>

Returns:

the total number of values that have been added to this sketch

getMinValue

public double getMinValue()

Specified by:

getMinValue in interface QuantileSketch<DDSketch>

Returns:

the minimum value that has been added to this sketch

getMaxValue

public double getMaxValue()

Specified by:

getMaxValue in interface QuantileSketch<DDSketch>

Returns:

the maximum value that has been added to this sketch

getValueAtQuantile

public double getValueAtQuantile(double quantile)

Specified by:

getValueAtQuantile in interface QuantileSketch<DDSketch>

Parameters:

quantile - a number between 0 and 1 (both included)

Returns:

the value at the specified quantile

getValuesAtQuantiles

public double[] getValuesAtQuantiles(double[] quantiles)

Specified by:

getValuesAtQuantiles in interface QuantileSketch<DDSketch>

Parameters:

quantiles - number between 0 and 1 (both included)

Returns:

the values at the respective specified quantiles

toProto

public com.datadoghq.sketch.ddsketch.proto.DDSketch toProto()

Generates a protobuf representation of this DDSketch.

Returns:

a protobuf representation of this DDSketch

fromProto

public static DDSketch fromProto(java.util.function.Supplier<? extends Store> storeSupplier,
                                 com.datadoghq.sketch.ddsketch.proto.DDSketch proto)

Builds a new instance of DDSketch based on the provided protobuf representation.

Parameters:

storeSupplier - the constructor of the Store implementation to be used for encoding bin counters

proto - the protobuf representation of a sketch

Returns:

an instance of DDSketch that matches the protobuf representation

balanced

@Deprecated
public static DDSketch balanced(double relativeAccuracy)

Deprecated. Use DDSketches.unboundedDense(double).

balancedCollapsingLowest

@Deprecated
public static DDSketch balancedCollapsingLowest(double relativeAccuracy,
                                                            int maxNumBins)

Deprecated. Use DDSketches.collapsingLowestDense(double, int).

balancedCollapsingHighest

@Deprecated
public static DDSketch balancedCollapsingHighest(double relativeAccuracy,
                                                             int maxNumBins)

Deprecated. Use DDSketches.collapsingHighestDense(double, int).

fast

@Deprecated
public static DDSketch fast(double relativeAccuracy)

Deprecated. Use new DDSketch(new BitwiseLinearlyInterpolatedMapping(relativeAccuracy), UnboundedSizeDenseStore::new).

fastCollapsingLowest

@Deprecated
public static DDSketch fastCollapsingLowest(double relativeAccuracy,
                                                        int maxNumBins)

Deprecated. Use new DDSketch(new BitwiseLinearlyInterpolatedMapping(relativeAccuracy), () -> new CollapsingLowestDenseStore(maxNumBins).

fastCollapsingHighest

@Deprecated
public static DDSketch fastCollapsingHighest(double relativeAccuracy,
                                                         int maxNumBins)

Deprecated. Use new DDSketch(new BitwiseLinearlyInterpolatedMapping(relativeAccuracy), () -> new CollapsingHighestDenseStore(maxNumBins)).

memoryOptimal

@Deprecated
public static DDSketch memoryOptimal(double relativeAccuracy)

Deprecated. Use DDSketches.logarithmicCollapsingLowestDense(double, int).

memoryOptimalCollapsingLowest

@Deprecated
public static DDSketch memoryOptimalCollapsingLowest(double relativeAccuracy,
                                                                 int maxNumBins)

Deprecated. Use DDSketches.logarithmicCollapsingLowestDense(double, int).

memoryOptimalCollapsingHighest

@Deprecated
public static DDSketch memoryOptimalCollapsingHighest(double relativeAccuracy,
                                                                  int maxNumBins)

Deprecated. Use DDSketches.logarithmicCollapsingHighestDense(double, int).