Package io.airlift.stats

Class QuantileDigest

  • @NotThreadSafe
public class QuantileDigest
extends Object
    Implements http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.132.7343, a data structure for approximating quantiles by trading off error with memory requirements.

    The size of the digest is adjusted dynamically to achieve the error bound and requires O(log2(U) / maxError) space, where U is the number of bits needed to represent the domain of the values added to the digest. The error is defined as the discrepancy between the real rank of the value returned in a quantile query and the rank corresponding to the queried quantile.

    Thus, for a query for quantile q that returns value v, the error is |rank(v) - q * N| / N, where N is the number of elements added to the digest and rank(v) is the real rank of v

    This class also supports exponential decay. The implementation is based on the ideas laid out in http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.159.3978

    • Constructor Detail

      • QuantileDigest

        public QuantileDigest​(double maxError)

        Create a QuantileDigest with a maximum error guarantee of "maxError" and no decay.

        Parameters:
        maxError - the max error tolerance

      • QuantileDigest

        public QuantileDigest​(double maxError,
                      double alpha)

        Create a QuantileDigest with a maximum error guarantee of "maxError" and exponential decay with factor "alpha".

        Parameters:
        maxError - the max error tolerance
        alpha - the exponential decay factor

      • QuantileDigest

        public QuantileDigest​(io.airlift.slice.Slice serialized)

    • Method Detail

      • getMaxError

        public double getMaxError()

      • getAlpha

        public double getAlpha()

      • add

        public void add​(long value)

      • add

        public void add​(long value,
                double weight)
        Adds a value to this digest. The value must be >= 0

      • add

        public void add​(long value,
                long weight)

      • getQuantilesLowerBound

        public List<Long> getQuantilesLowerBound​(List<Double> quantiles)
        Get a lower bound on the quantiles for the given proportions. A returned q quantile is guaranteed to be within the q - maxError and q quantiles.

        The input list of quantile proportions must be sorted in increasing order, and each value must be in the range [0, 1]

      • getQuantilesUpperBound

        public List<Long> getQuantilesUpperBound​(List<Double> quantiles)
        Get an upper bound on the quantiles for the given proportions. A returned q quantile is guaranteed to be within the q and q + maxError quantiles.

        The input list of quantile proportions must be sorted in increasing order, and each value must be in the range [0, 1]

      • getQuantile

        public long getQuantile​(double quantile)
        Gets the value at the specified quantile +/- maxError. The quantile must be in the range [0, 1]

      • getQuantileLowerBound

        public long getQuantileLowerBound​(double quantile)

      • getQuantileUpperBound

        public long getQuantileUpperBound​(double quantile)

      • getCount

        public double getCount()
        Number (decayed) of elements added to this quantile digest

      • getMin

        public long getMin()

      • getMax

        public long getMax()

      • estimatedInMemorySizeInBytes

        public int estimatedInMemorySizeInBytes()

      • estimatedSerializedSizeInBytes

        public int estimatedSerializedSizeInBytes()

      • serialize

        public io.airlift.slice.Slice serialize()

      • getConfidenceFactor

        public double getConfidenceFactor()
        Computes the maximum error of the current digest

      • toGraphviz

        public String toGraphviz()