Package org.apache.beam.runners.spark.structuredstreaming.metrics

Class MetricsAccumulator

  • All Implemented Interfaces:
    java.io.Serializable, scala.Serializable
    public class MetricsAccumulator
extends org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,​org.apache.beam.runners.core.metrics.MetricsContainerStepMap>
    AccumulatorV2 for Beam metrics captured in MetricsContainerStepMap.
    See Also:
    accumulatorsV2, Serialized Form

    • Method Summary

      All Methods Static Methods Instance Methods Concrete Methods 
      Modifier and Type Method Description
      void add​(org.apache.beam.runners.core.metrics.MetricsContainerStepMap other)  
      static void clear()  
      MetricsAccumulator copy()  
      static MetricsAccumulator getInstance​(org.apache.spark.sql.SparkSession session)
      Get the MetricsAccumulator on this driver.
      boolean isZero()  
      void merge​(org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,​org.apache.beam.runners.core.metrics.MetricsContainerStepMap> other)  
      void reset()  
      org.apache.beam.runners.core.metrics.MetricsContainerStepMap value()  

      • Methods inherited from class org.apache.spark.util.AccumulatorV2

        copyAndReset, countFailedValues, id, isAtDriverSide, isRegistered, metadata, metadata_$eq, name, register, register$default$2, register$default$3, toInfo, toString, withBufferSerialized, writeReplace

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

    • Constructor Detail

      • MetricsAccumulator

        public MetricsAccumulator()

    • Method Detail

      • isZero

        public boolean isZero()
        Specified by:
        isZero in class org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,​org.apache.beam.runners.core.metrics.MetricsContainerStepMap>

      • copy

        public MetricsAccumulator copy()
        Specified by:
        copy in class org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,​org.apache.beam.runners.core.metrics.MetricsContainerStepMap>

      • reset

        public void reset()
        Specified by:
        reset in class org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,​org.apache.beam.runners.core.metrics.MetricsContainerStepMap>

      • add

        public void add​(org.apache.beam.runners.core.metrics.MetricsContainerStepMap other)
        Specified by:
        add in class org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,​org.apache.beam.runners.core.metrics.MetricsContainerStepMap>

      • merge

        public void merge​(org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,​org.apache.beam.runners.core.metrics.MetricsContainerStepMap> other)
        Specified by:
        merge in class org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,​org.apache.beam.runners.core.metrics.MetricsContainerStepMap>

      • value

        public org.apache.beam.runners.core.metrics.MetricsContainerStepMap value()
        Specified by:
        value in class org.apache.spark.util.AccumulatorV2<org.apache.beam.runners.core.metrics.MetricsContainerStepMap,​org.apache.beam.runners.core.metrics.MetricsContainerStepMap>

      • getInstance

        public static MetricsAccumulator getInstance​(org.apache.spark.sql.SparkSession session)
        Get the MetricsAccumulator on this driver. If there's no such accumulator yet, it will be created and registered using the provided SparkSession.

      • clear

        public static void clear()