org.apache.beam.fn.harness.debug

Class OutputSampler<T>

java.lang.Object

org.apache.beam.fn.harness.debug.OutputSampler<T>

Type Parameters:

T - the element type of the PCollection.

public class OutputSampler<T>
extends java.lang.Object

This class holds samples for a single PCollection until queried by the parent DataSampler. This class is meant to hold only a limited number of elements in memory. So old values are constantly being overridden in a circular buffer.

Constructor Summary

Constructors

Constructor and Description
OutputSampler(org.apache.beam.sdk.coders.Coder<?> coder, int maxElements, int sampleEveryN, boolean onlySampleExceptions)

Method Summary

Modifier and Type	Method and Description
void	exception(ElementSample<T> elementSample, java.lang.Exception e, java.lang.String ptransformId, java.lang.String processBundleId) Samples an exceptional element to be later queried.
ElementSample<T>	sample(org.apache.beam.sdk.util.WindowedValue<T> element) Samples every sampleEveryNth element or if it is part of the first 10 in the (local) PCollection.
java.util.List<org.apache.beam.model.fnexecution.v1.BeamFnApi.SampledElement>	samples() Clears samples at end of call.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

OutputSampler

public OutputSampler(org.apache.beam.sdk.coders.Coder<?> coder,
                     int maxElements,
                     int sampleEveryN,
                     boolean onlySampleExceptions)

Method Detail

sample

public ElementSample<T> sample(org.apache.beam.sdk.util.WindowedValue<T> element)

Samples every sampleEveryNth element or if it is part of the first 10 in the (local) PCollection.

This method is invoked in parallel by multiple bundle processing threads and in parallel to any samples() being returned to a thread handling a sample request.

Parameters:

element - the element to sample.

exception

public void exception(ElementSample<T> elementSample,
                      java.lang.Exception e,
                      java.lang.String ptransformId,
                      java.lang.String processBundleId)

Samples an exceptional element to be later queried. The enforces that only one exception occurs per bundle.

Parameters:

elementSample - the sampled element to add an exception to.

e - the exception.

ptransformId - the source of the exception.

processBundleId - the failing bundle.

samples

public java.util.List<org.apache.beam.model.fnexecution.v1.BeamFnApi.SampledElement> samples()
                                                                                      throws java.io.IOException

Clears samples at end of call. This is to help mitigate memory use.

This method is invoked by a thread handling a data sampling request in parallel to any calls to sample(org.apache.beam.sdk.util.WindowedValue<T>).

Returns:

samples taken since last call.

Throws:

java.io.IOException