Skip navigation links
A B C D E F G H I J K L M N O P R S T U V W 

A

addBoolean(Map<String, Object>, String, boolean) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
addCollectionToSingletonOutput(PCollection<?>, String, PCollectionView<?>) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.StepTranslationContext
Adds an output to this CollectionToSingleton Dataflow step, consuming the specified input PValue and producing the specified output PValue.
addDictionary(Map<String, Object>, String, Map<String, Object>) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
addDouble(Map<String, Object>, String, Double) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
addEncodingInput(Coder<?>) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.StepTranslationContext
Sets the encoding for this Dataflow step.
addInput(String, Boolean) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.StepTranslationContext
Adds an input with the given name and value to this Dataflow step.
addInput(String, String) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.StepTranslationContext
Adds an input with the given name and value to this Dataflow step.
addInput(String, Long) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.StepTranslationContext
Adds an input with the given name and value to this Dataflow step.
addInput(String, PInput) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.StepTranslationContext
Adds an input with the given name to this Dataflow step, coming from the specified input PValue.
addInput(String, Map<String, Object>) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.StepTranslationContext
Adds an input that is a dictionary of strings to objects.
addInput(String, List<? extends Map<String, Object>>) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.StepTranslationContext
Adds an input that is a list of objects.
addList(Map<String, Object>, String, List<? extends Map<String, Object>>) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
addList(Map<String, Object>, String, T[]) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
addLong(Map<String, Object>, String, long) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
addLongs(Map<String, Object>, String, long...) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
addNull(Map<String, Object>, String) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
addObject(Map<String, Object>, String, Map<String, Object>) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
addOutput(String, PCollection<?>) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.StepTranslationContext
Adds a primitive output to this Dataflow step with the given name as the local output name, producing the specified output PValue, including its Coder if a TypedPValue.
addOverrideForClass(Class<?>, DataflowWorkerLoggingOptions.Level) - Method in class org.apache.beam.runners.dataflow.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides
Deprecated.
Overrides the default log level for the passed in class.
addOverrideForName(String, DataflowWorkerLoggingOptions.Level) - Method in class org.apache.beam.runners.dataflow.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides
Deprecated.
Overrides the default log level for the passed in name.
addOverrideForPackage(Package, DataflowWorkerLoggingOptions.Level) - Method in class org.apache.beam.runners.dataflow.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides
Deprecated.
Overrides the default log level for the passed in package.
addStep(PTransform<?, ?>, String) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.TranslationContext
Adds a step to the Dataflow workflow for the given transform, with the given Dataflow step type.
addString(Map<String, Object>, String, String) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
addStringList(Map<String, Object>, String, List<String>) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
array() - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData
Returns the backing array.
asCloudObject(Coder<?>, SdkComponents) - Static method in class org.apache.beam.runners.dataflow.util.CloudObjects
Convert the provided Coder into a CloudObject.
asInputStream(int, int) - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData
Returns an InputStream wrapper which supplies the portion of this backing byte buffer starting at offset and up to length bytes.
asOutputReference(PValue, AppliedPTransform<?, ?, ?>) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.TranslationContext
Encode a PValue reference as an output reference.
asOutputStream() - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData
Returns an output stream which writes to the backing buffer from the current position.

B

BatchStatefulParDoOverrides - Class in org.apache.beam.runners.dataflow
PTransformOverrideFactories that expands to correctly implement stateful ParDo using window-unaware BatchViewOverrides.GroupByKeyAndSortValuesOnly to linearize processing per key.
BatchStatefulParDoOverrides() - Constructor for class org.apache.beam.runners.dataflow.BatchStatefulParDoOverrides
 
BatchStatefulParDoOverrides.BatchStatefulDoFn<K,V,OutputT> - Class in org.apache.beam.runners.dataflow
A key-preserving DoFn that explodes an iterable that has been grouped by key and window.

C

cancel() - Method in class org.apache.beam.runners.dataflow.DataflowPipelineJob
 
cancel() - Method in class org.apache.beam.runners.dataflow.util.DataflowTemplateJob
 
classesToTranslators() - Method in interface org.apache.beam.runners.dataflow.util.CoderCloudObjectTranslatorRegistrar
Gets a map from Coder to a CloudObjectTranslator that can translate that Coder.
classesToTranslators() - Method in class org.apache.beam.runners.dataflow.util.DefaultCoderCloudObjectTranslatorRegistrar
 
classNamesToTranslators() - Method in interface org.apache.beam.runners.dataflow.util.CoderCloudObjectTranslatorRegistrar
Gets a map from the name returned by CloudObject.getClassName() to a translator that can convert into the equivalent Coder.
classNamesToTranslators() - Method in class org.apache.beam.runners.dataflow.util.DefaultCoderCloudObjectTranslatorRegistrar
 
clone() - Method in class org.apache.beam.runners.dataflow.util.CloudObject
 
CloudDebuggerOptions - Interface in org.apache.beam.runners.dataflow.options
Options for controlling Cloud Debugger.
CloudObject - Class in org.apache.beam.runners.dataflow.util
A representation of an arbitrary Java object to be instantiated by Dataflow workers.
cloudObjectClassName() - Method in interface org.apache.beam.runners.dataflow.util.CloudObjectTranslator
Gets the class name that will represent the CloudObject created by this CloudObjectTranslator.
CloudObjects - Class in org.apache.beam.runners.dataflow.util
Utilities for converting an object to a CloudObject.
CloudObjectTranslator<T> - Interface in org.apache.beam.runners.dataflow.util
A translator that takes an object and creates a CloudObject which can be converted back to the original object.
CO_GBK_RESULT_SCHEMA - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
CoderCloudObjectTranslatorRegistrar - Interface in org.apache.beam.runners.dataflow.util
Coder authors have the ability to automatically have their Coder registered with the Dataflow Runner by creating a ServiceLoader entry and a concrete implementation of this interface.
coderFromCloudObject(CloudObject) - Static method in class org.apache.beam.runners.dataflow.util.CloudObjects
 
commonPrefixLength(RandomAccessData, RandomAccessData) - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData.UnsignedLexicographicalComparator
Compute the length of the common prefix of the two provided sets of bytes.
compare(JobMessage, JobMessage) - Method in class org.apache.beam.runners.dataflow.util.MonitoringUtil.TimeStampComparator
 
compare(RandomAccessData, RandomAccessData) - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData.UnsignedLexicographicalComparator
 
compare(RandomAccessData, RandomAccessData, int) - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData.UnsignedLexicographicalComparator
Compare the two sets of bytes starting at the given offset.
COMPONENT_ENCODINGS - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
consistentWithEquals() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.FooterCoder
 
consistentWithEquals() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecordCoder
 
consistentWithEquals() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmShardCoder
 
consistentWithEquals() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.KeyPrefixCoder
 
consistentWithEquals() - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData.RandomAccessDataCoder
 
copy() - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData
Returns a copy of this RandomAccessData.
create(DataflowPipelineOptions) - Static method in class org.apache.beam.runners.dataflow.DataflowClient
 
create(PCollectionView<?>, Coder<T>) - Static method in class org.apache.beam.runners.dataflow.DataflowRunner.StreamingPCollectionViewWriterFn
 
create(PipelineOptions) - Method in class org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions.DataflowClientFactory
 
create(PipelineOptions) - Method in class org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions.StagerFactory
 
create(PipelineOptions) - Method in class org.apache.beam.runners.dataflow.options.DataflowPipelineOptions.StagingLocationFactory
 
create(PipelineOptions) - Method in class org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions.WorkerHarnessContainerImageFactory
 
create() - Static method in class org.apache.beam.runners.dataflow.PrimitiveParDoSingleFactory.PayloadTranslator
 
CreateDataflowView<ElemT,ViewT> - Class in org.apache.beam.runners.dataflow
A DataflowRunner marker class for creating a PCollectionView.
createJob(Job) - Method in class org.apache.beam.runners.dataflow.DataflowClient
Creates the Dataflow Job.
CUSTOM_SOURCE_FORMAT - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
CustomSources - Class in org.apache.beam.runners.dataflow.internal
A helper class for supporting sources defined as Source.
CustomSources() - Constructor for class org.apache.beam.runners.dataflow.internal.CustomSources
 

D

DataflowClient - Class in org.apache.beam.runners.dataflow
Wrapper around the generated Dataflow client to provide common functionality.
DataflowClientFactory() - Constructor for class org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions.DataflowClientFactory
 
DataflowJobAlreadyExistsException - Exception in org.apache.beam.runners.dataflow
An exception that is thrown if the unique job name constraint of the Dataflow service is broken because an existing job with the same job name is currently active.
DataflowJobAlreadyExistsException(DataflowPipelineJob, String) - Constructor for exception org.apache.beam.runners.dataflow.DataflowJobAlreadyExistsException
Create a new DataflowJobAlreadyExistsException with the specified DataflowPipelineJob and message.
DataflowJobAlreadyUpdatedException - Exception in org.apache.beam.runners.dataflow
An exception that is thrown if the existing job has already been updated within the Dataflow service and is no longer able to be updated.
DataflowJobAlreadyUpdatedException(DataflowPipelineJob, String) - Constructor for exception org.apache.beam.runners.dataflow.DataflowJobAlreadyUpdatedException
Create a new DataflowJobAlreadyUpdatedException with the specified DataflowPipelineJob and message.
DataflowJobException - Exception in org.apache.beam.runners.dataflow
A RuntimeException that contains information about a DataflowPipelineJob.
DataflowPipelineDebugOptions - Interface in org.apache.beam.runners.dataflow.options
Internal.
DataflowPipelineDebugOptions.DataflowClientFactory - Class in org.apache.beam.runners.dataflow.options
Returns the default Dataflow client built from the passed in PipelineOptions.
DataflowPipelineDebugOptions.StagerFactory - Class in org.apache.beam.runners.dataflow.options
Creates a Stager object using the class specified in DataflowPipelineDebugOptions.getStagerClass().
DataflowPipelineJob - Class in org.apache.beam.runners.dataflow
A DataflowPipelineJob represents a job submitted to Dataflow using DataflowRunner.
DataflowPipelineJob(DataflowClient, String, DataflowPipelineOptions, Map<AppliedPTransform<?, ?, ?>, String>) - Constructor for class org.apache.beam.runners.dataflow.DataflowPipelineJob
Constructs the job.
DataflowPipelineOptions - Interface in org.apache.beam.runners.dataflow.options
Options that can be used to configure the DataflowRunner.
DataflowPipelineOptions.StagingLocationFactory - Class in org.apache.beam.runners.dataflow.options
Returns a default staging location under GcpOptions.getGcpTempLocation().
DataflowPipelineRegistrar - Class in org.apache.beam.runners.dataflow
Contains the PipelineOptionsRegistrar and PipelineRunnerRegistrar for the DataflowRunner.
DataflowPipelineRegistrar.Options - Class in org.apache.beam.runners.dataflow
DataflowPipelineRegistrar.Runner - Class in org.apache.beam.runners.dataflow
Register the DataflowRunner.
DataflowPipelineTranslator - Class in org.apache.beam.runners.dataflow
DataflowPipelineTranslator knows how to translate Pipeline objects into Cloud Dataflow Service API Jobs.
DataflowPipelineTranslator.JobSpecification - Class in org.apache.beam.runners.dataflow
The result of a job translation.
DataflowPipelineWorkerPoolOptions - Interface in org.apache.beam.runners.dataflow.options
Options that are used to configure the Dataflow pipeline worker pool.
DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType - Enum in org.apache.beam.runners.dataflow.options
Type of autoscaling algorithm to use.
DataflowPipelineWorkerPoolOptions.WorkerHarnessContainerImageFactory - Class in org.apache.beam.runners.dataflow.options
Returns the default Docker container image that executes Dataflow worker harness, residing in Google Container Registry.
DataflowProfilingAgentConfiguration() - Constructor for class org.apache.beam.runners.dataflow.options.DataflowProfilingOptions.DataflowProfilingAgentConfiguration
 
DataflowProfilingOptions - Interface in org.apache.beam.runners.dataflow.options
Options for controlling profiling of pipeline execution.
DataflowProfilingOptions.DataflowProfilingAgentConfiguration - Class in org.apache.beam.runners.dataflow.options
Configuration the for profiling agent.
DataflowRunner - Class in org.apache.beam.runners.dataflow
A PipelineRunner that executes the operations in the pipeline by first translating them to the Dataflow representation using the DataflowPipelineTranslator and then submitting them to a Dataflow service for execution.
DataflowRunner(DataflowPipelineOptions) - Constructor for class org.apache.beam.runners.dataflow.DataflowRunner
 
DataflowRunner.StreamingPCollectionViewWriterFn<T> - Class in org.apache.beam.runners.dataflow
A marker DoFn for writing the contents of a PCollection to a streaming PCollectionView backend implementation.
DataflowRunnerHooks - Class in org.apache.beam.runners.dataflow
An instance of this class can be passed to the DataflowRunner to add user defined hooks to be invoked at various times during pipeline execution.
DataflowRunnerHooks() - Constructor for class org.apache.beam.runners.dataflow.DataflowRunnerHooks
 
DataflowRunnerInfo - Class in org.apache.beam.runners.dataflow
Populates versioning and other information for DataflowRunner.
DataflowServiceException - Exception in org.apache.beam.runners.dataflow
Signals there was an error retrieving information about a job from the Cloud Dataflow Service.
DataflowTemplateJob - Class in org.apache.beam.runners.dataflow.util
A DataflowPipelineJob that is returned when --templateRunner is set.
DataflowTemplateJob() - Constructor for class org.apache.beam.runners.dataflow.util.DataflowTemplateJob
 
DataflowTransport - Class in org.apache.beam.runners.dataflow.util
Helpers for cloud communication.
DataflowTransport() - Constructor for class org.apache.beam.runners.dataflow.util.DataflowTransport
 
DataflowWorkerHarnessOptions - Interface in org.apache.beam.runners.dataflow.options
Options that are used exclusively within the Dataflow worker harness.
DataflowWorkerLoggingOptions - Interface in org.apache.beam.runners.dataflow.options
Deprecated.
This interface will no longer be the source of truth for worker logging configuration once jobs are executed using a dedicated SDK harness instead of user code being co-located alongside Dataflow worker code. Please set the option below and also the corresponding option within SdkHarnessOptions to ensure forward compatibility.
DataflowWorkerLoggingOptions.Level - Enum in org.apache.beam.runners.dataflow.options
Deprecated.
The set of log levels that can be used on the Dataflow worker.
DataflowWorkerLoggingOptions.WorkerLogLevelOverrides - Class in org.apache.beam.runners.dataflow.options
Deprecated.
Defines a log level override for a specific class, package, or name.
decode(InputStream) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.FooterCoder
 
decode(InputStream) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecordCoder
 
decode(InputStream) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmShardCoder
 
decode(InputStream) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.KeyPrefixCoder
 
decode(InputStream) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.MetadataKeyCoder
 
decode(InputStream) - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData.RandomAccessDataCoder
 
decode(InputStream, Coder.Context) - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData.RandomAccessDataCoder
 
DefaultCoderCloudObjectTranslatorRegistrar - Class in org.apache.beam.runners.dataflow.util
The CoderCloudObjectTranslatorRegistrar containing the default collection of Coder Cloud Object Translators.
DefaultCoderCloudObjectTranslatorRegistrar() - Constructor for class org.apache.beam.runners.dataflow.util.DefaultCoderCloudObjectTranslatorRegistrar
 
delegate() - Method in class org.apache.beam.runners.dataflow.PrimitiveParDoSingleFactory.ParDoSingle
 
DISALLOW_COMBINER_LIFTING - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
DISPLAY_DATA - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 

E

encode(IsmFormat.Footer, OutputStream) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.FooterCoder
 
encode(IsmFormat.IsmRecord<V>, OutputStream) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecordCoder
 
encode(IsmFormat.IsmShard, OutputStream) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmShardCoder
 
encode(IsmFormat.KeyPrefix, OutputStream) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.KeyPrefixCoder
 
encode(K, OutputStream) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.MetadataKeyCoder
 
encode(RandomAccessData, OutputStream) - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData.RandomAccessDataCoder
 
encode(RandomAccessData, OutputStream, Coder.Context) - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData.RandomAccessDataCoder
 
encodeAndHash(List<?>, RandomAccessData) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecordCoder
Computes the shard id for the given key component(s).
encodeAndHash(List<?>, RandomAccessData, List<Integer>) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecordCoder
Computes the shard id for the given key component(s).
ENCODING - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
ENVIRONMENT_VERSION_JOB_TYPE_KEY - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
ENVIRONMENT_VERSION_MAJOR_KEY - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
equals(Object) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecordCoder
 
equals(Object) - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData
 
expand(PCollection<ElemT>) - Method in class org.apache.beam.runners.dataflow.CreateDataflowView
 
expand(PCollection<? extends InputT>) - Method in class org.apache.beam.runners.dataflow.PrimitiveParDoSingleFactory.ParDoSingle
 

F

FIXED_LENGTH - Static variable in class org.apache.beam.runners.dataflow.internal.IsmFormat.Footer
 
Footer() - Constructor for class org.apache.beam.runners.dataflow.internal.IsmFormat.Footer
 
FooterCoder() - Constructor for class org.apache.beam.runners.dataflow.internal.IsmFormat.FooterCoder
 
forBatch(PCollectionView<ViewT>) - Static method in class org.apache.beam.runners.dataflow.CreateDataflowView
 
forBoolean(Boolean) - Static method in class org.apache.beam.runners.dataflow.util.CloudObject
Constructs a CloudObject representing the given value.
forClass(Class<?>) - Static method in class org.apache.beam.runners.dataflow.util.CloudObject
Constructs a CloudObject to be used for serializing an instance of the supplied class for transport via the Dataflow API.
forClassName(String) - Static method in class org.apache.beam.runners.dataflow.util.CloudObject
Constructs a CloudObject to be used for serializing data to be deserialized using the supplied class name the supplied class name for transport via the Dataflow API.
forFloat(Float) - Static method in class org.apache.beam.runners.dataflow.util.CloudObject
Constructs a CloudObject representing the given value.
forFloat(Double) - Static method in class org.apache.beam.runners.dataflow.util.CloudObject
Constructs a CloudObject representing the given value.
forInteger(Long) - Static method in class org.apache.beam.runners.dataflow.util.CloudObject
Constructs a CloudObject representing the given value.
forInteger(Integer) - Static method in class org.apache.beam.runners.dataflow.util.CloudObject
Constructs a CloudObject representing the given value.
forKnownType(Object) - Static method in class org.apache.beam.runners.dataflow.util.CloudObject
Constructs a CloudObject representing the given value of a well-known cloud object type.
FORMAT - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
forStreaming(PCollectionView<ViewT>) - Static method in class org.apache.beam.runners.dataflow.CreateDataflowView
 
forString(String) - Static method in class org.apache.beam.runners.dataflow.util.CloudObject
Constructs a CloudObject representing the given value.
from(Map<String, String>) - Static method in class org.apache.beam.runners.dataflow.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides
Deprecated.
Expects a map keyed by logger Names with values representing Levels.
fromCloudDuration(String) - Static method in class org.apache.beam.runners.dataflow.util.TimeUtil
Converts a Dataflow API duration string into a Duration.
fromCloudObject(CloudObject) - Method in interface org.apache.beam.runners.dataflow.util.CloudObjectTranslator
Converts back into the original object from a provided CloudObject.
fromCloudTime(String) - Static method in class org.apache.beam.runners.dataflow.util.TimeUtil
Converts a time value received via the Dataflow API into the corresponding Instant.
fromOptions(DataflowPipelineOptions) - Static method in class org.apache.beam.runners.dataflow.DataflowPipelineTranslator
Constructs a translator from the provided options.
fromOptions(PipelineOptions) - Static method in class org.apache.beam.runners.dataflow.DataflowRunner
Construct a runner from the provided options.
fromOptions(PipelineOptions) - Static method in class org.apache.beam.runners.dataflow.TestDataflowRunner
Constructs a runner from the provided options.
fromOptions(PipelineOptions) - Static method in class org.apache.beam.runners.dataflow.util.GcsStager
 
fromSpec(Map<String, Object>) - Static method in class org.apache.beam.runners.dataflow.util.CloudObject
Constructs a CloudObject by copying the supplied serialized object spec, which must represent an SDK object serialized for transport via the Dataflow API.

G

GcsStager - Class in org.apache.beam.runners.dataflow.util
Utility class for staging files to GCS.
getAdditionalInputs() - Method in class org.apache.beam.runners.dataflow.PrimitiveParDoSingleFactory.ParDoSingle
 
getAlgorithm() - Method in enum org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType
Returns the string representation of this type.
getApiRootUrl() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
The root URL for the Dataflow API.
getAutoscalingAlgorithm() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
[Experimental] The autoscaling algorithm to use for the workerpool.
getBlockOffset() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmShard
Return the absolute position within the Ism file where the data block begins.
getBloomFilterPosition() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.Footer
 
getBoolean(Map<String, Object>, String) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
getBoolean(Map<String, Object>, String, Boolean) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
getBytes(Map<String, Object>, String) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
getBytes(Map<String, Object>, String, byte[]) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
getClassName() - Method in class org.apache.beam.runners.dataflow.util.CloudObject
Gets the name of the Java class that this CloudObject represents.
getCoderArguments() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecordCoder
 
getCoderArguments() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.MetadataKeyCoder
 
getContainerVersion() - Method in class org.apache.beam.runners.dataflow.DataflowRunnerInfo
Provides the container version that will be used for constructing harness image paths.
getCurrentParent() - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.TranslationContext
Gets the parent composite transform to the current transform, if one exists.
getCurrentTransform() - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.TranslationContext
 
getDataCoder() - Method in class org.apache.beam.runners.dataflow.DataflowRunner.StreamingPCollectionViewWriterFn
 
getDataflowClient() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
An instance of the Dataflow client.
getDataflowEndpoint() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
Dataflow endpoint to use.
getDataflowJobFile() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
The path to write the translated Dataflow job specification out to at job submission time.
getDataflowOptions() - Method in class org.apache.beam.runners.dataflow.DataflowPipelineJob
 
getDataflowRunnerInfo() - Static method in class org.apache.beam.runners.dataflow.DataflowRunnerInfo
Returns an instance of DataflowRunnerInfo.
getDataflowWorkerJar() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
 
getDebuggee() - Method in interface org.apache.beam.runners.dataflow.options.CloudDebuggerOptions
The Cloud Debugger debuggee to associate with.
getDefaultWorkerLogLevel() - Method in interface org.apache.beam.runners.dataflow.options.DataflowWorkerLoggingOptions
Deprecated.
This option controls the default log level of all loggers without a log level override.
getDictionary(Map<String, Object>, String) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
getDictionary(Map<String, Object>, String, Map<String, Object>) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
getDiskSizeGb() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
Remote worker disk size, in gigabytes, or 0 to use the default size.
getDumpHeapOnOOM() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
If true, save a heap dump before killing a thread or process which is GC thrashing or out of memory.
getEnableCloudDebugger() - Method in interface org.apache.beam.runners.dataflow.options.CloudDebuggerOptions
Whether to enable the Cloud Debugger snapshot agent for the current job.
getEncodedElementByteSize(IsmFormat.Footer) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.FooterCoder
 
getEncodedElementByteSize(IsmFormat.KeyPrefix) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.KeyPrefixCoder
 
getEncodedElementByteSize(RandomAccessData) - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData.RandomAccessDataCoder
 
getFilesToStage() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
List of local files to make available to workers.
getFn() - Method in class org.apache.beam.runners.dataflow.PrimitiveParDoSingleFactory.ParDoSingle
 
getFnApiEnvironmentMajorVersion() - Method in class org.apache.beam.runners.dataflow.DataflowRunnerInfo
Provides the FnAPI environment's major version number.
getFullName(PTransform<?, ?>) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.TranslationContext
Returns the full name of the currently being translated transform.
getGcloudCancelCommand(DataflowPipelineOptions, String) - Static method in class org.apache.beam.runners.dataflow.util.MonitoringUtil
 
getId() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmShard
Return the shard id.
getIndexOffset() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmShard
Return the absolute position within the Ism file where the index block begins.
getIndexPosition() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.Footer
 
getInput(PTransform<InputT, ?>) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.TranslationContext
 
getInputs(PTransform<InputT, ?>) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.TranslationContext
Returns the input of the currently being translated transform.
getInt(Map<String, Object>, String) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
getInt(Map<String, Object>, String, Integer) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
getJob(String) - Method in class org.apache.beam.runners.dataflow.DataflowClient
Gets the Dataflow Job with the given jobId.
getJob() - Method in exception org.apache.beam.runners.dataflow.DataflowJobException
Returns the failed job.
getJob() - Method in class org.apache.beam.runners.dataflow.DataflowPipelineTranslator.JobSpecification
 
getJobId() - Method in class org.apache.beam.runners.dataflow.DataflowPipelineJob
Get the id of this job.
getJobId() - Method in interface org.apache.beam.runners.dataflow.options.DataflowWorkerHarnessOptions
The identity of the Dataflow job.
getJobId() - Method in class org.apache.beam.runners.dataflow.util.DataflowTemplateJob
 
getJobMessages(String, long) - Method in class org.apache.beam.runners.dataflow.util.MonitoringUtil
Return job messages sorted in ascending order by timestamp.
getJobMetrics(String) - Method in class org.apache.beam.runners.dataflow.DataflowClient
Gets the JobMetrics with the given jobId.
getJobMonitoringPageURL(String, String) - Static method in class org.apache.beam.runners.dataflow.util.MonitoringUtil
Deprecated.
this method defaults the region to "us-central1". Prefer using the overload with an explicit regionId parameter.
getJobMonitoringPageURL(String, String, String) - Static method in class org.apache.beam.runners.dataflow.util.MonitoringUtil
 
getKeyCoder() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.MetadataKeyCoder
 
getKeyComponent(int) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecord
Returns the key component at the specified index.
getKeyComponentCoder(int) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecordCoder
Returns the key coder at the specified index.
getKeyComponentCoders() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecordCoder
Returns the list of key component coders.
getKeyComponents() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecord
Returns the list of key components.
getLabels() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
Labels that will be applied to the billing records for this job.
getLegacyEnvironmentMajorVersion() - Method in class org.apache.beam.runners.dataflow.DataflowRunnerInfo
Provides the legacy environment's major version number.
getListOfMaps(Map<String, Object>, String, List<Map<String, Object>>) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
getLong(Map<String, Object>, String) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
getLong(Map<String, Object>, String, Long) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
getMainOutputTag() - Method in class org.apache.beam.runners.dataflow.PrimitiveParDoSingleFactory.ParDoSingle
 
getMaxConditionCost() - Method in interface org.apache.beam.runners.dataflow.options.CloudDebuggerOptions
The maximum cost (as a ratio of CPU time) allowed for evaluating conditional snapshots.
getMaxNumWorkers() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
The maximum number of workers to use for the workerpool.
getMetadata() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecord
Returns the metadata.
getMetadataKey() - Static method in class org.apache.beam.runners.dataflow.internal.IsmFormat
An object representing a wild card for a key component.
getMinCpuPlatform() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
Specifies a Minimum CPU platform for VM instances.
getNetwork() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
GCE network for launching workers.
getNumberOfKeys() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.Footer
 
getNumberOfShardKeyCoders(List<?>) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecordCoder
 
getNumberOfWorkerHarnessThreads() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
Number of threads to use on the Dataflow worker harness.
getNumWorkers() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
Number of workers to use when executing the Dataflow job.
getObject(Map<String, Object>, String) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
getObject(Map<String, Object>, String, Map<String, Object>) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
getOutput(PTransform<?, OutputT>) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.TranslationContext
 
getOutputs(PTransform<?, OutputT>) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.TranslationContext
Returns the output of the currently being translated transform.
getOutputTypeDescriptor() - Method in class org.apache.beam.runners.dataflow.BatchStatefulParDoOverrides.BatchStatefulDoFn
 
getOverrideWindmillBinary() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
Custom windmill_main binary to use with the streaming runner.
getPipelineOptions() - Method in class org.apache.beam.runners.dataflow.DataflowPipelineRegistrar.Options
 
getPipelineOptions() - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.TranslationContext
Returns the configured pipeline options.
getPipelineProto() - Method in class org.apache.beam.runners.dataflow.DataflowPipelineTranslator.JobSpecification
 
getPipelineRunners() - Method in class org.apache.beam.runners.dataflow.DataflowPipelineRegistrar.Runner
 
getPipelineUrl() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
The URL of the staged portable pipeline.
getProducer(PValue) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.TranslationContext
Get the AppliedPTransform that produced the provided PValue.
getProfilingAgentConfiguration() - Method in interface org.apache.beam.runners.dataflow.options.DataflowProfilingOptions
 
getProject() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
 
getProjectId() - Method in class org.apache.beam.runners.dataflow.DataflowPipelineJob
Get the project this job exists in.
getProjectId() - Method in class org.apache.beam.runners.dataflow.util.DataflowTemplateJob
 
getProperties() - Method in class org.apache.beam.runners.dataflow.DataflowRunnerInfo
 
getRegion() - Method in class org.apache.beam.runners.dataflow.DataflowPipelineJob
Get the region this job exists in.
getRegion() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
The Google Compute Engine region for creating Dataflow jobs.
getReplacedByJob() - Method in class org.apache.beam.runners.dataflow.DataflowPipelineJob
Returns a new DataflowPipelineJob for the job that replaced this one, if applicable.
getReplacedByJob() - Method in class org.apache.beam.runners.dataflow.util.DataflowTemplateJob
 
getReplacementTransform(AppliedPTransform<PCollection<? extends InputT>, PCollection<OutputT>, ParDo.SingleOutput<InputT, OutputT>>) - Method in class org.apache.beam.runners.dataflow.PrimitiveParDoSingleFactory
 
getSaveHeapDumpsToGcsPath() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
CAUTION: This option implies dumpHeapOnOOM, and has similar caveats.
getSaveProfilesToGcs() - Method in interface org.apache.beam.runners.dataflow.options.DataflowProfilingOptions
 
getSdkComponents() - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.TranslationContext
 
getServiceAccount() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
Run the job as a specific service account, instead of the default GCE robot.
getSharedKeySize() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.KeyPrefix
 
getSideInputs() - Method in class org.apache.beam.runners.dataflow.PrimitiveParDoSingleFactory.ParDoSingle
 
getStager() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
The resource stager instance that should be used to stage resources.
getStagerClass() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
The class responsible for staging resources to be accessible by workers during job execution.
getStagingLocation() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
GCS path for staging local files, e.g.
getState() - Method in class org.apache.beam.runners.dataflow.DataflowPipelineJob
 
getState() - Method in class org.apache.beam.runners.dataflow.util.DataflowTemplateJob
 
getStepNames() - Method in class org.apache.beam.runners.dataflow.DataflowPipelineTranslator.JobSpecification
Returns the mapping of AppliedPTransforms to the internal step name for that AppliedPTransform.
getString(Map<String, Object>, String) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
getString(Map<String, Object>, String, String) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
getStrings(Map<String, Object>, String, List<String>) - Static method in class org.apache.beam.runners.dataflow.util.Structs
 
getSubnetwork() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
GCE subnetwork for launching workers.
getSupportedClass() - Method in interface org.apache.beam.runners.dataflow.util.CloudObjectTranslator
Gets the class this CloudObjectTranslator is capable of converting.
getTemplateLocation() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
Where the runner should generate a template file.
getTransformNameMapping() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
Mapping of old PTranform names to new ones, specified as JSON {"oldName":"newName",...} .
getTransformPayloadTranslators() - Method in class org.apache.beam.runners.dataflow.PrimitiveParDoSingleFactory.Registrar
 
getTransformTranslator(Class<TransformT>) - Method in class org.apache.beam.runners.dataflow.DataflowPipelineTranslator
Returns the TransformTranslator to use for instances of the specified PTransform class, or null if none registered.
getTranslator() - Method in class org.apache.beam.runners.dataflow.DataflowRunner
Returns the DataflowPipelineTranslator associated with this object.
getUnderlyingDoFn() - Method in class org.apache.beam.runners.dataflow.BatchStatefulParDoOverrides.BatchStatefulDoFn
 
getUnsharedKeySize() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.KeyPrefix
 
getUrn(PrimitiveParDoSingleFactory.ParDoSingle<?, ?>) - Method in class org.apache.beam.runners.dataflow.PrimitiveParDoSingleFactory.PayloadTranslator
 
getUsePublicIps() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
Specifies whether worker pools should be started with public IP addresses.
getValue() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecord
Returns the value.
getValueCoder() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecordCoder
Returns the value coder.
getVersion() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.Footer
 
getView() - Method in class org.apache.beam.runners.dataflow.CreateDataflowView
 
getView() - Method in class org.apache.beam.runners.dataflow.DataflowRunner.StreamingPCollectionViewWriterFn
 
getWindmillServiceEndpoint() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
Custom windmill service endpoint.
getWindmillServicePort() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
 
getWorkerCacheMb() - Method in interface org.apache.beam.runners.dataflow.options.DataflowWorkerHarnessOptions
The size of the worker's in-memory cache, in megabytes.
getWorkerDiskType() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
Specifies what type of persistent disk is used.
getWorkerHarnessContainerImage() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
Docker container image that executes Dataflow worker harness, residing in Google Container Registry.
getWorkerId() - Method in interface org.apache.beam.runners.dataflow.options.DataflowWorkerHarnessOptions
The identity of the worker running this pipeline.
getWorkerLogLevelOverrides() - Method in interface org.apache.beam.runners.dataflow.options.DataflowWorkerLoggingOptions
Deprecated.
This option controls the log levels for specifically named loggers.
getWorkerMachineType() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
Machine type to create Dataflow worker VMs as.
getWorkerSystemErrMessageLevel() - Method in interface org.apache.beam.runners.dataflow.options.DataflowWorkerLoggingOptions
Deprecated.
Controls the log level given to messages printed to System.err.
getWorkerSystemOutMessageLevel() - Method in interface org.apache.beam.runners.dataflow.options.DataflowWorkerLoggingOptions
Deprecated.
Controls the log level given to messages printed to System.out.
getZone() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
GCE availability zone for launching workers.

H

hasExperiment(DataflowPipelineDebugOptions, String) - Static method in class org.apache.beam.runners.dataflow.DataflowRunner
Returns true if the specified experiment is enabled, handling null experiments.
hash(List<?>) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecordCoder
Computes the shard id for the given key component(s).
hashCode() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecordCoder
 
hashCode() - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData
 

I

IMPULSE_ELEMENT - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
increment() - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData
Returns a RandomAccessData that is the smallest value of same length which is strictly greater than this.
INPUTS - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
IS_MERGING_WINDOW_FN - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
IS_PAIR_LIKE - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
IS_STREAM_LIKE - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
IS_WRAPPER - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
isFnApi() - Method in interface org.apache.beam.runners.dataflow.TransformTranslator.TranslationContext
 
ISM_SHARD_INDEX_CODER - Static variable in class org.apache.beam.runners.dataflow.internal.IsmFormat
A ListCoder wrapping a IsmFormat.IsmShardCoder used to encode the shard index.
isMetadataKey(List<?>) - Static method in class org.apache.beam.runners.dataflow.internal.IsmFormat
Returns true if and only if any of the passed in key components represent a metadata key.
IsmFormat - Class in org.apache.beam.runners.dataflow.internal
An Ism file is a prefix encoded composite key value file broken into shards.
IsmFormat() - Constructor for class org.apache.beam.runners.dataflow.internal.IsmFormat
 
IsmFormat.Footer - Class in org.apache.beam.runners.dataflow.internal
The footer stores the relevant information required to locate the index and bloom filter.
IsmFormat.FooterCoder - Class in org.apache.beam.runners.dataflow.internal
A Coder for IsmFormat.Footer.
IsmFormat.IsmRecord<V> - Class in org.apache.beam.runners.dataflow.internal
A record containing a composite key and either a value or metadata.
IsmFormat.IsmRecordCoder<V> - Class in org.apache.beam.runners.dataflow.internal
A Coder for IsmFormat.IsmRecords.
IsmFormat.IsmShard - Class in org.apache.beam.runners.dataflow.internal
A shard descriptor containing shard id, the data block offset, and the index offset for the given shard.
IsmFormat.IsmShardCoder - Class in org.apache.beam.runners.dataflow.internal
A coder for IsmFormat.IsmShards.
IsmFormat.KeyPrefix - Class in org.apache.beam.runners.dataflow.internal
The prefix used before each key which contains the number of shared and unshared bytes from the previous key that was read.
IsmFormat.KeyPrefixCoder - Class in org.apache.beam.runners.dataflow.internal
A Coder for IsmFormat.KeyPrefix.
IsmFormat.MetadataKeyCoder<K> - Class in org.apache.beam.runners.dataflow.internal
A coder for metadata key component.
isRegisterByteSizeObserverCheap(IsmFormat.Footer) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.FooterCoder
 
isRegisterByteSizeObserverCheap(IsmFormat.KeyPrefix) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.KeyPrefixCoder
 
isRegisterByteSizeObserverCheap(RandomAccessData) - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData.RandomAccessDataCoder
 
isUpdate() - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
Whether to update the currently running pipeline with the same name as this one.

J

jobId - Variable in class org.apache.beam.runners.dataflow.DataflowPipelineJob
The id for the job.
JobSpecification(Job, RunnerApi.Pipeline, Map<AppliedPTransform<?, ?, ?>, String>) - Constructor for class org.apache.beam.runners.dataflow.DataflowPipelineTranslator.JobSpecification
 
jobToString(Job) - Static method in class org.apache.beam.runners.dataflow.DataflowPipelineTranslator
Renders a Job as a string.

K

KeyPrefix() - Constructor for class org.apache.beam.runners.dataflow.internal.IsmFormat.KeyPrefix
 
KeyPrefixCoder() - Constructor for class org.apache.beam.runners.dataflow.internal.IsmFormat.KeyPrefixCoder
 

L

leaseWorkItem(String, LeaseWorkItemRequest) - Method in class org.apache.beam.runners.dataflow.DataflowClient
Leases the work item for jobId.
listJobMessages(String, String) - Method in class org.apache.beam.runners.dataflow.DataflowClient
Lists job messages with the given jobId.
listJobs(String) - Method in class org.apache.beam.runners.dataflow.DataflowClient
Lists Dataflow Jobs in the project associated with the DataflowPipelineOptions.
LoggingHandler() - Constructor for class org.apache.beam.runners.dataflow.util.MonitoringUtil.LoggingHandler
 
LONG_BYTES - Static variable in class org.apache.beam.runners.dataflow.internal.IsmFormat.Footer
 

M

meta(List<?>, byte[]) - Static method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecord
 
metrics() - Method in class org.apache.beam.runners.dataflow.DataflowPipelineJob
 
modifyEnvironmentBeforeSubmission(Environment) - Method in class org.apache.beam.runners.dataflow.DataflowRunnerHooks
Allows the user to modify the environment of their job before their job is submitted to the service for execution.
MonitoringUtil - Class in org.apache.beam.runners.dataflow.util
A helper class for monitoring jobs submitted to the service.
MonitoringUtil(DataflowClient) - Constructor for class org.apache.beam.runners.dataflow.util.MonitoringUtil
Construct a helper for monitoring.
MonitoringUtil.JobMessagesHandler - Interface in org.apache.beam.runners.dataflow.util
An interface that can be used for defining callbacks to receive a list of JobMessages containing monitoring information.
MonitoringUtil.LoggingHandler - Class in org.apache.beam.runners.dataflow.util
A handler that logs monitoring messages.
MonitoringUtil.TimeStampComparator - Class in org.apache.beam.runners.dataflow.util
Comparator for sorting rows in increasing order based on timestamp.
multiOutputOverrideFactory(DataflowPipelineOptions) - Static method in class org.apache.beam.runners.dataflow.BatchStatefulParDoOverrides
Returns a PTransformOverrideFactory that replaces a multi-output ParDo with a composite transform specialized for the DataflowRunner.

N

newClouddebuggerClient(DataflowPipelineOptions) - Static method in class org.apache.beam.runners.dataflow.util.DataflowTransport
 
newDataflowClient(DataflowPipelineOptions) - Static method in class org.apache.beam.runners.dataflow.util.DataflowTransport
Returns a Google Cloud Dataflow client builder.
NON_PARALLEL_INPUTS - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 

O

OBJECT_TYPE_NAME - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
of(long, long, long) - Static method in class org.apache.beam.runners.dataflow.internal.IsmFormat.Footer
 
of() - Static method in class org.apache.beam.runners.dataflow.internal.IsmFormat.FooterCoder
 
of(List<?>, V) - Static method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecord
Returns an IsmRecord with the specified key components and value.
of(int, int, List<Coder<?>>, Coder<V>) - Static method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecordCoder
Returns an IsmRecordCoder with the specified key component coders, value coder.
of(int, long) - Static method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmShard
Returns an IsmShard with the given id, block offset and no index offset.
of(int, long, long) - Static method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmShard
Returns an IsmShard with the given id, block offset, and index offset.
of() - Static method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmShardCoder
Returns an IsmShardCoder.
of(int, int) - Static method in class org.apache.beam.runners.dataflow.internal.IsmFormat.KeyPrefix
 
of() - Static method in class org.apache.beam.runners.dataflow.internal.IsmFormat.KeyPrefixCoder
 
of(Coder<K>) - Static method in class org.apache.beam.runners.dataflow.internal.IsmFormat.MetadataKeyCoder
 
of() - Static method in class org.apache.beam.runners.dataflow.util.RandomAccessData.RandomAccessDataCoder
 
Options() - Constructor for class org.apache.beam.runners.dataflow.DataflowPipelineRegistrar.Options
 
org.apache.beam.runners.dataflow - package org.apache.beam.runners.dataflow
Provides a Beam runner that executes pipelines on the Google Cloud Dataflow service.
org.apache.beam.runners.dataflow.internal - package org.apache.beam.runners.dataflow.internal
Implementation of the DataflowRunner.
org.apache.beam.runners.dataflow.options - package org.apache.beam.runners.dataflow.options
Provides PipelineOptions specific to Google Cloud Dataflow.
org.apache.beam.runners.dataflow.util - package org.apache.beam.runners.dataflow.util
Provides miscellaneous internal utilities used by the Google Cloud Dataflow runner.
OUTPUT - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
OUTPUT_INFO - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
OUTPUT_NAME - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
OutputReference - Class in org.apache.beam.runners.dataflow.util
A representation used by Steps to reference the output of other Steps.
OutputReference(String, String) - Constructor for class org.apache.beam.runners.dataflow.util.OutputReference
 

P

PARALLEL_INPUT - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
PIPELINE_PROTO_CODER_ID - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
PrimitiveParDoSingleFactory<InputT,OutputT> - Class in org.apache.beam.runners.dataflow
A PTransformOverrideFactory that produces PrimitiveParDoSingleFactory.ParDoSingle instances from ParDo.SingleOutput instances.
PrimitiveParDoSingleFactory() - Constructor for class org.apache.beam.runners.dataflow.PrimitiveParDoSingleFactory
 
PrimitiveParDoSingleFactory.ParDoSingle<InputT,OutputT> - Class in org.apache.beam.runners.dataflow
A single-output primitive ParDo.
PrimitiveParDoSingleFactory.PayloadTranslator - Class in org.apache.beam.runners.dataflow
PrimitiveParDoSingleFactory.Registrar - Class in org.apache.beam.runners.dataflow
process(List<JobMessage>) - Method in interface org.apache.beam.runners.dataflow.util.MonitoringUtil.JobMessagesHandler
Process the rows.
process(List<JobMessage>) - Method in class org.apache.beam.runners.dataflow.util.MonitoringUtil.LoggingHandler
 
processElement(DoFn<KV<K, Iterable<KV<Instant, WindowedValue<KV<K, V>>>>>, OutputT>.ProcessContext, BoundedWindow) - Method in class org.apache.beam.runners.dataflow.BatchStatefulParDoOverrides.BatchStatefulDoFn
 
processElement(DoFn<Iterable<T>, T>.ProcessContext, BoundedWindow) - Method in class org.apache.beam.runners.dataflow.DataflowRunner.StreamingPCollectionViewWriterFn
 
PROJECT_ID_REGEXP - Static variable in class org.apache.beam.runners.dataflow.DataflowRunner
Project IDs must contain lowercase letters, digits, or dashes.
PropertyNames - Class in org.apache.beam.runners.dataflow.util
Constant property names used by the SDK in CloudWorkflow specifications.
PropertyNames() - Constructor for class org.apache.beam.runners.dataflow.util.PropertyNames
 
PUBSUB_ID_ATTRIBUTE - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
PUBSUB_SERIALIZED_ATTRIBUTES_FN - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
PUBSUB_SUBSCRIPTION - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
PUBSUB_SUBSCRIPTION_OVERRIDE - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
PUBSUB_TIMESTAMP_ATTRIBUTE - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
PUBSUB_TOPIC - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
PUBSUB_TOPIC_OVERRIDE - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 

R

RandomAccessData - Class in org.apache.beam.runners.dataflow.util
An elastic-sized byte array which allows you to manipulate it as a stream, or access it directly.
RandomAccessData() - Constructor for class org.apache.beam.runners.dataflow.util.RandomAccessData
Constructs a RandomAccessData with a default buffer size.
RandomAccessData(byte[]) - Constructor for class org.apache.beam.runners.dataflow.util.RandomAccessData
Constructs a RandomAccessData with the initial buffer.
RandomAccessData(int) - Constructor for class org.apache.beam.runners.dataflow.util.RandomAccessData
Constructs a RandomAccessData with the given buffer size.
RandomAccessData.RandomAccessDataCoder - Class in org.apache.beam.runners.dataflow.util
A Coder which encodes the valid parts of this stream.
RandomAccessData.UnsignedLexicographicalComparator - Class in org.apache.beam.runners.dataflow.util
A Comparator that compares two byte arrays lexicographically.
RandomAccessDataCoder() - Constructor for class org.apache.beam.runners.dataflow.util.RandomAccessData.RandomAccessDataCoder
 
readFrom(InputStream, int, int) - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData
Reads length bytes from the specified input stream writing them into the backing data store starting at offset.
registerTransformTranslator(Class<TransformT>, TransformTranslator<? extends TransformT>) - Static method in class org.apache.beam.runners.dataflow.DataflowPipelineTranslator
Records that instances of the specified PTransform class should be translated by default by the corresponding TransformTranslator.
Registrar() - Constructor for class org.apache.beam.runners.dataflow.PrimitiveParDoSingleFactory.Registrar
 
replaceTransforms(Pipeline) - Method in class org.apache.beam.runners.dataflow.DataflowRunner
 
reportWorkItemStatus(String, ReportWorkItemStatusRequest) - Method in class org.apache.beam.runners.dataflow.DataflowClient
Reports the status of the work item for jobId.
resetTo(int) - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData
Resets the end of the stream to the specified position.
RESTRICTION_CODER - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
run(Pipeline) - Method in class org.apache.beam.runners.dataflow.DataflowRunner
 
run(Pipeline) - Method in class org.apache.beam.runners.dataflow.TestDataflowRunner
 
Runner() - Constructor for class org.apache.beam.runners.dataflow.DataflowPipelineRegistrar.Runner
 

S

SCALAR_FIELD_NAME - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
SERIALIZED_FN - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
serializeToCloudSource(Source<?>, PipelineOptions) - Static method in class org.apache.beam.runners.dataflow.internal.CustomSources
 
setApiRootUrl(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
 
setAutoscalingAlgorithm(DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
 
setDataflowClient(Dataflow) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
 
setDataflowEndpoint(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
 
setDataflowJobFile(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
 
setDataflowWorkerJar(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
 
setDebuggee(Debuggee) - Method in interface org.apache.beam.runners.dataflow.options.CloudDebuggerOptions
 
setDefaultWorkerLogLevel(DataflowWorkerLoggingOptions.Level) - Method in interface org.apache.beam.runners.dataflow.options.DataflowWorkerLoggingOptions
Deprecated.
 
setDiskSizeGb(int) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
 
setDumpHeapOnOOM(boolean) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
 
setEnableCloudDebugger(boolean) - Method in interface org.apache.beam.runners.dataflow.options.CloudDebuggerOptions
 
setFilesToStage(List<String>) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
 
setHooks(DataflowRunnerHooks) - Method in class org.apache.beam.runners.dataflow.DataflowRunner
Sets callbacks to invoke during execution see DataflowRunnerHooks.
setJobId(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowWorkerHarnessOptions
 
setLabels(Map<String, String>) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
 
setMaxConditionCost(double) - Method in interface org.apache.beam.runners.dataflow.options.CloudDebuggerOptions
 
setMaxNumWorkers(int) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
 
setMinCpuPlatform(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
 
setNetwork(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
 
setNumberOfWorkerHarnessThreads(int) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
 
setNumWorkers(int) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
 
setOverrideWindmillBinary(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
 
setPipelineUrl(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
 
setProfilingAgentConfiguration(DataflowProfilingOptions.DataflowProfilingAgentConfiguration) - Method in interface org.apache.beam.runners.dataflow.options.DataflowProfilingOptions
 
setProject(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
 
setRegion(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
 
setSaveHeapDumpsToGcsPath(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
 
setSaveProfilesToGcs(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowProfilingOptions
 
setServiceAccount(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
 
setStager(Stager) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
 
setStagerClass(Class<? extends Stager>) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
 
setStagingLocation(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
 
setSubnetwork(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
 
setTemplateLocation(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
 
setTransformNameMapping(Map<String, String>) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
 
setup() - Method in class org.apache.beam.runners.dataflow.BatchStatefulParDoOverrides.BatchStatefulDoFn
 
setUpdate(boolean) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineOptions
 
setUsePublicIps(Boolean) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
 
setWindmillServiceEndpoint(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
 
setWindmillServicePort(int) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions
 
setWorkerCacheMb(Integer) - Method in interface org.apache.beam.runners.dataflow.options.DataflowWorkerHarnessOptions
 
setWorkerDiskType(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
 
setWorkerHarnessContainerImage(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
 
setWorkerId(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowWorkerHarnessOptions
 
setWorkerLogLevelOverrides(DataflowWorkerLoggingOptions.WorkerLogLevelOverrides) - Method in interface org.apache.beam.runners.dataflow.options.DataflowWorkerLoggingOptions
Deprecated.
 
setWorkerMachineType(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
 
setWorkerSystemErrMessageLevel(DataflowWorkerLoggingOptions.Level) - Method in interface org.apache.beam.runners.dataflow.options.DataflowWorkerLoggingOptions
Deprecated.
 
setWorkerSystemOutMessageLevel(DataflowWorkerLoggingOptions.Level) - Method in interface org.apache.beam.runners.dataflow.options.DataflowWorkerLoggingOptions
Deprecated.
 
setZone(String) - Method in interface org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions
 
SHARD_BITS - Static variable in class org.apache.beam.runners.dataflow.internal.IsmFormat
 
singleOutputOverrideFactory(DataflowPipelineOptions) - Static method in class org.apache.beam.runners.dataflow.BatchStatefulParDoOverrides
Returns a PTransformOverrideFactory that replaces a single-output ParDo with a composite transform specialized for the DataflowRunner.
size() - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData
Returns the number of bytes in the backing array that are valid.
SORT_VALUES - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
SOURCE_DOES_NOT_NEED_SPLITTING - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
SOURCE_ESTIMATED_SIZE_BYTES - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
SOURCE_IS_INFINITE - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
SOURCE_METADATA - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
SOURCE_SPEC - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
SOURCE_STEP_INPUT - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
stageDefaultFiles() - Method in class org.apache.beam.runners.dataflow.util.GcsStager
Stages DataflowPipelineWorkerPoolOptions.getFilesToStage(), which defaults to every file on the classpath unless overridden, as well as DataflowPipelineDebugOptions.getOverrideWindmillBinary() if specified.
stageDefaultFiles() - Method in interface org.apache.beam.runners.dataflow.util.Stager
Stage default files and return a list of DataflowPackage objects describing the actual location at which each file was staged.
stageFiles(List<String>) - Method in class org.apache.beam.runners.dataflow.util.GcsStager
Stages files to DataflowPipelineOptions.getStagingLocation(), suffixed with their md5 hash to avoid collisions.
stageFiles(List<String>) - Method in interface org.apache.beam.runners.dataflow.util.Stager
Stage files and return a list of packages DataflowPackage objects describing th actual location at which each file was staged.
Stager - Interface in org.apache.beam.runners.dataflow.util
Interface for staging files needed for running a Dataflow pipeline.
StagerFactory() - Constructor for class org.apache.beam.runners.dataflow.options.DataflowPipelineDebugOptions.StagerFactory
 
stageToFile(byte[], String) - Method in class org.apache.beam.runners.dataflow.util.GcsStager
 
stageToFile(byte[], String) - Method in interface org.apache.beam.runners.dataflow.util.Stager
Stage bytes to a target file name wherever this stager stages things.
StagingLocationFactory() - Constructor for class org.apache.beam.runners.dataflow.options.DataflowPipelineOptions.StagingLocationFactory
 
STATUS_BACKOFF_FACTORY - Static variable in class org.apache.beam.runners.dataflow.DataflowPipelineJob
 
Structs - Class in org.apache.beam.runners.dataflow.util
A collection of static methods for manipulating datastructure representations transferred via the Dataflow API.
structuralValue(IsmFormat.IsmRecord<V>) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecordCoder
 

T

teardown() - Method in class org.apache.beam.runners.dataflow.BatchStatefulParDoOverrides.BatchStatefulDoFn
 
TestDataflowPipelineOptions - Interface in org.apache.beam.runners.dataflow
A set of options used to configure the TestPipeline.
TestDataflowRunner - Class in org.apache.beam.runners.dataflow
TestDataflowRunner is a pipeline runner that wraps a DataflowRunner when running tests against the TestPipeline.
TimeStampComparator() - Constructor for class org.apache.beam.runners.dataflow.util.MonitoringUtil.TimeStampComparator
 
TimeUtil - Class in org.apache.beam.runners.dataflow.util
A helper class for converting between Dataflow API and SDK time representations.
toCloudDuration(ReadableDuration) - Static method in class org.apache.beam.runners.dataflow.util.TimeUtil
Converts a ReadableDuration into a Dataflow API duration string.
toCloudObject(T, SdkComponents) - Method in interface org.apache.beam.runners.dataflow.util.CloudObjectTranslator
Converts the provided object into an equivalent CloudObject.
toCloudTime(ReadableInstant) - Static method in class org.apache.beam.runners.dataflow.util.TimeUtil
Converts a ReadableInstant into a Dateflow API time value.
toState(String) - Static method in class org.apache.beam.runners.dataflow.util.MonitoringUtil
 
toString() - Method in class org.apache.beam.runners.dataflow.DataflowPipelineTranslator
 
toString() - Method in class org.apache.beam.runners.dataflow.DataflowRunner
 
toString() - Method in class org.apache.beam.runners.dataflow.TestDataflowRunner
 
toString() - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData
 
transformStepNames - Variable in class org.apache.beam.runners.dataflow.DataflowPipelineJob
 
TransformTranslator<TransformT extends org.apache.beam.sdk.transforms.PTransform> - Interface in org.apache.beam.runners.dataflow
A TransformTranslator knows how to translate a particular subclass of PTransform for the Cloud Dataflow service.
TransformTranslator.StepTranslationContext - Interface in org.apache.beam.runners.dataflow
The interface for a TransformTranslator to build a Dataflow step.
TransformTranslator.TranslationContext - Interface in org.apache.beam.runners.dataflow
The interface provided to registered callbacks for interacting with the DataflowRunner, including reading and writing the values of PCollections and side inputs.
translate(Pipeline, DataflowRunner, List<DataflowPackage>) - Method in class org.apache.beam.runners.dataflow.DataflowPipelineTranslator
Translates a Pipeline into a JobSpecification.
translate(AppliedPTransform<?, ?, PrimitiveParDoSingleFactory.ParDoSingle<?, ?>>, SdkComponents) - Method in class org.apache.beam.runners.dataflow.PrimitiveParDoSingleFactory.PayloadTranslator
 
translate(TransformT, TransformTranslator.TranslationContext) - Method in interface org.apache.beam.runners.dataflow.TransformTranslator
 
TUPLE_TAGS - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
type - Variable in class org.apache.beam.runners.dataflow.util.OutputReference
 

U

UNSIGNED_LEXICOGRAPHICAL_COMPARATOR - Static variable in class org.apache.beam.runners.dataflow.util.RandomAccessData
 
updateJob(String, Job) - Method in class org.apache.beam.runners.dataflow.DataflowClient
Updates the Dataflow Job with the given jobId.
USE_INDEXED_FORMAT - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
USER_FN - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
USER_NAME - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
USES_KEYED_STATE - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 

V

validateCoderIsCompatible(IsmFormat.IsmRecordCoder<?>) - Static method in class org.apache.beam.runners.dataflow.internal.IsmFormat
Validates that the key portion of the given coder is deterministic.
VALUE - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
valueOf(String) - Static method in enum org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType
Returns the enum constant of this type with the specified name.
valueOf(String) - Static method in enum org.apache.beam.runners.dataflow.options.DataflowWorkerLoggingOptions.Level
Deprecated.
Returns the enum constant of this type with the specified name.
values() - Static method in enum org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions.AutoscalingAlgorithmType
Returns an array containing the constants of this enum type, in the order they are declared.
values() - Static method in enum org.apache.beam.runners.dataflow.options.DataflowWorkerLoggingOptions.Level
Deprecated.
Returns an array containing the constants of this enum type, in the order they are declared.
verifyDeterministic() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.FooterCoder
 
verifyDeterministic() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmRecordCoder
 
verifyDeterministic() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmShardCoder
 
verifyDeterministic() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.KeyPrefixCoder
 
verifyDeterministic() - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.MetadataKeyCoder
 
verifyDeterministic() - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData.RandomAccessDataCoder
 
VERSION - Static variable in class org.apache.beam.runners.dataflow.internal.IsmFormat.Footer
 

W

waitUntilFinish() - Method in class org.apache.beam.runners.dataflow.DataflowPipelineJob
 
waitUntilFinish(Duration) - Method in class org.apache.beam.runners.dataflow.DataflowPipelineJob
 
waitUntilFinish(Duration, MonitoringUtil.JobMessagesHandler) - Method in class org.apache.beam.runners.dataflow.DataflowPipelineJob
Waits until the pipeline finishes and returns the final status.
WINDOWING_STRATEGY - Static variable in class org.apache.beam.runners.dataflow.util.PropertyNames
 
withIndexOffset(long) - Method in class org.apache.beam.runners.dataflow.internal.IsmFormat.IsmShard
Returns a new IsmShard like this one with the specified index offset.
WorkerHarnessContainerImageFactory() - Constructor for class org.apache.beam.runners.dataflow.options.DataflowPipelineWorkerPoolOptions.WorkerHarnessContainerImageFactory
 
WorkerLogLevelOverrides() - Constructor for class org.apache.beam.runners.dataflow.options.DataflowWorkerLoggingOptions.WorkerLogLevelOverrides
Deprecated.
 
writeTo(OutputStream, int, int) - Method in class org.apache.beam.runners.dataflow.util.RandomAccessData
Writes length bytes starting at offset from the backing data store to the specified output stream.
A B C D E F G H I J K L M N O P R S T U V W 
Skip navigation links