com.google.cloud.bigtable.beam

Class CloudBigtableIO.CloudBigtableMultiTableWriteFn

java.lang.Object

org.apache.beam.sdk.transforms.DoFn<In,Out>

com.google.cloud.bigtable.beam.AbstractCloudBigtableTableDoFn<InputType,Void>

com.google.cloud.bigtable.beam.CloudBigtableIO.CloudBigtableMultiTableWriteFn

All Implemented Interfaces:

Serializable, HasDisplayData

Enclosing class:

CloudBigtableIO

@InternalExtensionOnly
public static class CloudBigtableIO.CloudBigtableMultiTableWriteFn
extends AbstractCloudBigtableTableDoFn<InputType,Void>

A DoFn that can write either a bounded or unbounded PCollection of KV of (String tableName, List of Mutations) to the specified table.

NOTE: This DoFn will write Puts and Deletes, not Appends and Increments. This limitation exists because if the batch fails partway through, Appends/Increments might be re-run, causing the Mutation to be executed twice, which is never the user's intent. Re-running a Delete will not cause any differences. Re-running a Put isn't normally a problem, but might cause problems in some cases when the number of versions supported by the column family is greater than one. In a case where multiple versions could be a problem, it's best to add a timestamp to the Put.

See Also:

Serialized Form

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.beam.sdk.transforms.DoFn

DoFn.AlwaysFetched, DoFn.BoundedPerElement, DoFn.BundleFinalizer, DoFn.Element, DoFn.FieldAccess, DoFn.FinishBundle, DoFn.FinishBundleContext, DoFn.GetInitialRestriction, DoFn.GetInitialWatermarkEstimatorState, DoFn.GetRestrictionCoder, DoFn.GetSize, DoFn.GetWatermarkEstimatorStateCoder, DoFn.Key, DoFn.MultiOutputReceiver, DoFn.NewTracker, DoFn.NewWatermarkEstimator, DoFn.OnTimer, DoFn.OnTimerContext, DoFn.OnTimerFamily, DoFn.OnWindowExpiration, DoFn.OnWindowExpirationContext, DoFn.OutputReceiver<T>, DoFn.ProcessContext, DoFn.ProcessContinuation, DoFn.ProcessElement, DoFn.RequiresStableInput, DoFn.RequiresTimeSortedInput, DoFn.Restriction, DoFn.Setup, DoFn.SideInput, DoFn.SplitRestriction, DoFn.StartBundle, DoFn.StartBundleContext, DoFn.StateId, DoFn.Teardown, DoFn.TimerFamily, DoFn.TimerId, DoFn.Timestamp, DoFn.TruncateRestriction, DoFn.UnboundedPerElement, DoFn.WatermarkEstimatorState, DoFn.WindowedContext

Field Summary

Fields inherited from class com.google.cloud.bigtable.beam.AbstractCloudBigtableTableDoFn

config, connection, DOFN_LOG

Constructor Summary

Constructors

Constructor and Description
CloudBigtableMultiTableWriteFn(CloudBigtableConfiguration config)

Method Summary

Modifier and Type	Method and Description
void	finishBundle(DoFn.FinishBundleContext c)
void	processElement(DoFn.ProcessContext context) Uses the connection to create a new Table to write the Mutations to.
void	startBundle()

Methods inherited from class com.google.cloud.bigtable.beam.AbstractCloudBigtableTableDoFn

getConfig, getConnection, logExceptions, logRetriesExhaustedWithDetailsException, populateDisplayData, rethrowException

Methods inherited from class org.apache.beam.sdk.transforms.DoFn

getAllowedTimestampSkew, getInputTypeDescriptor, getOutputTypeDescriptor, prepareForProcessing

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CloudBigtableMultiTableWriteFn

public CloudBigtableMultiTableWriteFn(CloudBigtableConfiguration config)

Method Detail

startBundle

@DoFn.StartBundle
public void startBundle()
                                   throws Exception

Throws:

Exception

processElement

@DoFn.ProcessElement
public void processElement(DoFn.ProcessContext context)
                                         throws Exception

Uses the connection to create a new Table to write the Mutations to.

NOTE: This method does not create a new table in Cloud Bigtable. The table must already exist.

Parameters:

context - The context for the DoFn.

Throws:

Exception

finishBundle

@DoFn.FinishBundle
public void finishBundle(DoFn.FinishBundleContext c)
                                     throws Exception

Throws:

Exception