org.apache.beam.sdk.io.gcp.bigquery

Class StreamingInserts<DestinationT>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<DestinationT,com.google.api.services.bigquery.model.TableRow>>,WriteResult>

org.apache.beam.sdk.io.gcp.bigquery.StreamingInserts<DestinationT>

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

public class StreamingInserts<DestinationT>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<DestinationT,com.google.api.services.bigquery.model.TableRow>>,WriteResult>

PTransform that performs streaming BigQuery write. To increase consistency, it leverages BigQuery's best effort de-dup mechanism.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name

Constructor Summary

Constructors

Constructor and Description
StreamingInserts(BigQueryIO.Write.CreateDisposition createDisposition, DynamicDestinations<?,DestinationT> dynamicDestinations) Constructor.

Method Summary

Modifier and Type	Method and Description
WriteResult	expand(org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<DestinationT,com.google.api.services.bigquery.model.TableRow>> input)
StreamingInserts<DestinationT>	withExtendedErrorInfo(boolean extendedErrorInfo) Specify whether to use extended error info or not.
StreamingInserts<DestinationT>	withInsertRetryPolicy(InsertRetryPolicy retryPolicy) Specify a retry policy for failed inserts.

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, populateDisplayData, toString, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

StreamingInserts

public StreamingInserts(BigQueryIO.Write.CreateDisposition createDisposition,
                        DynamicDestinations<?,DestinationT> dynamicDestinations)

Constructor.

Method Detail

withInsertRetryPolicy

public StreamingInserts<DestinationT> withInsertRetryPolicy(InsertRetryPolicy retryPolicy)

Specify a retry policy for failed inserts.

withExtendedErrorInfo

public StreamingInserts<DestinationT> withExtendedErrorInfo(boolean extendedErrorInfo)

Specify whether to use extended error info or not.

expand

public WriteResult expand(org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<DestinationT,com.google.api.services.bigquery.model.TableRow>> input)

Specified by:

expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<DestinationT,com.google.api.services.bigquery.model.TableRow>>,WriteResult>