org.apache.beam.sdk.io.gcp.bigquery

Class StreamingWriteTables<ElementT>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<TableDestination,ElementT>>,WriteResult>

org.apache.beam.sdk.io.gcp.bigquery.StreamingWriteTables<ElementT>

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

public class StreamingWriteTables<ElementT>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<TableDestination,ElementT>>,WriteResult>

This transform takes in key-value pairs of TableRow entries and the TableDestination it should be written to. The BigQuery streaming-write service is used to stream these writes to the appropriate table.

This transform assumes that all destination tables already exist by the time it sees a write for that table.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name, resourceHints

Constructor Summary

Constructors

Constructor and Description
StreamingWriteTables()

Method Summary

Modifier and Type	Method and Description
WriteResult	expand(org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<TableDestination,ElementT>> input)
<TupleTagT> org.apache.beam.sdk.values.TupleTag<TupleTagT>	getFailedRowsTupleTag()

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

StreamingWriteTables

public StreamingWriteTables()

Method Detail

getFailedRowsTupleTag

public <TupleTagT> org.apache.beam.sdk.values.TupleTag<TupleTagT> getFailedRowsTupleTag()

expand

public WriteResult expand(org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<TableDestination,ElementT>> input)

Specified by:

expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<org.apache.beam.sdk.values.KV<TableDestination,ElementT>>,WriteResult>