org.apache.beam.sdk.io.gcp.bigquery

Class BigQueryStorageStreamSource<T>

java.lang.Object

org.apache.beam.sdk.io.Source<T>

org.apache.beam.sdk.io.BoundedSource<T>

org.apache.beam.sdk.io.gcp.bigquery.BigQueryStorageStreamSource<T>

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

@Experimental(value=SOURCE_SINK)
public class BigQueryStorageStreamSource<T>
extends org.apache.beam.sdk.io.BoundedSource<T>

A Source representing a single stream in a read session.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	BigQueryStorageStreamSource.BigQueryStorageStreamReader<T> A Source.Reader which reads records from a stream.

Nested classes/interfaces inherited from class org.apache.beam.sdk.io.BoundedSource

org.apache.beam.sdk.io.BoundedSource.BoundedReader<T>

Nested classes/interfaces inherited from class org.apache.beam.sdk.io.Source

org.apache.beam.sdk.io.Source.Reader<T>

Method Summary

Modifier and Type	Method and Description
static <T> BigQueryStorageStreamSource<T>	create(com.google.cloud.bigquery.storage.v1beta1.Storage.ReadSession readSession, com.google.cloud.bigquery.storage.v1beta1.Storage.Stream stream, com.google.api.services.bigquery.model.TableSchema tableSchema, org.apache.beam.sdk.transforms.SerializableFunction<SchemaAndRecord,T> parseFn, org.apache.beam.sdk.coders.Coder<T> outputCoder, BigQueryServices bqServices)
BigQueryStorageStreamSource.BigQueryStorageStreamReader<T>	createReader(org.apache.beam.sdk.options.PipelineOptions options)
BigQueryStorageStreamSource<T>	fromExisting(com.google.cloud.bigquery.storage.v1beta1.Storage.Stream newStream) Creates a new source with the same properties as this one, except with a different Storage.Stream.
long	getEstimatedSizeBytes(org.apache.beam.sdk.options.PipelineOptions options)
org.apache.beam.sdk.coders.Coder<T>	getOutputCoder()
void	populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
java.util.List<? extends org.apache.beam.sdk.io.BoundedSource<T>>	split(long desiredBundleSizeBytes, org.apache.beam.sdk.options.PipelineOptions options)
java.lang.String	toString()

Methods inherited from class org.apache.beam.sdk.io.Source

getDefaultOutputCoder, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

create

public static <T> BigQueryStorageStreamSource<T> create(com.google.cloud.bigquery.storage.v1beta1.Storage.ReadSession readSession,
                                                        com.google.cloud.bigquery.storage.v1beta1.Storage.Stream stream,
                                                        com.google.api.services.bigquery.model.TableSchema tableSchema,
                                                        org.apache.beam.sdk.transforms.SerializableFunction<SchemaAndRecord,T> parseFn,
                                                        org.apache.beam.sdk.coders.Coder<T> outputCoder,
                                                        BigQueryServices bqServices)

fromExisting

public BigQueryStorageStreamSource<T> fromExisting(com.google.cloud.bigquery.storage.v1beta1.Storage.Stream newStream)

Creates a new source with the same properties as this one, except with a different Storage.Stream.

getOutputCoder

public org.apache.beam.sdk.coders.Coder<T> getOutputCoder()

Overrides:

getOutputCoder in class org.apache.beam.sdk.io.Source<T>

populateDisplayData

public void populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)

Specified by:

populateDisplayData in interface org.apache.beam.sdk.transforms.display.HasDisplayData

Overrides:

populateDisplayData in class org.apache.beam.sdk.io.Source<T>

getEstimatedSizeBytes

public long getEstimatedSizeBytes(org.apache.beam.sdk.options.PipelineOptions options)

Specified by:

getEstimatedSizeBytes in class org.apache.beam.sdk.io.BoundedSource<T>

split

public java.util.List<? extends org.apache.beam.sdk.io.BoundedSource<T>> split(long desiredBundleSizeBytes,
                                                                               org.apache.beam.sdk.options.PipelineOptions options)

Specified by:

split in class org.apache.beam.sdk.io.BoundedSource<T>

createReader

public BigQueryStorageStreamSource.BigQueryStorageStreamReader<T> createReader(org.apache.beam.sdk.options.PipelineOptions options)
                                                                        throws java.io.IOException

Specified by:

createReader in class org.apache.beam.sdk.io.BoundedSource<T>

Throws:

java.io.IOException

toString

public java.lang.String toString()

Overrides:

toString in class java.lang.Object