org.apache.beam.sdk.io.gcp.bigquery

Class BigQueryIO.Read

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,org.apache.beam.sdk.values.PCollection<com.google.api.services.bigquery.model.TableRow>>

org.apache.beam.sdk.io.gcp.bigquery.BigQueryIO.Read

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

Enclosing class:

BigQueryIO

public static class BigQueryIO.Read
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,org.apache.beam.sdk.values.PCollection<com.google.api.services.bigquery.model.TableRow>>

Implementation of BigQueryIO.read().

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

name, resourceHints

Method Summary

Modifier and Type	Method and Description
org.apache.beam.sdk.values.PCollection<com.google.api.services.bigquery.model.TableRow>	expand(org.apache.beam.sdk.values.PBegin input)
BigQueryIO.Read	from(java.lang.String tableSpec) Reads a BigQuery table specified as "[project_id]:[dataset_id].[table_id]" or "[dataset_id].[table_id]" for tables within the current project.
BigQueryIO.Read	from(com.google.api.services.bigquery.model.TableReference table) Read from table specified by a TableReference.
BigQueryIO.Read	from(org.apache.beam.sdk.options.ValueProvider<java.lang.String> tableSpec) Same as from(String), but with a ValueProvider.
BigQueryIO.Read	fromQuery(java.lang.String query) Reads results received after executing the given query.
BigQueryIO.Read	fromQuery(org.apache.beam.sdk.options.ValueProvider<java.lang.String> query) Same as fromQuery(String), but with a ValueProvider.
@Nullable com.google.api.services.bigquery.model.TableReference	getTable() Returns the table to read, or null if reading from a query instead.
@Nullable org.apache.beam.sdk.options.ValueProvider<com.google.api.services.bigquery.model.TableReference>	getTableProvider() Returns the table to read, or null if reading from a query instead.
void	populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)
BigQueryIO.Read	usingStandardSql() Enables BigQuery's Standard SQL dialect when reading from a query.
BigQueryIO.Read	withoutResultFlattening() Disable flattening of query results.
BigQueryIO.Read	withoutValidation() Disable validation that the table exists or the query succeeds prior to pipeline submission.
BigQueryIO.Read	withTemplateCompatibility() Use new template-compatible source implementation.
BigQueryIO.Read	withTestServices(BigQueryServices testServices)

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

compose, compose, getAdditionalInputs, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Method Detail

expand

public org.apache.beam.sdk.values.PCollection<com.google.api.services.bigquery.model.TableRow> expand(org.apache.beam.sdk.values.PBegin input)

Specified by:

expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,org.apache.beam.sdk.values.PCollection<com.google.api.services.bigquery.model.TableRow>>

populateDisplayData

public void populateDisplayData(org.apache.beam.sdk.transforms.display.DisplayData.Builder builder)

Specified by:

populateDisplayData in interface org.apache.beam.sdk.transforms.display.HasDisplayData

Overrides:

populateDisplayData in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PBegin,org.apache.beam.sdk.values.PCollection<com.google.api.services.bigquery.model.TableRow>>

withTestServices

public BigQueryIO.Read withTestServices(BigQueryServices testServices)

getTableProvider

public @Nullable org.apache.beam.sdk.options.ValueProvider<com.google.api.services.bigquery.model.TableReference> getTableProvider()

Returns the table to read, or null if reading from a query instead.

getTable

public @Nullable com.google.api.services.bigquery.model.TableReference getTable()

Returns the table to read, or null if reading from a query instead.

from

public BigQueryIO.Read from(java.lang.String tableSpec)

Reads a BigQuery table specified as "[project_id]:[dataset_id].[table_id]" or "[dataset_id].[table_id]" for tables within the current project.

from

public BigQueryIO.Read from(org.apache.beam.sdk.options.ValueProvider<java.lang.String> tableSpec)

Same as from(String), but with a ValueProvider.

from

public BigQueryIO.Read from(com.google.api.services.bigquery.model.TableReference table)

Read from table specified by a TableReference.

fromQuery

public BigQueryIO.Read fromQuery(java.lang.String query)

Reads results received after executing the given query.

By default, the query results will be flattened -- see "flattenResults" in the Jobs documentation for more information. To disable flattening, use withoutResultFlattening().

By default, the query will use BigQuery's legacy SQL dialect. To use the BigQuery Standard SQL dialect, use usingStandardSql().

fromQuery

public BigQueryIO.Read fromQuery(org.apache.beam.sdk.options.ValueProvider<java.lang.String> query)

Same as fromQuery(String), but with a ValueProvider.

withoutValidation

public BigQueryIO.Read withoutValidation()

Disable validation that the table exists or the query succeeds prior to pipeline submission. Basic validation (such as ensuring that a query or table is specified) still occurs.

withoutResultFlattening

public BigQueryIO.Read withoutResultFlattening()

Disable flattening of query results.

Only valid when a query is used (fromQuery(java.lang.String)). Setting this option when reading from a table will cause an error during validation.

usingStandardSql

public BigQueryIO.Read usingStandardSql()

Enables BigQuery's Standard SQL dialect when reading from a query.

Only valid when a query is used (fromQuery(java.lang.String)). Setting this option when reading from a table will cause an error during validation.

withTemplateCompatibility

public BigQueryIO.Read withTemplateCompatibility()

Use new template-compatible source implementation.

Use new template-compatible source implementation. This implementation is compatible with repeated template invocations. It does not support dynamic work rebalancing.