Package org.apache.beam.sdk.io.jdbc

Class JdbcIO.WriteWithResults<T,V extends JdbcWriteResult>

java.lang.Object

org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,org.apache.beam.sdk.values.PCollection<V>>

org.apache.beam.sdk.io.jdbc.JdbcIO.WriteWithResults<T,V>

All Implemented Interfaces:

java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData

Enclosing class:

JdbcIO

public abstract static class JdbcIO.WriteWithResults<T,V extends JdbcWriteResult>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,org.apache.beam.sdk.values.PCollection<V>>

A PTransform to write to a JDBC datasource. Executes statements one by one.

The INSERT, UPDATE, and DELETE commands sometimes have an optional RETURNING clause that supports obtaining data from modified rows while they are being manipulated. Output PCollection of this transform is a collection of such returning results mapped by JdbcIO.RowMapper.

See Also:

Serialized Form

Field Summary

Fields inherited from class org.apache.beam.sdk.transforms.PTransform

annotations, displayData, name, resourceHints

Constructor Summary

Constructors

Constructor	Description
WriteWithResults()

Method Summary

Modifier and Type	Method	Description
org.apache.beam.sdk.values.PCollection<V>	expand(org.apache.beam.sdk.values.PCollection<T> input)
JdbcIO.WriteWithResults<T,V>	withAutoSharding()	If true, enables using a dynamically determined number of shards to write.
JdbcIO.WriteWithResults<T,V>	withDataSourceConfiguration(JdbcIO.DataSourceConfiguration config)
JdbcIO.WriteWithResults<T,V>	withDataSourceProviderFn(org.apache.beam.sdk.transforms.SerializableFunction<java.lang.Void,javax.sql.DataSource> dataSourceProviderFn)
JdbcIO.WriteWithResults<T,V>	withPreparedStatementSetter(JdbcIO.PreparedStatementSetter<T> setter)
JdbcIO.WriteWithResults<T,V>	withRetryConfiguration(JdbcIO.RetryConfiguration retryConfiguration)	When a SQL exception occurs, JdbcIO.Write uses this JdbcIO.RetryConfiguration to exponentially back off and retry the statements based on the JdbcIO.RetryConfiguration mentioned.
JdbcIO.WriteWithResults<T,V>	withRetryStrategy(JdbcIO.RetryStrategy retryStrategy)	When a SQL exception occurs, JdbcIO.Write uses this JdbcIO.RetryStrategy to determine if it will retry the statements.
JdbcIO.WriteWithResults<T,V>	withRowMapper(JdbcIO.RowMapper<V> rowMapper)
JdbcIO.WriteWithResults<T,V>	withStatement(java.lang.String statement)
JdbcIO.WriteWithResults<T,V>	withStatement(org.apache.beam.sdk.options.ValueProvider<java.lang.String> statement)
JdbcIO.WriteWithResults<T,V>	withTable(java.lang.String table)

Methods inherited from class org.apache.beam.sdk.transforms.PTransform

addAnnotation, compose, compose, getAdditionalInputs, getAnnotations, getDefaultOutputCoder, getDefaultOutputCoder, getDefaultOutputCoder, getKindString, getName, getResourceHints, populateDisplayData, setDisplayData, setResourceHints, toString, validate, validate

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

WriteWithResults

public WriteWithResults()

Method Detail

withDataSourceConfiguration

public JdbcIO.WriteWithResults<T,V> withDataSourceConfiguration(JdbcIO.DataSourceConfiguration config)

withDataSourceProviderFn

public JdbcIO.WriteWithResults<T,V> withDataSourceProviderFn(org.apache.beam.sdk.transforms.SerializableFunction<java.lang.Void,javax.sql.DataSource> dataSourceProviderFn)

withStatement

public JdbcIO.WriteWithResults<T,V> withStatement(java.lang.String statement)

withStatement

public JdbcIO.WriteWithResults<T,V> withStatement(org.apache.beam.sdk.options.ValueProvider<java.lang.String> statement)

withPreparedStatementSetter

public JdbcIO.WriteWithResults<T,V> withPreparedStatementSetter(JdbcIO.PreparedStatementSetter<T> setter)

withAutoSharding

public JdbcIO.WriteWithResults<T,V> withAutoSharding()

If true, enables using a dynamically determined number of shards to write.

withRetryStrategy

public JdbcIO.WriteWithResults<T,V> withRetryStrategy(JdbcIO.RetryStrategy retryStrategy)

When a SQL exception occurs, JdbcIO.Write uses this JdbcIO.RetryStrategy to determine if it will retry the statements. If JdbcIO.RetryStrategy.apply(SQLException) returns true, then JdbcIO.Write retries the statements.

withRetryConfiguration

public JdbcIO.WriteWithResults<T,V> withRetryConfiguration(JdbcIO.RetryConfiguration retryConfiguration)

When a SQL exception occurs, JdbcIO.Write uses this JdbcIO.RetryConfiguration to exponentially back off and retry the statements based on the JdbcIO.RetryConfiguration mentioned.

Usage of RetryConfiguration -

 pipeline.apply(JdbcIO.<T>write())
    .withReturningResults(...)
    .withDataSourceConfiguration(...)
    .withRetryStrategy(...)
    .withRetryConfiguration(JdbcIO.RetryConfiguration.
        create(5, Duration.standardSeconds(5), Duration.standardSeconds(1))

 

maxDuration and initialDuration are Nullable

 pipeline.apply(JdbcIO.<T>write())
    .withReturningResults(...)
    .withDataSourceConfiguration(...)
    .withRetryStrategy(...)
    .withRetryConfiguration(JdbcIO.RetryConfiguration.
        create(5, null, null)

 

withTable

public JdbcIO.WriteWithResults<T,V> withTable(java.lang.String table)

withRowMapper

public JdbcIO.WriteWithResults<T,V> withRowMapper(JdbcIO.RowMapper<V> rowMapper)

expand

public org.apache.beam.sdk.values.PCollection<V> expand(org.apache.beam.sdk.values.PCollection<T> input)

Specified by:

expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,org.apache.beam.sdk.values.PCollection<V extends JdbcWriteResult>>