Package org.apache.beam.sdk.io.jdbc

Class JdbcIO.WriteVoid<T>

  • All Implemented Interfaces:
    java.io.Serializable, org.apache.beam.sdk.transforms.display.HasDisplayData
    Enclosing class:
    JdbcIO
    public abstract static class JdbcIO.WriteVoid<T>
extends org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,​org.apache.beam.sdk.values.PCollection<java.lang.Void>>
    A PTransform to write to a JDBC datasource. Executes statements in a batch, and returns a trivial result.
    See Also:
    Serialized Form

    • Constructor Detail

      • WriteVoid

        public WriteVoid()

    • Method Detail

      • withAutoSharding

        public JdbcIO.WriteVoid<T> withAutoSharding()
        If true, enables using a dynamically determined number of shards to write.

      • withDataSourceProviderFn

        public JdbcIO.WriteVoid<T> withDataSourceProviderFn​(org.apache.beam.sdk.transforms.SerializableFunction<java.lang.Void,​javax.sql.DataSource> dataSourceProviderFn)

      • withStatement

        public JdbcIO.WriteVoid<T> withStatement​(java.lang.String statement)

      • withStatement

        public JdbcIO.WriteVoid<T> withStatement​(org.apache.beam.sdk.options.ValueProvider<java.lang.String> statement)

      • withBatchSize

        public JdbcIO.WriteVoid<T> withBatchSize​(long batchSize)
        Provide a maximum size in number of SQL statement for the batch. Default is 1000. The pipeline will either commit a batch when this maximum is reached or its maximum buffering time has been reached. See withMaxBatchBufferingDuration(long)
        Parameters:
        batchSize - maximum batch size in number of statements

      • withMaxBatchBufferingDuration

        public JdbcIO.WriteVoid<T> withMaxBatchBufferingDuration​(long maxBatchBufferingDuration)
        Provide maximum buffering time to batch elements before committing SQL statement. Default is 200 The pipeline will either commit a batch when this maximum buffering time has been reached or the maximum amount of elements has been collected. See withBatchSize(long)
        Parameters:
        maxBatchBufferingDuration - maximum time in milliseconds before batch is committed

      • withRetryConfiguration

        public JdbcIO.WriteVoid<T> withRetryConfiguration​(JdbcIO.RetryConfiguration retryConfiguration)
        When a SQL exception occurs, JdbcIO.Write uses this JdbcIO.RetryConfiguration to exponentially back off and retry the statements based on the JdbcIO.RetryConfiguration mentioned.

        Usage of RetryConfiguration - 

        
 pipeline.apply(JdbcIO.<T>write())
    .withDataSourceConfiguration(...)
    .withRetryStrategy(...)
    .withRetryConfiguration(JdbcIO.RetryConfiguration.
        create(5, Duration.standardSeconds(5), Duration.standardSeconds(1))
        maxDuration and initialDuration are Nullable 
        
 pipeline.apply(JdbcIO.<T>write())
    .withDataSourceConfiguration(...)
    .withRetryStrategy(...)
    .withRetryConfiguration(JdbcIO.RetryConfiguration.
        create(5, null, null)

      • expand

        public org.apache.beam.sdk.values.PCollection<java.lang.Void> expand​(org.apache.beam.sdk.values.PCollection<T> input)
        Specified by:
        expand in class org.apache.beam.sdk.transforms.PTransform<org.apache.beam.sdk.values.PCollection<T>,​org.apache.beam.sdk.values.PCollection<java.lang.Void>>