Package com.d3x.morpheus.db

Class DbSinkOptions<R,C>

java.lang.Object

com.d3x.morpheus.db.DbSinkOptions<R,C>

public class DbSinkOptions<R,C>
extends java.lang.Object

A class that defines all the options that can be configured to write a DataFrame to a SQL data store.

This is open source software released under the Apache 2.0 License

Author:

Xavier Witdouck

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
class	DbSinkOptions.ColumnMappings	A class that maintains a mapping between a DataFrame column type and a JDBC column type

Field Summary

Fields

Modifier and Type	Field	Description
private java.lang.String	autoIncrementColumnName
private int	batchSize
private DbSinkOptions.ColumnMappings	columnMappings
private java.util.function.Function<C,java.lang.String>	columnNames
private java.sql.Connection	connection
private com.d3x.morpheus.util.sql.SQLPlatform	platform
private java.lang.String	rowKeyColumn
private com.d3x.morpheus.util.functions.Function1<R,?>	rowKeyMapper
private java.lang.Class<?>	rowKeySqlClass
private java.lang.String	tableName

Constructor Summary

Constructors

Constructor	Description
DbSinkOptions()	Constructor

Method Summary

Modifier and Type	Method	Description
(package private) java.util.Optional<java.lang.String>	getAutoIncrementColumnName()	Returns the optional column name for the auto increment field
(package private) int	getBatchSize()	Returns the batch size for inserts
DbSinkOptions.ColumnMappings	getColumnMappings()	Returns the column mappings for these options
(package private) java.util.function.Function<C,java.lang.String>	getColumnNames()	Returns the function that yield column names for DataFrame column keys
(package private) java.sql.Connection	getConnection()	Returns the connection for these options
(package private) java.util.Optional<com.d3x.morpheus.util.sql.SQLPlatform>	getPlatform()	Returns the SQL platform for these options
(package private) java.util.Optional<java.lang.String>	getRowKeyColumn()	Returns the column name to store DataFrame row keys in
(package private) java.util.Optional<com.d3x.morpheus.util.functions.Function1<R,?>>	getRowKeyMapper()	Returns the mapper that maps row keys to the appropriate JDBC type
(package private) java.util.Optional<java.lang.Class<?>>	getRowKeySqlClass()	Returns the SQL type to apply row keys to
(package private) java.lang.String	getTableName()	Returns the table name to write to
void	setAutoIncrementColumnName(java.lang.String autoIncrementColumnName)	Sets the name of an auto increment column name if one exists in the table
void	setBatchSize(int batchSize)	Sets the batch size for write operations
void	setColumnMappings(java.util.function.Consumer<DbSinkOptions.ColumnMappings> configurator)	Sets the column mappings for these options
void	setColumnNames(java.util.function.Function<C,java.lang.String> columnNames)	Sets the column name generating function
void	setConnection(java.lang.String url, java.lang.String username, java.lang.String password)	Sets the JDBC connection URL and optional credentials This method should ideally only be used for testing, a connection pool is preferable for production
void	setConnection(java.sql.Connection connection)	Sets the JDBC connection for this request
void	setConnection(javax.sql.DataSource dataSource)	Sets the JDBC data source for this request
void	setPlatform(com.d3x.morpheus.util.sql.SQLPlatform platform)	Sets the target SQL platform for these options
<T> void	setRowKeyMapping(java.lang.String colName, java.lang.Class<T> sqlType, com.d3x.morpheus.util.functions.Function1<R,T> mapper)	Sets the row key mapping for these database options
void	setTableName(java.lang.String tableName)	Sets the target table name to write to

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

batchSize

private int batchSize

tableName

private java.lang.String tableName

connection

private java.sql.Connection connection

platform

private com.d3x.morpheus.util.sql.SQLPlatform platform

columnMappings

private DbSinkOptions.ColumnMappings columnMappings

autoIncrementColumnName

private java.lang.String autoIncrementColumnName

columnNames

private java.util.function.Function<C,java.lang.String> columnNames

rowKeyColumn

private java.lang.String rowKeyColumn

rowKeySqlClass

private java.lang.Class<?> rowKeySqlClass

rowKeyMapper

private com.d3x.morpheus.util.functions.Function1<R,?> rowKeyMapper

Constructor Detail

DbSinkOptions

public DbSinkOptions()

Constructor

Method Detail

getBatchSize

int getBatchSize()

Returns the batch size for inserts

Returns:

the batch size

getTableName

java.lang.String getTableName()

Returns the table name to write to

Returns:

the table name

getPlatform

java.util.Optional<com.d3x.morpheus.util.sql.SQLPlatform> getPlatform()

Returns the SQL platform for these options

Returns:

the SQL platform

getConnection

java.sql.Connection getConnection()

Returns the connection for these options

Returns:

the database connection

getRowKeyColumn

java.util.Optional<java.lang.String> getRowKeyColumn()

Returns the column name to store DataFrame row keys in

Returns:

the optional column to store row keys

getRowKeySqlClass

java.util.Optional<java.lang.Class<?>> getRowKeySqlClass()

Returns the SQL type to apply row keys to

Returns:

the SQL type for row keys

getRowKeyMapper

java.util.Optional<com.d3x.morpheus.util.functions.Function1<R,?>> getRowKeyMapper()

Returns the mapper that maps row keys to the appropriate JDBC type

Returns:

the row key mapper if specified

getAutoIncrementColumnName

java.util.Optional<java.lang.String> getAutoIncrementColumnName()

Returns the optional column name for the auto increment field

Returns:

the optional auto increment column name

getColumnMappings

public DbSinkOptions.ColumnMappings getColumnMappings()

Returns the column mappings for these options

Returns:

the column mappings

getColumnNames

java.util.function.Function<C,java.lang.String> getColumnNames()

Returns the function that yield column names for DataFrame column keys

Returns:

the function that yields column names for frame column keys

setBatchSize

public void setBatchSize(int batchSize)

Sets the batch size for write operations

Parameters:

batchSize - the batch size

setTableName

public void setTableName(java.lang.String tableName)

Sets the target table name to write to

Parameters:

tableName - the fully qualified table name

setConnection

public void setConnection(java.sql.Connection connection)

Sets the JDBC connection for this request

Parameters:

connection - the connection for this request

setConnection

public void setConnection(javax.sql.DataSource dataSource)

Sets the JDBC data source for this request

Parameters:

dataSource - the DataSource the grab a connection from

setConnection

public void setConnection(java.lang.String url,
                          java.lang.String username,
                          java.lang.String password)

Sets the JDBC connection URL and optional credentials This method should ideally only be used for testing, a connection pool is preferable for production

Parameters:

url - the JDBC connection url

username - the JDBC connection username, can be null

password - the JDBC connection password, can be null

setRowKeyMapping

public <T> void setRowKeyMapping(java.lang.String colName,
                                 java.lang.Class<T> sqlType,
                                 com.d3x.morpheus.util.functions.Function1<R,T> mapper)

Sets the row key mapping for these database options

Type Parameters:

T - the mapped row key type

Parameters:

colName - the column to apply row keys to

sqlType - the target SQL type for row key column

mapper - the row key mapper function

setAutoIncrementColumnName

public void setAutoIncrementColumnName(java.lang.String autoIncrementColumnName)

Sets the name of an auto increment column name if one exists in the table

Parameters:

autoIncrementColumnName - the auto increment column name

setPlatform

public void setPlatform(com.d3x.morpheus.util.sql.SQLPlatform platform)

Sets the target SQL platform for these options

Parameters:

platform - the target SQL platform

setColumnNames

public void setColumnNames(java.util.function.Function<C,java.lang.String> columnNames)

Sets the column name generating function

Parameters:

columnNames - the column name generating function

setColumnMappings

public void setColumnMappings(java.util.function.Consumer<DbSinkOptions.ColumnMappings> configurator)

Sets the column mappings for these options

Parameters:

configurator - the configurator for column mappings