Package com.amazonaws.athena.connectors.jdbc.manager

Class JdbcMetadataHandler

java.lang.Object

com.amazonaws.athena.connector.lambda.handlers.MetadataHandler

com.amazonaws.athena.connectors.jdbc.manager.JdbcMetadataHandler

All Implemented Interfaces:

com.amazonaws.services.lambda.runtime.RequestStreamHandler

Direct Known Subclasses:

MultiplexingJdbcMetadataHandler

public abstract class JdbcMetadataHandler
extends com.amazonaws.athena.connector.lambda.handlers.MetadataHandler

Abstracts JDBC metadata handler and provides common reusable metadata handling.

Field Summary

Fields

Modifier and Type	Field	Description
protected JDBCCaseResolver	caseResolver
protected JdbcQueryPassthrough	jdbcQueryPassthrough
static String	TABLES_AND_VIEWS

Fields inherited from class com.amazonaws.athena.connector.lambda.handlers.MetadataHandler

configOptions, DISABLE_SPILL_ENCRYPTION, KMS_KEY_ID_ENV, SPILL_BUCKET_ENV, SPILL_PREFIX_ENV

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	JdbcMetadataHandler(DatabaseConnectionConfig databaseConnectionConfig, JdbcConnectionFactory jdbcConnectionFactory, Map<String,String> configOptions)
protected	JdbcMetadataHandler(DatabaseConnectionConfig databaseConnectionConfig, JdbcConnectionFactory jdbcConnectionFactory, Map<String,String> configOptions, JDBCCaseResolver caseResolver)
protected	JdbcMetadataHandler(DatabaseConnectionConfig databaseConnectionConfig, software.amazon.awssdk.services.secretsmanager.SecretsManagerClient secretsManager, software.amazon.awssdk.services.athena.AthenaClient athena, JdbcConnectionFactory jdbcConnectionFactory, Map<String,String> configOptions)
protected	JdbcMetadataHandler(DatabaseConnectionConfig databaseConnectionConfig, software.amazon.awssdk.services.secretsmanager.SecretsManagerClient secretsManager, software.amazon.awssdk.services.athena.AthenaClient athena, JdbcConnectionFactory jdbcConnectionFactory, Map<String,String> configOptions, JDBCCaseResolver caseResolver)
protected	JdbcMetadataHandler(String sourceType, Map<String,String> configOptions)	Used only by Multiplexing handler.
protected	JdbcMetadataHandler(String sourceType, Map<String,String> configOptions, JDBCCaseResolver caseResolver)	Used only by Multiplexing handler.

Method Summary

Modifier and Type	Method	Description
protected Optional<org.apache.arrow.vector.types.pojo.ArrowType>	convertDatasourceTypeToArrow(int columnIndex, int precision, Map<String,String> configOptions, ResultSetMetaData metadata)	A method that takes in a JDBC type; and converts it to Arrow Type This can be overriden by other Metadata Handlers extending JDBC
com.amazonaws.athena.connector.lambda.metadata.GetTableResponse	doGetQueryPassthroughSchema(com.amazonaws.athena.connector.lambda.data.BlockAllocator blockAllocator, com.amazonaws.athena.connector.lambda.metadata.GetTableRequest getTableRequest)
abstract com.amazonaws.athena.connector.lambda.metadata.GetSplitsResponse	doGetSplits(com.amazonaws.athena.connector.lambda.data.BlockAllocator blockAllocator, com.amazonaws.athena.connector.lambda.metadata.GetSplitsRequest getSplitsRequest)
com.amazonaws.athena.connector.lambda.metadata.GetTableResponse	doGetTable(com.amazonaws.athena.connector.lambda.data.BlockAllocator blockAllocator, com.amazonaws.athena.connector.lambda.metadata.GetTableRequest getTableRequest)
com.amazonaws.athena.connector.lambda.metadata.ListSchemasResponse	doListSchemaNames(com.amazonaws.athena.connector.lambda.data.BlockAllocator blockAllocator, com.amazonaws.athena.connector.lambda.metadata.ListSchemasRequest listSchemasRequest)
com.amazonaws.athena.connector.lambda.metadata.ListTablesResponse	doListTables(com.amazonaws.athena.connector.lambda.data.BlockAllocator blockAllocator, com.amazonaws.athena.connector.lambda.metadata.ListTablesRequest listTablesRequest)
protected String	escapeNamePattern(String name, String escape)
protected org.apache.arrow.vector.types.pojo.ArrowType	getArrayArrowTypeFromTypeName(String typeName, int precision, int scale)	Converts an ARRAY column's TYPE_NAME (provided by the jdbc metadata) to an ArrowType.
protected ResultSet	getColumns(String catalogName, com.amazonaws.athena.connector.lambda.domain.TableName tableHandle, DatabaseMetaData metadata)
protected com.amazonaws.athena.connector.credentials.CredentialsProvider	getCredentialProvider()
protected JdbcConnectionFactory	getJdbcConnectionFactory()
abstract void	getPartitions(com.amazonaws.athena.connector.lambda.data.BlockWriter blockWriter, com.amazonaws.athena.connector.lambda.metadata.GetTableLayoutRequest request, com.amazonaws.athena.connector.lambda.QueryStatusChecker queryStatusChecker)
abstract org.apache.arrow.vector.types.pojo.Schema	getPartitionSchema(String catalogName)	Delegates creation of partition schema to database type implementation.
protected org.apache.arrow.vector.types.pojo.Schema	getSchema(Connection jdbcConnection, com.amazonaws.athena.connector.lambda.domain.TableName tableName, org.apache.arrow.vector.types.pojo.Schema partitionSchema)
protected List<String>	getSplitClauses(com.amazonaws.athena.connector.lambda.domain.TableName tableName)
protected Set<String>	listDatabaseNames(Connection jdbcConnection)
protected com.amazonaws.athena.connector.lambda.metadata.ListTablesResponse	listPaginatedTables(Connection connection, com.amazonaws.athena.connector.lambda.metadata.ListTablesRequest listTablesRequest)	This is default getAllTables no pagination.
protected List<com.amazonaws.athena.connector.lambda.domain.TableName>	listTables(Connection jdbcConnection, String databaseName)
protected com.amazonaws.athena.connector.lambda.metadata.GetSplitsResponse	setupQueryPassthroughSplit(com.amazonaws.athena.connector.lambda.metadata.GetSplitsRequest request)	Helper function that provides a single partition for Query Pass-Through
protected String	wrapNameWithEscapedCharacter(String input)

Methods inherited from class com.amazonaws.athena.connector.lambda.handlers.MetadataHandler

doGetDataSourceCapabilities, doGetTableLayout, doHandleRequest, doPing, enhancePartitionSchema, getSecret, handleRequest, makeEncryptionKey, makeSpillLocation, onPing, resolveSecrets, resolveWithDefaultCredentials

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

TABLES_AND_VIEWS

public static final String TABLES_AND_VIEWS

See Also:

Constant Field Values

caseResolver

protected final JDBCCaseResolver caseResolver

jdbcQueryPassthrough

protected JdbcQueryPassthrough jdbcQueryPassthrough

Constructor Detail

JdbcMetadataHandler

protected JdbcMetadataHandler(String sourceType,
                              Map<String,String> configOptions)

Used only by Multiplexing handler. All calls will be delegated to respective database handler.

JdbcMetadataHandler

protected JdbcMetadataHandler(String sourceType,
                              Map<String,String> configOptions,
                              JDBCCaseResolver caseResolver)

Used only by Multiplexing handler. All calls will be delegated to respective database handler.

JdbcMetadataHandler

protected JdbcMetadataHandler(DatabaseConnectionConfig databaseConnectionConfig,
                              JdbcConnectionFactory jdbcConnectionFactory,
                              Map<String,String> configOptions)

JdbcMetadataHandler

protected JdbcMetadataHandler(DatabaseConnectionConfig databaseConnectionConfig,
                              JdbcConnectionFactory jdbcConnectionFactory,
                              Map<String,String> configOptions,
                              JDBCCaseResolver caseResolver)

JdbcMetadataHandler

protected JdbcMetadataHandler(DatabaseConnectionConfig databaseConnectionConfig,
                              software.amazon.awssdk.services.secretsmanager.SecretsManagerClient secretsManager,
                              software.amazon.awssdk.services.athena.AthenaClient athena,
                              JdbcConnectionFactory jdbcConnectionFactory,
                              Map<String,String> configOptions)

JdbcMetadataHandler

protected JdbcMetadataHandler(DatabaseConnectionConfig databaseConnectionConfig,
                              software.amazon.awssdk.services.secretsmanager.SecretsManagerClient secretsManager,
                              software.amazon.awssdk.services.athena.AthenaClient athena,
                              JdbcConnectionFactory jdbcConnectionFactory,
                              Map<String,String> configOptions,
                              JDBCCaseResolver caseResolver)

Method Detail

getJdbcConnectionFactory

protected JdbcConnectionFactory getJdbcConnectionFactory()

getCredentialProvider

protected com.amazonaws.athena.connector.credentials.CredentialsProvider getCredentialProvider()

doListSchemaNames

public com.amazonaws.athena.connector.lambda.metadata.ListSchemasResponse doListSchemaNames(com.amazonaws.athena.connector.lambda.data.BlockAllocator blockAllocator,
                                                                                            com.amazonaws.athena.connector.lambda.metadata.ListSchemasRequest listSchemasRequest)
                                                                                     throws Exception

Specified by:

doListSchemaNames in class com.amazonaws.athena.connector.lambda.handlers.MetadataHandler

Throws:

Exception

listDatabaseNames

protected Set<String> listDatabaseNames(Connection jdbcConnection)
                                 throws SQLException

Throws:

SQLException

doListTables

public com.amazonaws.athena.connector.lambda.metadata.ListTablesResponse doListTables(com.amazonaws.athena.connector.lambda.data.BlockAllocator blockAllocator,
                                                                                      com.amazonaws.athena.connector.lambda.metadata.ListTablesRequest listTablesRequest)
                                                                               throws Exception

Specified by:

doListTables in class com.amazonaws.athena.connector.lambda.handlers.MetadataHandler

Throws:

Exception

listPaginatedTables

protected com.amazonaws.athena.connector.lambda.metadata.ListTablesResponse listPaginatedTables(Connection connection,
                                                                                                com.amazonaws.athena.connector.lambda.metadata.ListTablesRequest listTablesRequest)
                                                                                         throws SQLException

This is default getAllTables no pagination. Override this if you want to support the behavior.

Parameters:

connection -

listTablesRequest -

Returns:

Throws:

SQLException

listTables

protected List<com.amazonaws.athena.connector.lambda.domain.TableName> listTables(Connection jdbcConnection,
                                                                                  String databaseName)
                                                                           throws SQLException

Throws:

SQLException

escapeNamePattern

protected String escapeNamePattern(String name,
                                   String escape)

doGetTable

public com.amazonaws.athena.connector.lambda.metadata.GetTableResponse doGetTable(com.amazonaws.athena.connector.lambda.data.BlockAllocator blockAllocator,
                                                                                  com.amazonaws.athena.connector.lambda.metadata.GetTableRequest getTableRequest)
                                                                           throws Exception

Specified by:

doGetTable in class com.amazonaws.athena.connector.lambda.handlers.MetadataHandler

Throws:

Exception

doGetQueryPassthroughSchema

public com.amazonaws.athena.connector.lambda.metadata.GetTableResponse doGetQueryPassthroughSchema(com.amazonaws.athena.connector.lambda.data.BlockAllocator blockAllocator,
                                                                                                   com.amazonaws.athena.connector.lambda.metadata.GetTableRequest getTableRequest)
                                                                                            throws Exception

Overrides:

doGetQueryPassthroughSchema in class com.amazonaws.athena.connector.lambda.handlers.MetadataHandler

Throws:

Exception

convertDatasourceTypeToArrow

protected Optional<org.apache.arrow.vector.types.pojo.ArrowType> convertDatasourceTypeToArrow(int columnIndex,
                                                                                              int precision,
                                                                                              Map<String,String> configOptions,
                                                                                              ResultSetMetaData metadata)
                                                                                       throws SQLException

A method that takes in a JDBC type; and converts it to Arrow Type This can be overriden by other Metadata Handlers extending JDBC

Parameters:

columnIndex -

precision -

configOptions -

metadata -

Returns:

Arrow Type

Throws:

SQLException

getSchema

protected org.apache.arrow.vector.types.pojo.Schema getSchema(Connection jdbcConnection,
                                                              com.amazonaws.athena.connector.lambda.domain.TableName tableName,
                                                              org.apache.arrow.vector.types.pojo.Schema partitionSchema)
                                                       throws Exception

Throws:

Exception

getColumns

protected ResultSet getColumns(String catalogName,
                               com.amazonaws.athena.connector.lambda.domain.TableName tableHandle,
                               DatabaseMetaData metadata)
                        throws SQLException

Throws:

SQLException

getPartitionSchema

public abstract org.apache.arrow.vector.types.pojo.Schema getPartitionSchema(String catalogName)

Delegates creation of partition schema to database type implementation.

Parameters:

catalogName - Athena provided catalog name.

Returns:

schema. See Schema

getPartitions

public abstract void getPartitions(com.amazonaws.athena.connector.lambda.data.BlockWriter blockWriter,
                                   com.amazonaws.athena.connector.lambda.metadata.GetTableLayoutRequest request,
                                   com.amazonaws.athena.connector.lambda.QueryStatusChecker queryStatusChecker)
                            throws Exception

Specified by:

getPartitions in class com.amazonaws.athena.connector.lambda.handlers.MetadataHandler

Throws:

Exception

doGetSplits

public abstract com.amazonaws.athena.connector.lambda.metadata.GetSplitsResponse doGetSplits(com.amazonaws.athena.connector.lambda.data.BlockAllocator blockAllocator,
                                                                                             com.amazonaws.athena.connector.lambda.metadata.GetSplitsRequest getSplitsRequest)

Specified by:

doGetSplits in class com.amazonaws.athena.connector.lambda.handlers.MetadataHandler

getSplitClauses

protected List<String> getSplitClauses(com.amazonaws.athena.connector.lambda.domain.TableName tableName)

getArrayArrowTypeFromTypeName

protected org.apache.arrow.vector.types.pojo.ArrowType getArrayArrowTypeFromTypeName(String typeName,
                                                                                     int precision,
                                                                                     int scale)

Converts an ARRAY column's TYPE_NAME (provided by the jdbc metadata) to an ArrowType.

Parameters:

typeName - The column's TYPE_NAME (e.g. _int4, _text, _float8, etc...)

precision - Used for BigDecimal ArrowType

scale - Used for BigDecimal ArrowType

Returns:

Utf8 ArrowType (VARCHAR)

setupQueryPassthroughSplit

protected com.amazonaws.athena.connector.lambda.metadata.GetSplitsResponse setupQueryPassthroughSplit(com.amazonaws.athena.connector.lambda.metadata.GetSplitsRequest request)

Helper function that provides a single partition for Query Pass-Through

wrapNameWithEscapedCharacter

protected String wrapNameWithEscapedCharacter(String input)