org.apache.hadoop.fs.s3a.s3guard

Class DynamoDBMetadataStoreTableManager

java.lang.Object

org.apache.hadoop.fs.s3a.s3guard.DynamoDBMetadataStoreTableManager

public class DynamoDBMetadataStoreTableManager
extends Object

Managing dynamo tables for S3Guard dynamodb based metadatastore. Factored out from DynamoDBMetadataStore.

Field Summary

Fields

Modifier and Type	Field and Description
static String	E_INCOMPATIBLE_ITEM_VERSION Error: version mismatch.
static String	E_INCOMPATIBLE_TAG_VERSION Error: version mismatch.
static String	E_NO_VERSION_MARKER_AND_NOT_EMPTY Error: version marker not found in table but the table is not empty.
static org.slf4j.Logger	LOG
static String	SSE_DEFAULT_MASTER_KEY The AWS managed CMK for DynamoDB server side encryption.

Constructor Summary

Constructors

Constructor and Description
DynamoDBMetadataStoreTableManager(com.amazonaws.services.dynamodbv2.document.DynamoDB dynamoDB, String tableName, String region, com.amazonaws.services.dynamodbv2.AmazonDynamoDB amazonDynamoDB, org.apache.hadoop.conf.Configuration conf, Invoker readOp, org.apache.hadoop.io.retry.RetryPolicy batchWriteCapacityExceededEvents)

Method Summary

Modifier and Type	Method and Description
void	destroy()
static com.amazonaws.SdkBaseException	extractInnerException(IllegalArgumentException ex) Take an IllegalArgumentException raised by a DDB operation and if it contains an inner SDK exception, unwrap it.
com.amazonaws.services.dynamodbv2.document.Table	getTable()
String	getTableArn()
List<com.amazonaws.services.dynamodbv2.model.Tag>	getTableTagsFromConfig() Return tags from configuration and the version marker for adding to dynamo table during creation.
protected static com.amazonaws.services.dynamodbv2.document.Item	getVersionMarkerFromTags(com.amazonaws.services.dynamodbv2.document.Table table, com.amazonaws.services.dynamodbv2.AmazonDynamoDB addb)
protected com.amazonaws.services.dynamodbv2.document.Item	getVersionMarkerItem() Get the version mark item in the existing DynamoDB table.
protected void	tagTableWithVersionMarker()
protected void	verifyVersionCompatibility() Verify that a table version is compatible with this S3Guard client.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

LOG

public static final org.slf4j.Logger LOG

E_NO_VERSION_MARKER_AND_NOT_EMPTY

public static final String E_NO_VERSION_MARKER_AND_NOT_EMPTY

Error: version marker not found in table but the table is not empty.

See Also:

Constant Field Values

E_INCOMPATIBLE_TAG_VERSION

public static final String E_INCOMPATIBLE_TAG_VERSION

Error: version mismatch.

See Also:

Constant Field Values

E_INCOMPATIBLE_ITEM_VERSION

public static final String E_INCOMPATIBLE_ITEM_VERSION

Error: version mismatch.

See Also:

Constant Field Values

SSE_DEFAULT_MASTER_KEY

public static final String SSE_DEFAULT_MASTER_KEY

The AWS managed CMK for DynamoDB server side encryption.

See Also:

Constant Field Values

Constructor Detail

DynamoDBMetadataStoreTableManager

public DynamoDBMetadataStoreTableManager(com.amazonaws.services.dynamodbv2.document.DynamoDB dynamoDB,
                                         String tableName,
                                         String region,
                                         com.amazonaws.services.dynamodbv2.AmazonDynamoDB amazonDynamoDB,
                                         org.apache.hadoop.conf.Configuration conf,
                                         Invoker readOp,
                                         org.apache.hadoop.io.retry.RetryPolicy batchWriteCapacityExceededEvents)

Method Detail

tagTableWithVersionMarker

protected void tagTableWithVersionMarker()
                                  throws com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException

Throws:

com.amazonaws.services.dynamodbv2.model.AmazonDynamoDBException

getVersionMarkerFromTags

protected static com.amazonaws.services.dynamodbv2.document.Item getVersionMarkerFromTags(com.amazonaws.services.dynamodbv2.document.Table table,
                                                                                          com.amazonaws.services.dynamodbv2.AmazonDynamoDB addb)
                                                                                   throws IOException

Throws:

IOException

getTableTagsFromConfig

@Retries.OnceRaw
public List<com.amazonaws.services.dynamodbv2.model.Tag> getTableTagsFromConfig()

Return tags from configuration and the version marker for adding to dynamo table during creation.

verifyVersionCompatibility

protected void verifyVersionCompatibility()
                                   throws IOException

Verify that a table version is compatible with this S3Guard client. Checks for consistency between the version marker as the item and tag.

   1. If the table lacks both version markers AND it's empty,
      both markers will be added.
      If the table is not empty the check throws IOException
   2. If there's no version marker ITEM, the compatibility with the TAG
      will be checked, and the version marker ITEM will be added if the
      TAG version is compatible.
      If the TAG version is not compatible, the check throws OException
   3. If there's no version marker TAG, the compatibility with the ITEM
      version marker will be checked, and the version marker ITEM will be
      added if the ITEM version is compatible.
      If the ITEM version is not compatible, the check throws IOException
   4. If the TAG and ITEM versions are both present then both will be checked
      for compatibility. If the ITEM or TAG version marker is not compatible,
      the check throws IOException
 

Throws:

IOException - on any incompatibility

extractInnerException

public static com.amazonaws.SdkBaseException extractInnerException(IllegalArgumentException ex)

Take an IllegalArgumentException raised by a DDB operation and if it contains an inner SDK exception, unwrap it.

Parameters:

ex - exception.

Returns:

the inner AWS exception or null.

getVersionMarkerItem

@Retries.RetryTranslated
protected com.amazonaws.services.dynamodbv2.document.Item getVersionMarkerItem()
                                                                                                 throws IOException

Get the version mark item in the existing DynamoDB table. As the version marker item may be created by another concurrent thread or process, we sleep and retry a limited number times if the lookup returns with a null value. DDB throttling is always retried.

Throws:

IOException

destroy

@Retries.RetryTranslated
public void destroy()
                                      throws IOException

Throws:

IOException

getTable

public com.amazonaws.services.dynamodbv2.document.Table getTable()

getTableArn

public String getTableArn()