org.apache.hadoop.dynamodb

Class DynamoDBUtil

java.lang.Object

org.apache.hadoop.dynamodb.DynamoDBUtil

public final class DynamoDBUtil
extends Object

Field Summary

Fields

Modifier and Type	Field and Description
static String	CHARACTER_ENCODING

Method Summary

Modifier and Type	Method and Description
static byte[]	base64DecodeString(String base64String) base64 decode a base64String using org.apache.commons.codec.binary.Base64
static String	base64EncodeByteArray(byte[] bytes) base64 encode a byte array using org.apache.commons.codec.binary.Base64
static List<ByteBuffer>	base64StringToByteBuffer(String... base64EncodedKeys) Converts a given list of base64EncodedKeys to a List of ByteBuffers
static ByteBuffer	base64StringToByteBuffer(String base64EncodedKey) Converts a base64 encoded key into a ByteBuffer
static int	calcMaxMapTasks(org.apache.hadoop.mapred.JobClient jobClient)
static Double	calculateAverageItemSize(com.amazonaws.services.dynamodbv2.model.TableDescription description)
static org.apache.hadoop.mapred.JobClient	createJobClient(org.apache.hadoop.mapred.JobConf jobConf)
static String	getDynamoDBEndpoint(org.apache.hadoop.conf.Configuration conf, String region) Calculates DynamoDB end-point.
static com.google.gson.Gson	getGson() Get a Gson reference with custom ByteBuffer serializer/deserializer.
static String	getValueFromConf(org.apache.hadoop.conf.Configuration conf, String confKey)
static String	getValueFromConf(org.apache.hadoop.conf.Configuration conf, String confKey, String defaultValue)
static void	verifyInterval(long intervalBeginTime, long intervalLength)

Methods inherited from class java.lang.Object

equals, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CHARACTER_ENCODING

public static final String CHARACTER_ENCODING

See Also:

Constant Field Values

Method Detail

calculateAverageItemSize

public static Double calculateAverageItemSize(com.amazonaws.services.dynamodbv2.model.TableDescription description)

base64EncodeByteArray

public static String base64EncodeByteArray(byte[] bytes)

base64 encode a byte array using org.apache.commons.codec.binary.Base64

Parameters:

bytes - bytes to encode

Returns:

base64 encoded representation of the provided byte array

base64DecodeString

public static byte[] base64DecodeString(String base64String)

base64 decode a base64String using org.apache.commons.codec.binary.Base64

Parameters:

base64String - string to base64 decode

Returns:

byte array representing the decoded base64 string

base64StringToByteBuffer

public static ByteBuffer base64StringToByteBuffer(String base64EncodedKey)

Converts a base64 encoded key into a ByteBuffer

Parameters:

base64EncodedKey - base64 encoded key to be converted

Returns:

ByteBuffer representation of the provided base64 encoded key string

base64StringToByteBuffer

public static List<ByteBuffer> base64StringToByteBuffer(String... base64EncodedKeys)

Converts a given list of base64EncodedKeys to a List of ByteBuffers

Parameters:

base64EncodedKeys - base64 encoded key(s) to be converted

Returns:

List of ByteBuffers representing the provided base64EncodedKeys

getGson

public static com.google.gson.Gson getGson()

Get a Gson reference with custom ByteBuffer serializer/deserializer.

Returns:

Gson reference with custom ByteBuffer serializer/deserializer

verifyInterval

public static void verifyInterval(long intervalBeginTime,
                                  long intervalLength)

getValueFromConf

public static String getValueFromConf(org.apache.hadoop.conf.Configuration conf,
                                      String confKey,
                                      String defaultValue)

getValueFromConf

public static String getValueFromConf(org.apache.hadoop.conf.Configuration conf,
                                      String confKey)

getDynamoDBEndpoint

public static String getDynamoDBEndpoint(org.apache.hadoop.conf.Configuration conf,
                                         String region)

Calculates DynamoDB end-point. Algorithm details:

1. Use endpoint in job configuration "dynamodb.endpoint" value if available
2. Use endpoint from region in job configuration "dynamodb.region" value if available
3. Use endpoint from region in job configuration "dynamodb.regionid" value if available
4. Use endpoint from EC2 Metadata of instance if available
5. If all previous attempts at retrieving endpoint fail, default to us-east-1 endpoint

Parameters:

conf - Job Configuration

region - optional preferred region

Returns:

end-point for DynamoDb service

createJobClient

public static org.apache.hadoop.mapred.JobClient createJobClient(org.apache.hadoop.mapred.JobConf jobConf)

calcMaxMapTasks

public static int calcMaxMapTasks(org.apache.hadoop.mapred.JobClient jobClient)
                           throws IOException

Throws:

IOException