com.amazonaws.services.kinesis.clientlibrary.proxies

Class KinesisProxy

java.lang.Object

com.amazonaws.services.kinesis.clientlibrary.proxies.KinesisProxy

All Implemented Interfaces:

IKinesisProxy, IKinesisProxyExtended

public class KinesisProxy
extends Object
implements IKinesisProxyExtended

Kinesis proxy - used to make calls to Amazon Kinesis (e.g. fetch data records and list of shards).

Field Summary

Fields

Modifier and Type	Field and Description
static Duration	CACHE_MAX_ALLOWED_AGE
static int	CACHE_MISS_WARNING_MODULUS
static int	MAX_CACHE_MISSES_BEFORE_RELOAD

Constructor Summary

Constructors

Constructor and Description
KinesisProxy(KinesisClientLibConfiguration config, com.amazonaws.services.kinesis.AmazonKinesis client) Public constructor.
KinesisProxy(String streamName, com.amazonaws.services.kinesis.AmazonKinesis client, long describeStreamBackoffTimeInMillis, int maxDescribeStreamRetryAttempts, long listShardsBackoffTimeInMillis, int maxListShardsRetryAttempts)
KinesisProxy(String streamName, com.amazonaws.auth.AWSCredentialsProvider credentialProvider, com.amazonaws.services.kinesis.AmazonKinesis kinesisClient, long describeStreamBackoffTimeInMillis, int maxDescribeStreamRetryAttempts, long listShardsBackoffTimeInMillis, int maxListShardsRetryAttempts) Deprecated. Deprecating constructor, this constructor doesn't use AWS best practices, moving forward please use KinesisProxy(KinesisClientLibConfiguration, AmazonKinesis) or KinesisProxy(String, AmazonKinesis, long, int, long, int) to create the object. Will be removed in the next major/minor release.
KinesisProxy(String streamName, com.amazonaws.auth.AWSCredentialsProvider credentialProvider, String endpoint) Deprecated. Deprecating constructor, this constructor doesn't use AWS best practices, moving forward please use KinesisProxy(KinesisClientLibConfiguration, AmazonKinesis) or KinesisProxy(String, AmazonKinesis, long, int, long, int) to create the object. Will be removed in the next major/minor release.
KinesisProxy(String streamName, com.amazonaws.auth.AWSCredentialsProvider credentialProvider, String endpoint, String serviceName, String regionId, long describeStreamBackoffTimeInMillis, int maxDescribeStreamRetryAttempts, long listShardsBackoffTimeInMillis, int maxListShardsRetryAttempts) Deprecated. Deprecating constructor, this constructor doesn't use AWS best practices, moving forward please use KinesisProxy(KinesisClientLibConfiguration, AmazonKinesis) or KinesisProxy(String, AmazonKinesis, long, int, long, int) to create the object. Will be removed in the next major/minor release.

Method Summary

Modifier and Type	Method and Description
com.amazonaws.services.kinesis.model.GetRecordsResult	get(String shardIterator, int maxRecords) Get records from stream.
Set<String>	getAllShardIds() Fetch the shardIds of all shards in the stream.
String	getIterator(String shardId, Date timestamp) Fetch a shard iterator from the specified position in the shard.
String	getIterator(String shardId, String iteratorType) Fetch a shard iterator from the specified position in the shard.
String	getIterator(String shardId, String iteratorType, String sequenceNumber) Fetch a shard iterator from the specified position in the shard.
com.amazonaws.services.kinesis.model.Shard	getShard(String shardId) Get the Shard corresponding to shardId associated with this IKinesisProxy.
List<com.amazonaws.services.kinesis.model.Shard>	getShardList() Fetch all the shards defined for the stream (e.g.
List<com.amazonaws.services.kinesis.model.Shard>	getShardListWithFilter(com.amazonaws.services.kinesis.model.ShardFilter shardFilter) Fetch a subset shards defined for the stream using a filter on the ListShards API.
com.amazonaws.services.kinesis.model.DescribeStreamResult	getStreamInfo(String startShardId) Deprecated.
com.amazonaws.services.kinesis.model.PutRecordResult	put(String exclusiveMinimumSequenceNumber, String explicitHashKey, String partitionKey, ByteBuffer data)
ShardClosureVerificationResponse	verifyShardClosure(String shardId) Used to verify during ShardConsumer shutdown if the provided shardId is for a shard that has been closed.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

MAX_CACHE_MISSES_BEFORE_RELOAD

public static final int MAX_CACHE_MISSES_BEFORE_RELOAD

See Also:

Constant Field Values

CACHE_MAX_ALLOWED_AGE

public static final Duration CACHE_MAX_ALLOWED_AGE

CACHE_MISS_WARNING_MODULUS

public static final int CACHE_MISS_WARNING_MODULUS

See Also:

Constant Field Values

Constructor Detail

KinesisProxy

@Deprecated
public KinesisProxy(String streamName,
                                com.amazonaws.auth.AWSCredentialsProvider credentialProvider,
                                String endpoint)

Deprecated. Deprecating constructor, this constructor doesn't use AWS best practices, moving forward please use KinesisProxy(KinesisClientLibConfiguration, AmazonKinesis) or KinesisProxy(String, AmazonKinesis, long, int, long, int) to create the object. Will be removed in the next major/minor release.

Public constructor.

Parameters:

streamName - Data records will be fetched from this stream

credentialProvider - Provides credentials for signing Kinesis requests

endpoint - Kinesis endpoint

KinesisProxy

@Deprecated
public KinesisProxy(String streamName,
                                com.amazonaws.auth.AWSCredentialsProvider credentialProvider,
                                String endpoint,
                                String serviceName,
                                String regionId,
                                long describeStreamBackoffTimeInMillis,
                                int maxDescribeStreamRetryAttempts,
                                long listShardsBackoffTimeInMillis,
                                int maxListShardsRetryAttempts)

Deprecated. Deprecating constructor, this constructor doesn't use AWS best practices, moving forward please use KinesisProxy(KinesisClientLibConfiguration, AmazonKinesis) or KinesisProxy(String, AmazonKinesis, long, int, long, int) to create the object. Will be removed in the next major/minor release.

Public constructor.

Parameters:

streamName - Data records will be fetched from this stream

credentialProvider - Provides credentials for signing Kinesis requests

endpoint - Kinesis endpoint

serviceName - service name

regionId - region id

describeStreamBackoffTimeInMillis - Backoff time for DescribeStream calls in milliseconds

maxDescribeStreamRetryAttempts - Number of retry attempts for DescribeStream calls

KinesisProxy

@Deprecated
public KinesisProxy(String streamName,
                                com.amazonaws.auth.AWSCredentialsProvider credentialProvider,
                                com.amazonaws.services.kinesis.AmazonKinesis kinesisClient,
                                long describeStreamBackoffTimeInMillis,
                                int maxDescribeStreamRetryAttempts,
                                long listShardsBackoffTimeInMillis,
                                int maxListShardsRetryAttempts)

Deprecated. Deprecating constructor, this constructor doesn't use AWS best practices, moving forward please use KinesisProxy(KinesisClientLibConfiguration, AmazonKinesis) or KinesisProxy(String, AmazonKinesis, long, int, long, int) to create the object. Will be removed in the next major/minor release.

Public constructor.

Parameters:

streamName - Data records will be fetched from this stream

credentialProvider - Provides credentials for signing Kinesis requests

kinesisClient - Kinesis client (used to fetch data from Kinesis)

describeStreamBackoffTimeInMillis - Backoff time for DescribeStream calls in milliseconds

maxDescribeStreamRetryAttempts - Number of retry attempts for DescribeStream calls

KinesisProxy

public KinesisProxy(KinesisClientLibConfiguration config,
                    com.amazonaws.services.kinesis.AmazonKinesis client)

Public constructor.

Parameters:

config -

KinesisProxy

public KinesisProxy(String streamName,
                    com.amazonaws.services.kinesis.AmazonKinesis client,
                    long describeStreamBackoffTimeInMillis,
                    int maxDescribeStreamRetryAttempts,
                    long listShardsBackoffTimeInMillis,
                    int maxListShardsRetryAttempts)

Method Detail

get

public com.amazonaws.services.kinesis.model.GetRecordsResult get(String shardIterator,
                                                                 int maxRecords)
                                                          throws com.amazonaws.services.kinesis.model.ResourceNotFoundException,
                                                                 com.amazonaws.services.kinesis.model.InvalidArgumentException,
                                                                 com.amazonaws.services.kinesis.model.ExpiredIteratorException

Get records from stream.

Specified by:

get in interface IKinesisProxy

Parameters:

shardIterator - Fetch data records using this shard iterator

maxRecords - Fetch at most this many records

Returns:

List of data records from Kinesis.

Throws:

com.amazonaws.services.kinesis.model.ResourceNotFoundException - The Kinesis stream or shard was not found

com.amazonaws.services.kinesis.model.InvalidArgumentException - Invalid input parameters

com.amazonaws.services.kinesis.model.ExpiredIteratorException - The iterator has expired

getStreamInfo

@Deprecated
public com.amazonaws.services.kinesis.model.DescribeStreamResult getStreamInfo(String startShardId)
                                                                                    throws com.amazonaws.services.kinesis.model.ResourceNotFoundException,
                                                                                           com.amazonaws.services.kinesis.model.LimitExceededException

Deprecated.

Fetch information about stream. Useful for fetching the list of shards in a stream.

Specified by:

getStreamInfo in interface IKinesisProxy

Parameters:

startShardId - exclusive start shardId - used when paginating the list of shards.

Returns:

DescribeStreamOutput object containing a description of the stream.

Throws:

com.amazonaws.services.kinesis.model.ResourceNotFoundException - The Kinesis stream was not found

com.amazonaws.services.kinesis.model.LimitExceededException

getShard

public com.amazonaws.services.kinesis.model.Shard getShard(String shardId)

Get the Shard corresponding to shardId associated with this IKinesisProxy.

Specified by:

getShard in interface IKinesisProxyExtended

Parameters:

shardId - Fetch the Shard with this given shardId

Returns:

the Shard with the given shardId

getShardList

public List<com.amazonaws.services.kinesis.model.Shard> getShardList()

Fetch all the shards defined for the stream (e.g. obtained via calls to the DescribeStream API). This can be used to discover new shards and consume data from them.

Specified by:

getShardList in interface IKinesisProxy

Returns:

List of all shards in the Kinesis stream.

getShardListWithFilter

public List<com.amazonaws.services.kinesis.model.Shard> getShardListWithFilter(com.amazonaws.services.kinesis.model.ShardFilter shardFilter)

Fetch a subset shards defined for the stream using a filter on the ListShards API. This can be used to discover new shards and consume data from them, while limiting the total number of shards returned for performance or efficiency reasons.

Specified by:

getShardListWithFilter in interface IKinesisProxy

Parameters:

shardFilter - currently supported filter types are AT_LATEST, AT_TRIM_HORIZON, AT_TIMESTAMP.

Returns:

List of all shards in the Kinesis stream.

verifyShardClosure

public ShardClosureVerificationResponse verifyShardClosure(String shardId)

Used to verify during ShardConsumer shutdown if the provided shardId is for a shard that has been closed. Also returns the list of shards wrapped in the response so that it can be used for lease creation (instead of calling getShardList() again).

Specified by:

verifyShardClosure in interface IKinesisProxy

Parameters:

shardId - Id of the shard that needs to be verified.

Returns:

an Object of type ShardClosureVerificationResponse.

getAllShardIds

public Set<String> getAllShardIds()
                           throws com.amazonaws.services.kinesis.model.ResourceNotFoundException

Fetch the shardIds of all shards in the stream.

Specified by:

getAllShardIds in interface IKinesisProxy

Returns:

Set of all shardIds

Throws:

com.amazonaws.services.kinesis.model.ResourceNotFoundException - If the specified Kinesis stream was not found

getIterator

public String getIterator(String shardId,
                          String iteratorType,
                          String sequenceNumber)

Fetch a shard iterator from the specified position in the shard. This is to fetch a shard iterator for ShardIteratorType AT_SEQUENCE_NUMBER or AFTER_SEQUENCE_NUMBER which requires the starting sequence number. NOTE: Currently this method continues to fetch iterators for ShardIteratorTypes TRIM_HORIZON, LATEST, AT_SEQUENCE_NUMBER and AFTER_SEQUENCE_NUMBER. But this behavior will change in the next release, after which this method will only serve AT_SEQUENCE_NUMBER or AFTER_SEQUENCE_NUMBER ShardIteratorTypes. We recommend users who call this method directly to use the appropriate getIterator method based on the ShardIteratorType.

Specified by:

getIterator in interface IKinesisProxy

Parameters:

shardId - Shard id

iteratorType - one of: TRIM_HORIZON, LATEST, AT_SEQUENCE_NUMBER, AFTER_SEQUENCE_NUMBER

sequenceNumber - the sequence number - must be null unless iteratorEnum is AT_SEQUENCE_NUMBER or AFTER_SEQUENCE_NUMBER

Returns:

shard iterator which can be used to read data from Kinesis.

getIterator

public String getIterator(String shardId,
                          String iteratorType)

Fetch a shard iterator from the specified position in the shard. This is to fetch a shard iterator for ShardIteratorType LATEST or TRIM_HORIZON which doesn't require a starting sequence number.

Specified by:

getIterator in interface IKinesisProxy

Parameters:

shardId - Shard id

iteratorType - Either TRIM_HORIZON or LATEST.

Returns:

shard iterator which can be used to read data from Kinesis.

getIterator

public String getIterator(String shardId,
                          Date timestamp)

Fetch a shard iterator from the specified position in the shard. This is to fetch a shard iterator for ShardIteratorType AT_TIMESTAMP which requires the timestamp field.

Specified by:

getIterator in interface IKinesisProxy

Parameters:

shardId - Shard id

timestamp - The timestamp.

Returns:

shard iterator which can be used to read data from Kinesis.

put

public com.amazonaws.services.kinesis.model.PutRecordResult put(String exclusiveMinimumSequenceNumber,
                                                                String explicitHashKey,
                                                                String partitionKey,
                                                                ByteBuffer data)
                                                         throws com.amazonaws.services.kinesis.model.ResourceNotFoundException,
                                                                com.amazonaws.services.kinesis.model.InvalidArgumentException

Specified by:

put in interface IKinesisProxy

Parameters:

exclusiveMinimumSequenceNumber - (optional) used for record ordering

explicitHashKey - optionally supplied transformation of partitionkey

partitionKey - for this record

data - payload

Returns:

PutRecordResult (contains the Kinesis sequence number of the record).

Throws:

com.amazonaws.services.kinesis.model.ResourceNotFoundException - The Kinesis stream was not found.

com.amazonaws.services.kinesis.model.InvalidArgumentException - InvalidArgumentException.