com.amazonaws.services.kinesis.clientlibrary.lib.worker

Class KinesisClientLibLeaseCoordinator

java.lang.Object

com.amazonaws.services.kinesis.leases.impl.LeaseCoordinator<KinesisClientLease>

com.amazonaws.services.kinesis.clientlibrary.lib.worker.KinesisClientLibLeaseCoordinator

All Implemented Interfaces:

ICheckpoint

public class KinesisClientLibLeaseCoordinator
extends LeaseCoordinator<KinesisClientLease>
implements ICheckpoint

This class is used to coordinate/manage leases owned by this worker process and to get/set checkpoints.

Field Summary

Fields inherited from class com.amazonaws.services.kinesis.leases.impl.LeaseCoordinator

metricsFactory, WORKER_IDENTIFIER_METRIC

Constructor Summary

Constructors

Constructor and Description
KinesisClientLibLeaseCoordinator(ILeaseManager<KinesisClientLease> leaseManager, ILeaseTaker<KinesisClientLease> leaseTaker, ILeaseRenewer<KinesisClientLease> leaseRenewer, long leaseDurationMillis, long epsilonMillis, int maxLeasesForWorker, int maxLeasesToStealAtOneTime, IMetricsFactory metricsFactory)
KinesisClientLibLeaseCoordinator(ILeaseManager<KinesisClientLease> leaseManager, LeaseSelector<KinesisClientLease> leaseSelector, String workerIdentifier, long leaseDurationMillis, long epsilonMillis, IMetricsFactory metricsFactory)
KinesisClientLibLeaseCoordinator(ILeaseManager<KinesisClientLease> leaseManager, LeaseSelector<KinesisClientLease> leaseSelector, String workerIdentifier, long leaseDurationMillis, long epsilonMillis, int maxLeasesForWorker, int maxLeasesToStealAtOneTime, int maxLeaseRenewerThreadCount, IMetricsFactory metricsFactory)
KinesisClientLibLeaseCoordinator(ILeaseManager<KinesisClientLease> leaseManager, String workerIdentifier, long leaseDurationMillis, long epsilonMillis)
KinesisClientLibLeaseCoordinator(ILeaseManager<KinesisClientLease> leaseManager, String workerIdentifier, long leaseDurationMillis, long epsilonMillis, LeaseSelector<KinesisClientLease> leaseSelector)

Method Summary

Modifier and Type	Method and Description
static List<ShardInfo>	convertLeasesToAssignments(Collection<KinesisClientLease> leases)
static ShardInfo	convertLeaseToAssignment(KinesisClientLease lease)
ExtendedSequenceNumber	getCheckpoint(String shardId) Get the current checkpoint stored for the specified shard.
Checkpoint	getCheckpointObject(String shardId) Get the current checkpoint stored for the specified shard, which holds the sequence numbers for the checkpoint and pending checkpoint.
List<ShardInfo>	getCurrentAssignments()
ILeaseManager<KinesisClientLease>	getLeaseManager() Used to get information about leases for Kinesis shards (e.g.
void	prepareCheckpoint(String shardId, ExtendedSequenceNumber pendingCheckpointValue, String concurrencyToken) Record intent to checkpoint for a shard.
void	setCheckpoint(String shardId, ExtendedSequenceNumber checkpointValue, String concurrencyToken) Record a checkpoint for a shard (e.g.
KinesisClientLibLeaseCoordinator	withInitialLeaseTableReadCapacity(long readCapacity)
KinesisClientLibLeaseCoordinator	withInitialLeaseTableWriteCapacity(long writeCapacity)

Methods inherited from class com.amazonaws.services.kinesis.leases.impl.LeaseCoordinator

dropLease, getAssignments, getCurrentlyHeldLease, getDefaultLeaseRenewalExecutorService, getWorkerIdentifier, isRunning, runRenewer, runTaker, start, stop, stopLeaseTaker, updateLease

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

KinesisClientLibLeaseCoordinator

public KinesisClientLibLeaseCoordinator(ILeaseManager<KinesisClientLease> leaseManager,
                                        String workerIdentifier,
                                        long leaseDurationMillis,
                                        long epsilonMillis,
                                        LeaseSelector<KinesisClientLease> leaseSelector)

Parameters:

leaseManager - Lease manager which provides CRUD lease operations.

workerIdentifier - Used to identify this worker process

leaseDurationMillis - Duration of a lease in milliseconds

epsilonMillis - Delta for timing operations (e.g. checking lease expiry)

leaseSelector - Lease selector which decides which leases to take

KinesisClientLibLeaseCoordinator

public KinesisClientLibLeaseCoordinator(ILeaseManager<KinesisClientLease> leaseManager,
                                        String workerIdentifier,
                                        long leaseDurationMillis,
                                        long epsilonMillis)

Parameters:

leaseManager - Lease manager which provides CRUD lease operations.

workerIdentifier - Used to identify this worker process

leaseDurationMillis - Duration of a lease in milliseconds

epsilonMillis - Delta for timing operations (e.g. checking lease expiry)

KinesisClientLibLeaseCoordinator

public KinesisClientLibLeaseCoordinator(ILeaseManager<KinesisClientLease> leaseManager,
                                        LeaseSelector<KinesisClientLease> leaseSelector,
                                        String workerIdentifier,
                                        long leaseDurationMillis,
                                        long epsilonMillis,
                                        IMetricsFactory metricsFactory)

Parameters:

leaseManager - Lease manager which provides CRUD lease operations.

leaseSelector - Lease selector which decides which leases to take

workerIdentifier - Used to identify this worker process

leaseDurationMillis - Duration of a lease in milliseconds

epsilonMillis - Delta for timing operations (e.g. checking lease expiry)

metricsFactory - Metrics factory used to emit metrics

KinesisClientLibLeaseCoordinator

public KinesisClientLibLeaseCoordinator(ILeaseManager<KinesisClientLease> leaseManager,
                                        LeaseSelector<KinesisClientLease> leaseSelector,
                                        String workerIdentifier,
                                        long leaseDurationMillis,
                                        long epsilonMillis,
                                        int maxLeasesForWorker,
                                        int maxLeasesToStealAtOneTime,
                                        int maxLeaseRenewerThreadCount,
                                        IMetricsFactory metricsFactory)

Parameters:

leaseManager - Lease manager which provides CRUD lease operations.

leaseSelector - Lease selector which decides which leases to take

workerIdentifier - Used to identify this worker process

leaseDurationMillis - Duration of a lease in milliseconds

epsilonMillis - Delta for timing operations (e.g. checking lease expiry)

maxLeasesForWorker - Max leases this worker can handle at a time

maxLeasesToStealAtOneTime - Steal up to this many leases at a time (for load balancing)

metricsFactory - Metrics factory used to emit metrics

KinesisClientLibLeaseCoordinator

public KinesisClientLibLeaseCoordinator(ILeaseManager<KinesisClientLease> leaseManager,
                                        ILeaseTaker<KinesisClientLease> leaseTaker,
                                        ILeaseRenewer<KinesisClientLease> leaseRenewer,
                                        long leaseDurationMillis,
                                        long epsilonMillis,
                                        int maxLeasesForWorker,
                                        int maxLeasesToStealAtOneTime,
                                        IMetricsFactory metricsFactory)

Method Detail

withInitialLeaseTableReadCapacity

public KinesisClientLibLeaseCoordinator withInitialLeaseTableReadCapacity(long readCapacity)

Parameters:

readCapacity - The DynamoDB table used for tracking leases will be provisioned with the specified initial read capacity

Returns:

KinesisClientLibLeaseCoordinator

withInitialLeaseTableWriteCapacity

public KinesisClientLibLeaseCoordinator withInitialLeaseTableWriteCapacity(long writeCapacity)

Parameters:

writeCapacity - The DynamoDB table used for tracking leases will be provisioned with the specified initial write capacity

Returns:

KinesisClientLibLeaseCoordinator

setCheckpoint

public void setCheckpoint(String shardId,
                          ExtendedSequenceNumber checkpointValue,
                          String concurrencyToken)
                   throws KinesisClientLibException

Record a checkpoint for a shard (e.g. sequence and subsequence numbers of last record processed by application). Upon failover, record processing is resumed from this point.

Specified by:

setCheckpoint in interface ICheckpoint

Parameters:

shardId - Checkpoint is specified for this shard.

checkpointValue - Value of the checkpoint (e.g. Kinesis sequence number and subsequence number)

concurrencyToken - Used with conditional writes to prevent stale updates (e.g. if there was a fail over to a different record processor, we don't want to overwrite it's checkpoint)

Throws:

KinesisClientLibException - Thrown if we were unable to save the checkpoint

getCheckpoint

public ExtendedSequenceNumber getCheckpoint(String shardId)
                                     throws KinesisClientLibException

Get the current checkpoint stored for the specified shard. Useful for checking that the parent shard has been completely processed before we start processing the child shard.

Specified by:

getCheckpoint in interface ICheckpoint

Parameters:

shardId - Current checkpoint for this shard is fetched

Returns:

Current checkpoint for this shard, null if there is no record for this shard.

Throws:

KinesisClientLibException - Thrown if we are unable to fetch the checkpoint

prepareCheckpoint

public void prepareCheckpoint(String shardId,
                              ExtendedSequenceNumber pendingCheckpointValue,
                              String concurrencyToken)
                       throws KinesisClientLibException

Record intent to checkpoint for a shard. Upon failover, the pendingCheckpointValue will be passed to the new RecordProcessor's initialize() method.

Specified by:

prepareCheckpoint in interface ICheckpoint

Parameters:

shardId - Checkpoint is specified for this shard.

pendingCheckpointValue - Value of the pending checkpoint (e.g. Kinesis sequence number and subsequence number)

concurrencyToken - Used with conditional writes to prevent stale updates (e.g. if there was a fail over to a different record processor, we don't want to overwrite it's checkpoint)

Throws:

KinesisClientLibException - Thrown if we were unable to save the checkpoint

getCheckpointObject

public Checkpoint getCheckpointObject(String shardId)
                               throws KinesisClientLibException

Get the current checkpoint stored for the specified shard, which holds the sequence numbers for the checkpoint and pending checkpoint. Useful for checking that the parent shard has been completely processed before we start processing the child shard.

Specified by:

getCheckpointObject in interface ICheckpoint

Parameters:

shardId - Current checkpoint for this shard is fetched

Returns:

Current checkpoint object for this shard, null if there is no record for this shard.

Throws:

KinesisClientLibException - Thrown if we are unable to fetch the checkpoint

getCurrentAssignments

public List<ShardInfo> getCurrentAssignments()

Returns:

Current shard/lease assignments

convertLeasesToAssignments

public static List<ShardInfo> convertLeasesToAssignments(Collection<KinesisClientLease> leases)

convertLeaseToAssignment

public static ShardInfo convertLeaseToAssignment(KinesisClientLease lease)

getLeaseManager

public ILeaseManager<KinesisClientLease> getLeaseManager()

Used to get information about leases for Kinesis shards (e.g. sync shards and leases, check on parent shard completion).

Returns:

LeaseManager