| Modifier and Type | Class and Description |
|---|---|
class |
DoesNothingPreparedCheckpointer
A special PreparedCheckpointer that does nothing, which can be used when preparing a checkpoint at the current
checkpoint sequence number where it is never necessary to do another checkpoint.
|
| Modifier and Type | Class and Description |
|---|---|
class |
DynamoDBCheckpointer |
class |
DynamoDBCheckpointFactory |
| Modifier and Type | Class and Description |
|---|---|
class |
ArnUtil |
class |
DiagnosticUtils |
class |
KinesisRequestsBuilder |
| Modifier and Type | Class and Description |
|---|---|
class |
CoordinatorState
DataModel for CoordinatorState, this data model is used to store various state information required
for coordination across the KCL worker fleet.
|
class |
CoordinatorStateDAO
Data Access Object to abstract accessing
CoordinatorState from
the CoordinatorState DDB table. |
class |
DynamicMigrationComponentsInitializer
This class is responsible for initializing the KCL components that supports
seamless upgrade from v2.x to v3.x.
|
class |
DynamoDbAsyncToSyncClientAdapter
DDB Lock client depends on DynamoDbClient and KCL only has DynamoDbAsyncClient configured.
|
class |
ExecutorStateEvent |
class |
MigrationAdaptiveLeaseAssignmentModeProvider
Provides the lease assignment mode KCL must operate in during migration
from 2.x to 3.x.
|
class |
Scheduler |
class |
SchedulerCoordinatorFactory |
| Modifier and Type | Interface and Description |
|---|---|
interface |
LeaseAssignmentDecider |
class |
LeaseAssignmentManager
Performs the LeaseAssignment for the application.
|
class |
VarianceBasedLeaseAssignmentDecider
VarianceBasedLeaseAssignmentDecider
This implementation of LeaseAssignmentDecider performs lease assignment by considering the WorkerMetricStats values of workers
with respect to fleet level average of that WorkerMetricStats.
|
| Modifier and Type | Class and Description |
|---|---|
class |
ClientVersionChangeMonitor
Change monitor for MigrationState.clientVersion to notify a callback if the value
changes from a given value.
|
class |
MigrationClientVersion2xState
State for CLIENT_VERSION_2X.
|
class |
MigrationClientVersion3xState
State for CLIENT_VERSION_3X which enables KCL to run 3.x algorithms on new KCLv3.x application
or successfully upgraded application which upgraded from v2.x.
|
class |
MigrationClientVersion3xWithRollbackState
State for CLIENT_VERSION_3X_WITH_ROLLBACK which enables KCL to run its 3.x compliant algorithms
during the upgrade process after all KCL workers in the fleet are 3.x complaint.
|
interface |
MigrationClientVersionState
Interface of a state implementation for the MigrationStateMachine
|
class |
MigrationClientVersionStateInitializer
Initializer to determine start state of the state machine which identifies the
state to initialize KCL when it is starting up.
|
class |
MigrationClientVersionUpgradeFrom2xState
State for CLIENT_VERSION_UPGRADE_FROM_2X.
|
class |
MigrationReadyMonitor
Monitor for KCL workers 3.x readiness.
|
class |
MigrationState
Data model of the Migration state.
|
interface |
MigrationStateMachine
State machine that provides:
1.
|
class |
MigrationStateMachineImpl
Implementation of
MigrationStateMachine |
| Modifier and Type | Class and Description |
|---|---|
class |
DynamoDBLockBasedLeaderDecider
Implementation for LeaderDecider to elect leader using lock on dynamo db table.
|
class |
MigrationAdaptiveLeaderDecider
MigrationAdaptiveLeaderDecider that wraps around the actual LeaderDecider which can dynamically
change based on the MigrationStateMachine.
|
| Modifier and Type | Class and Description |
|---|---|
class |
DynamoUtils
Static utility functions used by our LeaseSerializers.
|
class |
HierarchicalShardSyncer
Helper class to sync leases with shards of the Kinesis stream.
|
class |
KinesisShardDetector |
interface |
LeaseDiscoverer |
class |
LeaseStatsRecorder
This class records the stats for the leases.
|
static class |
LeaseStatsRecorder.LeaseStats |
class |
ShardSyncTask
This task syncs leases/activies with shards of the stream.
|
| Modifier and Type | Class and Description |
|---|---|
class |
DynamoDBLeaseCoordinator
LeaseCoordinator abstracts away LeaseTaker and LeaseRenewer from the application code that's using leasing.
|
class |
DynamoDBLeaseDiscoverer
An implementation of
LeaseDiscoverer, it uses LeaseRefresher to query
DynamoDBLeaseRefresher.LEASE_OWNER_TO_LEASE_KEY_INDEX_NAME and find the leases assigned
to current worker and then filter and returns the leases that have not started processing (looks at
LeaseRenewer.getCurrentlyHeldLeases() to find out which leases are currently held leases). |
class |
DynamoDBLeaseManagementFactory |
class |
DynamoDBLeaseRefresher
An implementation of
LeaseRefresher that uses DynamoDB. |
class |
DynamoDBLeaseRenewer
An implementation of
LeaseRenewer that uses DynamoDB via LeaseRefresher. |
class |
DynamoDBLeaseSerializer
An implementation of ILeaseSerializer for basic Lease objects.
|
class |
DynamoDBLeaseTaker
An implementation of
LeaseTaker that uses DynamoDB via LeaseRefresher. |
| Modifier and Type | Class and Description |
|---|---|
class |
BlockOnParentShardTask
Task to block until processing of all data records in the parent shard(s) is completed.
|
interface |
ConsumerTaskFactory |
class |
InitializeTask
Task for initializing shard position and invoking the ShardRecordProcessor initialize() API.
|
class |
KinesisConsumerTaskFactory |
class |
LeaseGracefulShutdownHandler
This class handles the graceful shutdown of shard consumers.
|
class |
ProcessTask
Task for fetching data records and invoking processRecords() on the record processor instance.
|
class |
ShardConsumer
Responsible for consuming data records of a (specified) shard.
|
class |
ShardConsumerArgument |
class |
ShardConsumerNotifyingSubscriber |
class |
ShardConsumerShutdownNotification
Contains callbacks for completion of stages in a requested record processor shutdown.
|
class |
ShutdownNotificationTask
Notifies record processor of incoming shutdown request, and gives them a chance to checkpoint.
|
class |
ShutdownTask
Task for invoking the ShardRecordProcessor shutdown() callback.
|
| Modifier and Type | Class and Description |
|---|---|
class |
MetricsCollectingTaskDecorator
Decorates an ConsumerTask and reports metrics about its timing and success/failure.
|
| Modifier and Type | Class and Description |
|---|---|
class |
AWSExceptionManager
Traverses a
Throwable class inheritance in search of a mapping
function which will convert that throwable into a RuntimeException. |
class |
BatchUniqueIdentifier |
interface |
GetRecordsResponseAdapter |
class |
IteratorBuilder |
class |
KinesisGetRecordsResponseAdapter |
| Modifier and Type | Class and Description |
|---|---|
class |
FanOutConsumerRegistration |
class |
FanOutRecordsPublisher |
class |
FanOutRetrievalFactory |
| Modifier and Type | Class and Description |
|---|---|
class |
AsynchronousGetRecordsRetrievalStrategy |
class |
BlockingRecordsPublisher
This is the BlockingRecordsPublisher class.
|
class |
KinesisDataFetcher
Used to get data from Amazon Kinesis.
|
class |
KinesisSleepTimeController |
class |
PrefetchRecordsPublisher
This is the prefetch caching class, this class spins up a thread if prefetching is enabled.
|
class |
SimpleRecordsFetcherFactory |
interface |
SleepTimeController |
class |
SleepTimeControllerConfig
Configuration for the
SleepTimeController. |
class |
SynchronousBlockingRetrievalFactory |
class |
SynchronousGetRecordsRetrievalStrategy |
| Modifier and Type | Class and Description |
|---|---|
class |
Cgroup |
class |
DdbUtil |
class |
ExponentialMovingAverage
|
class |
Statistics |
| Modifier and Type | Class and Description |
|---|---|
class |
WorkerMetricsSelector
Class to select appropriate WorkerMetricStats based on the operating range provider that is available on the instance.
|
| Modifier and Type | Class and Description |
|---|---|
class |
Cgroupv1CpuWorkerMetric
Utilizes Linux Control Groups by reading cpu time and available cpu from cgroup directory.This works for Elastic
Kubernetes Service (EKS) containers running on Linux instances which use cgroupv1.
|
class |
Cgroupv2CpuWorkerMetric
Utilizes Linux Control Groups by reading cpu time and available cpu from cgroup directory.
|
class |
EcsCpuWorkerMetric
Queries the Amazon ECS task metadata endpoint version 4 to get CPU metric stats as well as allocated CPU to the ECS task and
containers to calculate percent CPU utilization.
|
| Modifier and Type | Class and Description |
|---|---|
class |
HeapMemoryAfterGCWorkerMetric
Memory WorkerMetricStats that reads the heap memory after GC.
|
| Modifier and Type | Class and Description |
|---|---|
class |
LinuxCpuWorkerMetric
Reads CPU usage statistics out of /proc/stat file that is present on the EC2 instances.
|
class |
LinuxNetworkInWorkerMetric
Ref java doc for
LinuxNetworkWorkerMetricBase |
class |
LinuxNetworkOutWorkerMetric
Ref java doc for
LinuxNetworkWorkerMetricBase |
class |
LinuxNetworkWorkerMetricBase
Base class for EC2NetworkWorkerMetrics, this reads and parses /proc/net/dev file and look for the specific
interface and reads received and transmitted bytes.
|
| Modifier and Type | Class and Description |
|---|---|
class |
WorkerMetricStats
DataModel for a WorkerMetric, this data model is used to store the current state of a Worker in terms of relevant
WorkerMetric(CPU, Memory, Network).
|
class |
WorkerMetricStatsDAO |
class |
WorkerMetricStatsManager
WorkerMetricStatsManager is a class that manages the collection of raw WorkerMetricStats values for the list of WorkerMetricStats
periodically and store in a bounded in-memory queue.
|
class |
WorkerMetricStatsReporter
Reporter that is periodically executed to report WorkerMetricStats.
|
| Modifier and Type | Class and Description |
|---|---|
class |
Ec2Resource
Provides resource metadata for EC2.
|
class |
EcsResource
Provides resource metadata for ECS.
|
class |
EksResource
Provides resource metadata for EKS.
|
class |
OperatingRangeDataProvider
Enum representing the different operating range metadata providers.
|
interface |
ResourceMetadataProvider
Interface for providing resource metadata for worker.
|
Copyright © 2025. All rights reserved.