software.amazon.kinesis.coordinator

Class Scheduler

java.lang.Object

software.amazon.kinesis.coordinator.Scheduler

All Implemented Interfaces:

Runnable

public class Scheduler
extends Object
implements Runnable

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	gracefuleShutdownStarted

Constructor Summary

Constructors

Modifier	Constructor and Description
	Scheduler(@NonNull CheckpointConfig checkpointConfig, @NonNull CoordinatorConfig coordinatorConfig, @NonNull LeaseManagementConfig leaseManagementConfig, @NonNull LifecycleConfig lifecycleConfig, @NonNull MetricsConfig metricsConfig, @NonNull ProcessorConfig processorConfig, @NonNull RetrievalConfig retrievalConfig)
protected	Scheduler(@NonNull CheckpointConfig checkpointConfig, @NonNull CoordinatorConfig coordinatorConfig, @NonNull LeaseManagementConfig leaseManagementConfig, @NonNull LifecycleConfig lifecycleConfig, @NonNull MetricsConfig metricsConfig, @NonNull ProcessorConfig processorConfig, @NonNull RetrievalConfig retrievalConfig, @NonNull software.amazon.kinesis.coordinator.DiagnosticEventFactory diagnosticEventFactory) Customers do not currently have the ability to customize the DiagnosticEventFactory, but this visibility is desired for testing.

Method Summary

Modifier and Type	Method and Description
protected ShardConsumer	buildConsumer(@NonNull ShardInfo shardInfo, @NonNull ShardRecordProcessorFactory shardRecordProcessorFactory, @NonNull LeaseCleanupManager leaseCleanupManager)
Callable<Boolean>	createGracefulShutdownCallable() Creates a callable that will execute the graceful shutdown process.
boolean	hasGracefulShutdownStarted()
Future<Void>	requestShutdown() Deprecated.
void	run() Start consuming data from the stream, and pass it to the application record processors.
void	shutdown() Signals worker to shutdown.
CompletableFuture<Boolean>	startGracefulShutdown() Requests a graceful shutdown of the worker, notifying record processors, that implement ShutdownNotificationAware, of the impending shutdown.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

gracefuleShutdownStarted

protected boolean gracefuleShutdownStarted

Constructor Detail

Scheduler

public Scheduler(@NonNull
                 @NonNull CheckpointConfig checkpointConfig,
                 @NonNull
                 @NonNull CoordinatorConfig coordinatorConfig,
                 @NonNull
                 @NonNull LeaseManagementConfig leaseManagementConfig,
                 @NonNull
                 @NonNull LifecycleConfig lifecycleConfig,
                 @NonNull
                 @NonNull MetricsConfig metricsConfig,
                 @NonNull
                 @NonNull ProcessorConfig processorConfig,
                 @NonNull
                 @NonNull RetrievalConfig retrievalConfig)

Scheduler

protected Scheduler(@NonNull
                    @NonNull CheckpointConfig checkpointConfig,
                    @NonNull
                    @NonNull CoordinatorConfig coordinatorConfig,
                    @NonNull
                    @NonNull LeaseManagementConfig leaseManagementConfig,
                    @NonNull
                    @NonNull LifecycleConfig lifecycleConfig,
                    @NonNull
                    @NonNull MetricsConfig metricsConfig,
                    @NonNull
                    @NonNull ProcessorConfig processorConfig,
                    @NonNull
                    @NonNull RetrievalConfig retrievalConfig,
                    @NonNull
                    @NonNull software.amazon.kinesis.coordinator.DiagnosticEventFactory diagnosticEventFactory)

Customers do not currently have the ability to customize the DiagnosticEventFactory, but this visibility is desired for testing. This constructor is only used for testing to provide a mock DiagnosticEventFactory.

Method Detail

run

public void run()

Start consuming data from the stream, and pass it to the application record processors.

Specified by:

run in interface Runnable

startGracefulShutdown

public CompletableFuture<Boolean> startGracefulShutdown()

Requests a graceful shutdown of the worker, notifying record processors, that implement ShutdownNotificationAware, of the impending shutdown. This gives the record processor a final chance to checkpoint. This will only create a single shutdown future. Additional attempts to start a graceful shutdown will return the previous future. It's possible that a record processor won't be notify before being shutdown. This can occur if the lease is lost after requesting shutdown, but before the notification is dispatched.

Requested Shutdown Process

When a shutdown process is requested it operates slightly differently to allow the record processors a chance to checkpoint a final time.

1. Call to request shutdown invoked.
2. Worker stops attempting to acquire new leases
3. Record Processor Shutdown Begins
   1. Record processor is notified of the impending shutdown, and given a final chance to checkpoint
   2. The lease for the record processor is then dropped.
   3. The record processor enters into an idle state waiting for the worker to complete final termination
   4. The worker will detect a record processor that has lost it's lease, and will terminate the record processor with ShutdownReason.LEASE_LOST
4. The worker will shutdown all record processors.
5. Once all record processors have been terminated, the worker will terminate all owned resources.
6. Once the worker shutdown is complete, the returned future is completed.

Returns:

a future that will be set once the shutdown has completed. True indicates that the graceful shutdown completed successfully. A false value indicates that a non-exception case caused the shutdown process to terminate early.

createGracefulShutdownCallable

public Callable<Boolean> createGracefulShutdownCallable()

Creates a callable that will execute the graceful shutdown process. This callable can be used to execute graceful shutdowns in your own executor, or execute the shutdown synchronously.

Returns:

a callable that run the graceful shutdown process. This may return a callable that return true if the graceful shutdown has already been completed.

Throws:

IllegalStateException - thrown by the callable if another callable has already started the shutdown process.

hasGracefulShutdownStarted

public boolean hasGracefulShutdownStarted()

shutdown

public void shutdown()

Signals worker to shutdown. Worker will try initiating shutdown of all record processors. Note that if executor services were passed to the worker by the user, worker will not attempt to shutdown those resources.

Shutdown Process

When called this will start shutdown of the record processor, and eventually shutdown the worker itself.

1. Call to start shutdown invoked
2. Lease coordinator told to stop taking leases, and to drop existing leases.
3. Worker discovers record processors that no longer have leases.
4. Worker triggers shutdown with state ShutdownReason.LEASE_LOST.
5. Once all record processors are shutdown, worker terminates owned resources.
6. Shutdown complete.

buildConsumer

protected ShardConsumer buildConsumer(@NonNull
                                      @NonNull ShardInfo shardInfo,
                                      @NonNull
                                      @NonNull ShardRecordProcessorFactory shardRecordProcessorFactory,
                                      @NonNull
                                      @NonNull LeaseCleanupManager leaseCleanupManager)

requestShutdown

@Deprecated
public Future<Void> requestShutdown()

Deprecated.