com.staros.schedule

Class ShardSchedulerV2

java.lang.Object

com.staros.util.AbstractServer

com.staros.schedule.ShardSchedulerV2

All Implemented Interfaces:

Scheduler, Server

public class ShardSchedulerV2
extends AbstractServer
implements Scheduler

Designed to be multi-services shared scheduler Assumptions: Uniqueness: * serviceId + shardId, globally unique * shardGroupId, globally unique * workerGroupId, globally unique * workerId, globally unique Two phases scheduling: - phase 1: calculation * select worker as target to add/remove shard replica - phase 2: dispatch * send RPC call to worker to make the change of the adding/removing shard replica, and update shard info upon success. Some of the calls can skip phase 1 and directly go to phase 2, e.g. to re-add replica back to worker if the worker is restarted. TODO: handle following cases 1. handle shard migration 2. handle workerGroup balance (could be caused by worker online, or triggered by monitoring system) 3. thread safety refactor of serviceManager/shardManager/workerManager components for multi-threads scheduling

Field Summary

Fields inherited from class com.staros.util.AbstractServer

running

Constructor Summary

Constructors

Constructor and Description
ShardSchedulerV2(ServiceManager serviceManager, WorkerManager workerManager)

Method Summary

Modifier and Type	Method and Description
void	doStart()
void	doStop()
boolean	isIdle()
void	scheduleAddToDefaultGroup(java.lang.String serviceId, java.util.List<java.lang.Long> shardIds) Request to schedule a list of shards in service (serviceId), let schedule to choose the default worker group.
void	scheduleAddToGroup(java.lang.String serviceId, java.util.List<java.lang.Long> shardIds, long wgId) BLOCKING interface of scheduling a list of shards to a WorkerGroup.
void	scheduleAddToGroup(java.lang.String serviceId, long shardId, long wgId) BLOCKING interface to schedule single shard (shardId) in service (serviceId) in workerGroup (wgId).
void	scheduleAddToWorker(java.lang.String serviceId, java.util.List<java.lang.Long> shardIds, long workerId) Phase2: batch add list of shards to a worker
void	scheduleAddToWorker(java.lang.String serviceId, long shardId, long workerId) Phase2: add a replica of a shard to a worker without
void	scheduleAsyncAddToGroup(java.lang.String serviceId, long shardId, long wgId) Async schedule single shard (shardId) in service (serviceId) in wgId workerGroup (wgId), don't wait for result.
void	scheduleAsyncAddToWorker(java.lang.String serviceId, java.util.List<java.lang.Long> shardIds, long workerId) Phase2: Async interface to schedule a list of shards to the target worker, don't wait for the result.
void	scheduleAsyncRemoveFromGroup(java.lang.String serviceId, long shardId, long workerGroupId) Remove redundant replicas for the shard, running: 1.
void	scheduleAsyncRemoveFromWorker(java.lang.String serviceId, java.util.List<java.lang.Long> shardIds, long workerId) Phase2: Remove list of shards from target worker, don't wait for the result.
void	scheduleRemoveFromWorker(java.lang.String serviceId, java.util.List<java.lang.Long> shardIds, long workerId) Phase2: Remove list of shards from target worker, wait for the result back.
void	scheduleRemoveFromWorker(java.lang.String serviceId, long shardId, long workerId) Phase2: Remove single shard replica from target worker.

Methods inherited from class com.staros.util.AbstractServer

isRunning, start, stop

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ShardSchedulerV2

public ShardSchedulerV2(ServiceManager serviceManager,
                        WorkerManager workerManager)

Method Detail

scheduleAddToGroup

public void scheduleAddToGroup(java.lang.String serviceId,
                               long shardId,
                               long wgId)
                        throws StarException

BLOCKING interface to schedule single shard (shardId) in service (serviceId) in workerGroup (wgId).

Specified by:

scheduleAddToGroup in interface Scheduler

Parameters:

serviceId - service id

shardId - shard id

wgId - worker group id

Throws:

StarException - exception if fails

scheduleAddToGroup

public void scheduleAddToGroup(java.lang.String serviceId,
                               java.util.List<java.lang.Long> shardIds,
                               long wgId)
                        throws StarException

BLOCKING interface of scheduling a list of shards to a WorkerGroup. These shards are not necessarily to be in the same shard group.

Specified by:

scheduleAddToGroup in interface Scheduler

Parameters:

serviceId - shard serviceId

shardIds - shard id list to be scheduled

wgId - workerGroup id to be scheduled

Throws:

StarException - starException with ExceptionCode

scheduleAsyncAddToGroup

public void scheduleAsyncAddToGroup(java.lang.String serviceId,
                                    long shardId,
                                    long wgId)
                             throws StarException

Async schedule single shard (shardId) in service (serviceId) in wgId workerGroup (wgId), don't wait for result.

Specified by:

scheduleAsyncAddToGroup in interface Scheduler

Parameters:

serviceId - service identity

shardId - shard identity

wgId - worker group identity

Throws:

StarException - exception if fails

scheduleAddToDefaultGroup

public void scheduleAddToDefaultGroup(java.lang.String serviceId,
                                      java.util.List<java.lang.Long> shardIds)
                               throws StarException

Request to schedule a list of shards in service (serviceId), let schedule to choose the default worker group. Wait for the scheduling done.

Specified by:

scheduleAddToDefaultGroup in interface Scheduler

Parameters:

serviceId - service identity

shardIds - list of shard identities

Throws:

StarException - exception if fails

scheduleAsyncRemoveFromGroup

public void scheduleAsyncRemoveFromGroup(java.lang.String serviceId,
                                         long shardId,
                                         long workerGroupId)
                                  throws StarException

Remove redundant replicas for the shard, running: 1. with low priority 2. as background task 3. remove one replica at most each time

Specified by:

scheduleAsyncRemoveFromGroup in interface Scheduler

Parameters:

serviceId - target service id shard belongs to

shardId - shard id

workerGroupId - target worker group id

Throws:

StarException - throws star exception if the task can't be added to thread pool.

scheduleAddToWorker

public void scheduleAddToWorker(java.lang.String serviceId,
                                long shardId,
                                long workerId)
                         throws StarException

Phase2: add a replica of a shard to a worker without

Specified by:

scheduleAddToWorker in interface Scheduler

Parameters:

serviceId - service Id

shardId - shard id to be added

workerId - target worker Id to be added

Throws:

StarException - Schedule Error TODO: refactor with Priority Queue

scheduleAddToWorker

public void scheduleAddToWorker(java.lang.String serviceId,
                                java.util.List<java.lang.Long> shardIds,
                                long workerId)
                         throws StarException

Phase2: batch add list of shards to a worker

Specified by:

scheduleAddToWorker in interface Scheduler

Parameters:

serviceId - service Id

shardIds - list of shard ids to be added

workerId - target worker Id to be added

Throws:

StarException - Schedule Error

scheduleAsyncAddToWorker

public void scheduleAsyncAddToWorker(java.lang.String serviceId,
                                     java.util.List<java.lang.Long> shardIds,
                                     long workerId)
                              throws StarException

Phase2: Async interface to schedule a list of shards to the target worker, don't wait for the result.

Specified by:

scheduleAsyncAddToWorker in interface Scheduler

Parameters:

serviceId - service id

shardIds - list of shard ids

workerId - target worker id

Throws:

StarException - schedule error

scheduleRemoveFromWorker

public void scheduleRemoveFromWorker(java.lang.String serviceId,
                                     long shardId,
                                     long workerId)
                              throws StarException

Phase2: Remove single shard replica from target worker. worker should belong to the service (serviceId)

Specified by:

scheduleRemoveFromWorker in interface Scheduler

Parameters:

serviceId - shard service Id

shardId - shard id to be removed

workerId - worker id

Throws:

StarException

scheduleRemoveFromWorker

public void scheduleRemoveFromWorker(java.lang.String serviceId,
                                     java.util.List<java.lang.Long> shardIds,
                                     long workerId)
                              throws StarException

Phase2: Remove list of shards from target worker, wait for the result back.

Specified by:

scheduleRemoveFromWorker in interface Scheduler

Parameters:

serviceId - shard service Id

shardIds - list of shard ids to be removed

workerId - target worker id

Throws:

StarException - error out if the request can't be fulfilled. ExceptionCode will represent the error category.

scheduleAsyncRemoveFromWorker

public void scheduleAsyncRemoveFromWorker(java.lang.String serviceId,
                                          java.util.List<java.lang.Long> shardIds,
                                          long workerId)
                                   throws StarException

Phase2: Remove list of shards from target worker, don't wait for the result.

Specified by:

scheduleAsyncRemoveFromWorker in interface Scheduler

Parameters:

serviceId - shard service Id

shardIds - list of shard ids to be removed

workerId - target worker id

Throws:

StarException

isIdle

public boolean isIdle()

Specified by:

isIdle in interface Scheduler

doStart

public void doStart()

Specified by:

doStart in class AbstractServer

doStop

public void doStop()

Specified by:

doStop in class AbstractServer