net.greghaines.jesque.worker

Class WorkerImpl

java.lang.Object

net.greghaines.jesque.worker.WorkerImpl

All Implemented Interfaces:

Runnable, JobExecutor, Worker, WorkerEventEmitter

Direct Known Subclasses:

WorkerExitOnEmpty

public class WorkerImpl
extends Object
implements Worker

Basic implementation of the Worker interface. Obeys the contract of a Resque worker in Redis.

Author:

Greg Haines

Nested Class Summary

Nested classes/interfaces inherited from interface net.greghaines.jesque.worker.JobExecutor

JobExecutor.State

Field Summary

Fields

Modifier and Type	Field and Description
protected Config	config
protected static long	emptyQueueSleepTime
protected redis.clients.jedis.Jedis	jedis
protected WorkerListenerDelegate	listenerDelegate
protected String	namespace
protected BlockingDeque<String>	queueNames
protected static int	reconnectAttempts
protected static long	reconnectSleepTime
protected AtomicReference<JobExecutor.State>	state

Fields inherited from interface net.greghaines.jesque.worker.Worker

ALL_QUEUES

Constructor Summary

Constructors

Constructor and Description
WorkerImpl(Config config, Collection<String> queues, Map<String,? extends Class<?>> jobTypes) Creates a new WorkerImpl, which creates it's own connection to Redis using values from the config.

Method Summary

Methods

Modifier and Type	Method and Description
void	addJobType(String jobName, Class<?> jobType) Allow the given job type to be executed.
void	addListener(WorkerListener listener) Register a WorkerListener for all WorkerEvents.
void	addListener(WorkerListener listener, WorkerEvent... events) Register a WorkerListener for the specified WorkerEvents.
void	addQueue(String queueName) Poll the given queue.
protected void	checkJobType(String jobName, Class<?> jobType) Determine if a job name and job type are valid.
protected void	checkJobTypes(Map<String,? extends Class<?>> jobTypes) Verify the given job types are all valid.
protected void	checkPaused() Checks to see if worker is paused.
protected static void	checkQueues(Iterable<String> queues) Verify that the given queues are all valid.
protected String	createName() Creates a unique name, suitable for use with Resque.
void	end(boolean now) Shutdown this Worker. The worker cannot be started again; create a new worker in this case.
protected Object	execute(Job job, String curQueue, Object instance) Executes the given job.
protected String	failMsg(Exception ex, String queue, Job job) Create and serialize a JobFailure.
protected void	failure(Exception ex, Job job, String curQueue) Update the status in Redis on failure
ExceptionHandler	getExceptionHandler() The current exception handler.
Map<String,Class<?>>	getJobTypes() The allowed job names and types that this JobExecutor will execute.
String	getName() Returns the name of this Worker.
Collection<String>	getQueues() The queues that this Worker will poll.
protected int	getReconnectAttempts()
long	getWorkerId()
boolean	isPaused() Returns whether this worker is paused.
boolean	isProcessingJob() Returns whether this JobExecutor is currently processing a job.
boolean	isShutdown() Returns whether this JobExecutor is either shutdown or in the process of shutting down.
static boolean	isThreadNameChangingEnabled()
void	join(long millis) Wait for this JobExecutor to complete.
protected String	key(String... parts) Builds a namespaced Redis key with the given arguments.
protected String	pauseMsg() Create and serialize a WorkerStatus for a pause event.
protected void	poll() Polls the queues for jobs and executes them.
protected void	process(Job job, String curQueue) Materializes and executes the given job.
protected void	recoverFromException(String curQueue, Exception e) Handle an exception that was thrown from inside poll()
void	removeAllListeners() Unregister all WorkerListeners for all WorkerEvents.
void	removeAllListeners(WorkerEvent... events) Unregister all WorkerListeners for the specified WorkerEvents.
void	removeAllQueues() Stop polling all queues.
void	removeJobName(String jobName) Disallow the job name from being executed.
void	removeJobType(Class<?> jobType) Disallow the job type from being executed.
void	removeListener(WorkerListener listener) Unregister a WorkerListener for all WorkerEvents.
void	removeListener(WorkerListener listener, WorkerEvent... events) Unregister a WorkerListener for the specified WorkerEvents.
void	removeQueue(String queueName, boolean all) Stop polling the given queue.
protected void	renameThread(String msg) Rename the current thread with the given message.
void	run() Starts this worker.
void	setExceptionHandler(ExceptionHandler exceptionHandler) Set this JobExecutor's exception handler to the given handler.
void	setJobTypes(Map<String,? extends Class<?>> jobTypes) Clear any current allowed job types and use the given set.
void	setQueues(Collection<String> queues) Clear any current queues and poll the given queues.
static void	setThreadNameChangingEnabled(boolean enabled) Enable/disable worker thread renaming during normal operation.
protected String	statusMsg(String queue, Job job) Create and serialize a WorkerStatus.
protected void	success(Job job, Object runner, Object result, String curQueue) Update the status in Redis on success.
void	togglePause(boolean paused) Toggle whether this worker will process any new jobs.
String	toString()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

emptyQueueSleepTime

protected static final long emptyQueueSleepTime

See Also:

Constant Field Values

reconnectSleepTime

protected static final long reconnectSleepTime

See Also:

Constant Field Values

reconnectAttempts

protected static final int reconnectAttempts

See Also:

Constant Field Values

config

protected final Config config

jedis

protected final redis.clients.jedis.Jedis jedis

namespace

protected final String namespace

queueNames

protected final BlockingDeque<String> queueNames

listenerDelegate

protected final WorkerListenerDelegate listenerDelegate

state

protected final AtomicReference<JobExecutor.State> state

Constructor Detail

WorkerImpl

public WorkerImpl(Config config,
          Collection<String> queues,
          Map<String,? extends Class<?>> jobTypes)

Creates a new WorkerImpl, which creates it's own connection to Redis using values from the config. The worker will only listen to the supplied queues and only execute jobs that are in the supplied job types.

Parameters:

config - used to create a connection to Redis and the package prefix for incoming jobs

queues - the list of queues to poll

jobTypes - the map of job names and types to execute

Throws:

IllegalArgumentException - if the config is null, if the queues is null, or if the jobTypes is null or empty

Method Detail

isThreadNameChangingEnabled

public static boolean isThreadNameChangingEnabled()

Returns:

true if worker threads names will change during normal operation

setThreadNameChangingEnabled

public static void setThreadNameChangingEnabled(boolean enabled)

Enable/disable worker thread renaming during normal operation. (Disabled by default)

Warning: Enabling this feature is very expensive CPU-wise!
This feature is designed to assist in debugging worker state but should be disabled in production environments for performance reasons.

Parameters:

enabled - whether threads' names should change during normal operation

checkQueues

protected static void checkQueues(Iterable<String> queues)

Verify that the given queues are all valid.

Parameters:

queues - the given queues

getWorkerId

public long getWorkerId()

Returns:

this worker's identifier

run

public void run()

Starts this worker. Registers the worker in Redis and begins polling the queues for jobs. Stop this worker by calling end() on any thread.

Specified by:

run in interface Runnable

end

public void end(boolean now)

Shutdown this Worker.
The worker cannot be started again; create a new worker in this case.

Specified by:

end in interface JobExecutor

Parameters:

now - if true, an effort will be made to stop any job in progress

isShutdown

public boolean isShutdown()

Description copied from interface: JobExecutor

Returns whether this JobExecutor is either shutdown or in the process of shutting down.

Specified by:

isShutdown in interface JobExecutor

Returns:

true if this JobExecutor is either shutdown or in the process of shutting down

isPaused

public boolean isPaused()

Description copied from interface: Worker

Returns whether this worker is paused.

Specified by:

isPaused in interface Worker

Returns:

whether this worker is paused. If true, the worker is not processing any new jobs; if false, the worker is processing new jobs

isProcessingJob

public boolean isProcessingJob()

Description copied from interface: JobExecutor

Returns whether this JobExecutor is currently processing a job.

Specified by:

isProcessingJob in interface JobExecutor

Returns:

true if this JobExecutor is currently processing a job

togglePause

public void togglePause(boolean paused)

Description copied from interface: Worker

Toggle whether this worker will process any new jobs.

Specified by:

togglePause in interface Worker

Parameters:

paused - if true, the worker will not process any new jobs; if false, the worker will process new jobs

getName

public String getName()

Description copied from interface: Worker

Returns the name of this Worker.

Specified by:

getName in interface Worker

Returns:

the name of this Worker

addListener

public void addListener(WorkerListener listener)

Description copied from interface: WorkerEventEmitter

Register a WorkerListener for all WorkerEvents.

Specified by:

addListener in interface WorkerEventEmitter

Parameters:

listener - the WorkerListener to register

addListener

public void addListener(WorkerListener listener,
               WorkerEvent... events)

Description copied from interface: WorkerEventEmitter

Register a WorkerListener for the specified WorkerEvents.

Specified by:

addListener in interface WorkerEventEmitter

Parameters:

listener - the WorkerListener to register

events - the WorkerEvents to be notified of

removeListener

public void removeListener(WorkerListener listener)

Description copied from interface: WorkerEventEmitter

Unregister a WorkerListener for all WorkerEvents.

Specified by:

removeListener in interface WorkerEventEmitter

Parameters:

listener - the WorkerListener to unregister

removeListener

public void removeListener(WorkerListener listener,
                  WorkerEvent... events)

Description copied from interface: WorkerEventEmitter

Unregister a WorkerListener for the specified WorkerEvents.

Specified by:

removeListener in interface WorkerEventEmitter

Parameters:

listener - the WorkerListener to unregister

events - the WorkerEvents to no longer be notified of

removeAllListeners

public void removeAllListeners()

Description copied from interface: WorkerEventEmitter

Unregister all WorkerListeners for all WorkerEvents.

Specified by:

removeAllListeners in interface WorkerEventEmitter

removeAllListeners

public void removeAllListeners(WorkerEvent... events)

Description copied from interface: WorkerEventEmitter

Unregister all WorkerListeners for the specified WorkerEvents.

Specified by:

removeAllListeners in interface WorkerEventEmitter

Parameters:

events - the WorkerEvents to no longer be notified of

getQueues

public Collection<String> getQueues()

Description copied from interface: Worker

The queues that this Worker will poll.

Specified by:

getQueues in interface Worker

Returns:

an unmodifiable view of the queues to be polled

addQueue

public void addQueue(String queueName)

Description copied from interface: Worker

Poll the given queue. If the queue exists multiple times, it will be checked that many times per loop. This allows for a queue to be given higher priority by checking it more often.

Specified by:

addQueue in interface Worker

Parameters:

queueName - the name of the queue to poll

removeQueue

public void removeQueue(String queueName,
               boolean all)

Description copied from interface: Worker

Stop polling the given queue. If the all argument is true, all instances of the queue will be removed, otherwise, only one instance is removed.

Specified by:

removeQueue in interface Worker

Parameters:

queueName - the queue to stop polling

all - whether to remove all or only one of the instances

removeAllQueues

public void removeAllQueues()

Description copied from interface: Worker

Stop polling all queues.

Specified by:

removeAllQueues in interface Worker

setQueues

public void setQueues(Collection<String> queues)

Description copied from interface: Worker

Clear any current queues and poll the given queues.

Specified by:

setQueues in interface Worker

Parameters:

queues - the queues to poll

getJobTypes

public Map<String,Class<?>> getJobTypes()

Description copied from interface: JobExecutor

The allowed job names and types that this JobExecutor will execute.

Specified by:

getJobTypes in interface JobExecutor

Returns:

an unmodifiable view of the allowed job names and types

addJobType

public void addJobType(String jobName,
              Class<?> jobType)

Description copied from interface: JobExecutor

Allow the given job type to be executed.

Specified by:

addJobType in interface JobExecutor

Parameters:

jobName - the job name as seen

jobType - the job type to allow

removeJobType

public void removeJobType(Class<?> jobType)

Description copied from interface: JobExecutor

Disallow the job type from being executed.

Specified by:

removeJobType in interface JobExecutor

Parameters:

jobType - the job type to disallow

removeJobName

public void removeJobName(String jobName)

Description copied from interface: JobExecutor

Disallow the job name from being executed.

Specified by:

removeJobName in interface JobExecutor

Parameters:

jobName - the job name to disallow

setJobTypes

public void setJobTypes(Map<String,? extends Class<?>> jobTypes)

Description copied from interface: JobExecutor

Clear any current allowed job types and use the given set.

Specified by:

setJobTypes in interface JobExecutor

Parameters:

jobTypes - the job types to allow

getExceptionHandler

public ExceptionHandler getExceptionHandler()

Description copied from interface: JobExecutor

The current exception handler.

Specified by:

getExceptionHandler in interface JobExecutor

Returns:

the current exception handler.

setExceptionHandler

public void setExceptionHandler(ExceptionHandler exceptionHandler)

Description copied from interface: JobExecutor

Set this JobExecutor's exception handler to the given handler.

Specified by:

setExceptionHandler in interface JobExecutor

Parameters:

exceptionHandler - the exception handler to use

join

public void join(long millis)
          throws InterruptedException

Description copied from interface: JobExecutor

Wait for this JobExecutor to complete. A timeout of 0 means to wait forever.

This method will only return after a thread has called JobExecutor.end(boolean).

Specified by:

join in interface JobExecutor

Parameters:

millis - the time to wait in milliseconds

Throws:

InterruptedException - if any thread has interrupted the current thread

getReconnectAttempts

protected int getReconnectAttempts()

Returns:

the number of times this Worker will attempt to reconnect to Redis before giving up

poll

protected void poll()

Polls the queues for jobs and executes them.

recoverFromException

protected void recoverFromException(String curQueue,
                        Exception e)

Handle an exception that was thrown from inside poll()

Parameters:

curQueue - the name of the queue that was being processed when the exception was thrown

e - the exception that was thrown

checkPaused

protected void checkPaused()
                    throws IOException

Checks to see if worker is paused. If so, wait until unpaused.

Throws:

IOException - if there was an error creating the pause message

process

protected void process(Job job,
           String curQueue)

Materializes and executes the given job.

Parameters:

job - the Job to process

curQueue - the queue the payload came from

execute

protected Object execute(Job job,
             String curQueue,
             Object instance)
                  throws Exception

Executes the given job.

Parameters:

job - the job to execute

curQueue - the queue the job came from

instance - the materialized job

Returns:

result of the execution

Throws:

Exception - if the instance is a Callable and throws an exception

success

protected void success(Job job,
           Object runner,
           Object result,
           String curQueue)

Update the status in Redis on success.

Parameters:

job - the Job that succeeded

runner - the materialized Job

curQueue - the queue the Job came from

failure

protected void failure(Exception ex,
           Job job,
           String curQueue)

Update the status in Redis on failure

Parameters:

ex - the Exception that occurred

job - the Job that failed

curQueue - the queue the Job came from

failMsg

protected String failMsg(Exception ex,
             String queue,
             Job job)
                  throws IOException

Create and serialize a JobFailure.

Parameters:

ex - the Exception that occurred

queue - the queue the job came from

job - the Job that failed

Returns:

the JSON representation of a new JobFailure

Throws:

IOException - if there was an error serializing the JobFailure

statusMsg

protected String statusMsg(String queue,
               Job job)
                    throws IOException

Create and serialize a WorkerStatus.

Parameters:

queue - the queue the Job came from

job - the Job currently being processed

Returns:

the JSON representation of a new WorkerStatus

Throws:

IOException - if there was an error serializing the WorkerStatus

pauseMsg

protected String pauseMsg()
                   throws IOException

Create and serialize a WorkerStatus for a pause event.

Returns:

the JSON representation of a new WorkerStatus

Throws:

IOException - if there was an error serializing the WorkerStatus

createName

protected String createName()

Creates a unique name, suitable for use with Resque.

Returns:

a unique name for this worker

key

protected String key(String... parts)

Builds a namespaced Redis key with the given arguments.

Parameters:

parts - the key parts to be joined

Returns:

an assembled String key

renameThread

protected void renameThread(String msg)

Rename the current thread with the given message.

Parameters:

msg - the message to add to the thread name

checkJobTypes

protected void checkJobTypes(Map<String,? extends Class<?>> jobTypes)

Verify the given job types are all valid.

Parameters:

jobTypes - the given job types

Throws:

IllegalArgumentException - if the job types are invalid

checkJobType

protected void checkJobType(String jobName,
                Class<?> jobType)

Determine if a job name and job type are valid.

Parameters:

jobName - the name of the job

jobType - the class of the job

Throws:

IllegalArgumentException - if the name and type are invalid

toString

public String toString()

Overrides:

toString in class Object