org.apache.gobblin.source.workunit

Class MultiWorkUnit

java.lang.Object

org.apache.gobblin.configuration.State

org.apache.gobblin.source.workunit.WorkUnit

org.apache.gobblin.source.workunit.MultiWorkUnit

All Implemented Interfaces:

WritableShim

Direct Known Subclasses:

MultiWorkUnit

public class MultiWorkUnit
extends WorkUnit

A class that wraps multiple WorkUnits so they can executed within a single task.

This class also extends the State object and thus it is possible to set and get properties from this class. The setProp(String, Object) method will add the specified key, value pair to this class as well as to every WorkUnit in workUnits. The WorkUnit.getProp(String) methods will only return properties that have been explicitily set in this class (e.g. it will not retrieve properties from workUnits.

Constructor Summary

Constructors

Constructor and Description
MultiWorkUnit() Deprecated. Use createEmpty() instead.

Method Summary

Modifier and Type	Method and Description
void	addWorkUnit(WorkUnit workUnit) Add a single WorkUnit.
void	addWorkUnits(Collection<WorkUnit> workUnits) Add a collection of WorkUnits.
static MultiWorkUnit	createEmpty() Create a new empty MultiWorkUnit instance.
boolean	equals(Object object)
List<WorkUnit>	getWorkUnits() Get an immutable list of WorkUnits wrapped by this MultiWorkUnit.
int	hashCode()
void	readFields(DataInput in)
void	setProp(String key, Object value) Set the specified key, value pair in this MultiWorkUnit as well as in all the inner WorkUnits.
void	setPropExcludeInnerWorkUnits(String key, Object value) Set the specified key, value pair in this MultiWorkUnit only, but do not propagate it to all the inner WorkUnits.
void	write(DataOutput out)

Methods inherited from class org.apache.gobblin.source.workunit.WorkUnit

contains, copyOf, create, create, getExpectedHighWatermark, getExpectedHighWatermark, getExpectedHighWatermark, getExtract, getHighWaterMark, getLowWatermark, getLowWaterMark, getLowWatermark, getLowWatermark, getOutputFilePath, getProp, getProp, setHighWaterMark, setLowWaterMark, setWatermarkInterval, skip

Methods inherited from class org.apache.gobblin.configuration.State

addAll, addAll, addAllIfNotExist, addAllIfNotExist, appendToListProp, appendToSetProp, getId, getPropAsBoolean, getPropAsBoolean, getPropAsCaseInsensitiveSet, getPropAsCaseInsensitiveSet, getPropAsDouble, getPropAsDouble, getPropAsInt, getPropAsInt, getPropAsJsonArray, getPropAsList, getPropAsList, getPropAsLong, getPropAsLong, getPropAsSet, getPropAsSet, getPropAsShort, getPropAsShort, getPropAsShortWithRadix, getPropAsShortWithRadix, getProperties, getProperty, getProperty, getPropertyNames, overrideWith, overrideWith, removeProp, removePropsWithPrefix, setId, setProps, toString

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

MultiWorkUnit

@Deprecated
public MultiWorkUnit()

Deprecated. Use createEmpty() instead.

Method Detail

getWorkUnits

public List<WorkUnit> getWorkUnits()

Get an immutable list of WorkUnits wrapped by this MultiWorkUnit.

Returns:

immutable list of WorkUnits wrapped by this MultiWorkUnit

addWorkUnit

public void addWorkUnit(WorkUnit workUnit)

Add a single WorkUnit.

Parameters:

workUnit - WorkUnit to add

addWorkUnits

public void addWorkUnits(Collection<WorkUnit> workUnits)

Add a collection of WorkUnits.

Parameters:

workUnits - collection of WorkUnits to add

setProp

public void setProp(String key,
                    Object value)

Set the specified key, value pair in this MultiWorkUnit as well as in all the inner WorkUnits. Set a property.

Both key and value are stored as strings.

Overrides:

setProp in class State

Parameters:

key - property key

value - property value

See Also:

State.setProp(java.lang.String, java.lang.Object)

setPropExcludeInnerWorkUnits

public void setPropExcludeInnerWorkUnits(String key,
                                         Object value)

Set the specified key, value pair in this MultiWorkUnit only, but do not propagate it to all the inner WorkUnits.

Parameters:

key - property key

value - property value

readFields

public void readFields(DataInput in)
                throws IOException

Specified by:

readFields in interface WritableShim

Overrides:

readFields in class WorkUnit

Throws:

IOException

write

public void write(DataOutput out)
           throws IOException

Specified by:

write in interface WritableShim

Overrides:

write in class WorkUnit

Throws:

IOException

equals

public boolean equals(Object object)

Overrides:

equals in class WorkUnit

hashCode

public int hashCode()

Overrides:

hashCode in class WorkUnit

createEmpty

public static MultiWorkUnit createEmpty()

Create a new empty MultiWorkUnit instance.

Returns:

a new empty MultiWorkUnit instance