org.apache.gobblin.configuration

Class ImmutableWorkUnitState

java.lang.Object

org.apache.gobblin.configuration.State

org.apache.gobblin.configuration.WorkUnitState

org.apache.gobblin.configuration.ImmutableWorkUnitState

All Implemented Interfaces:

WritableShim

Direct Known Subclasses:

ImmutableWorkUnitState

public class ImmutableWorkUnitState
extends WorkUnitState

An immutable version of WorkUnitState.

Nested Class Summary

Nested classes/interfaces inherited from class org.apache.gobblin.configuration.WorkUnitState

WorkUnitState.WorkingState

Constructor Summary

Constructors

Constructor and Description
ImmutableWorkUnitState(WorkUnitState workUnitState)

Method Summary

Modifier and Type	Method and Description
void	addAll(Properties properties) Populates this instance with values of a Properties instance.
void	addAllIfNotExist(Properties properties) Add properties in a Properties instance that are not in the current instance.
void	appendToListProp(String key, String value) Appends the input value to a list property that can be retrieved with State.getPropAsList(java.lang.String).
void	backoffActualHighWatermark() Backoff the actual high watermark to the low watermark returned by WorkUnit.getLowWatermark().
void	overrideWith(Properties properties) Add properties in a Properties instance that are in the current instance.
void	readFields(DataInput in)
void	setActualHighWatermark(Watermark watermark) This method should set the actual, runtime high Watermark for this WorkUnitState.
void	setHighWaterMark(long value) Deprecated.
void	setId(String id) Set the id used for state persistence and logging.
void	setProp(String key, Object value) Set a property.
void	setWorkingState(WorkUnitState.WorkingState state) Set the current runtime state of the WorkUnit.

Methods inherited from class org.apache.gobblin.configuration.WorkUnitState

addFinalConstructState, contains, equals, getActualHighWatermark, getActualHighWatermark, getActualHighWatermark, getExtract, getFinalConstructStates, getHighWaterMark, getOutputFilePath, getPreviousTableState, getProp, getProp, getProperties, getProperty, getProperty, getPropertyNames, getTaskBroker, getTaskBrokerNullable, getWorkingState, getWorkunit, hashCode, removeProp, removePropsWithPrefix, setJobState, setWuProperties, toString, write

Methods inherited from class org.apache.gobblin.configuration.State

addAll, addAllIfNotExist, appendToSetProp, getId, getPropAsBoolean, getPropAsBoolean, getPropAsCaseInsensitiveSet, getPropAsCaseInsensitiveSet, getPropAsDouble, getPropAsDouble, getPropAsInt, getPropAsInt, getPropAsJsonArray, getPropAsList, getPropAsList, getPropAsLong, getPropAsLong, getPropAsSet, getPropAsSet, getPropAsShort, getPropAsShort, getPropAsShortWithRadix, getPropAsShortWithRadix, overrideWith, setProps

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

ImmutableWorkUnitState

public ImmutableWorkUnitState(WorkUnitState workUnitState)

Method Detail

setWorkingState

public void setWorkingState(WorkUnitState.WorkingState state)

Description copied from class: WorkUnitState

Set the current runtime state of the WorkUnit.

Overrides:

setWorkingState in class WorkUnitState

Parameters:

state - WorkUnitState.WorkingState of the WorkUnit

setActualHighWatermark

public void setActualHighWatermark(Watermark watermark)

Description copied from class: WorkUnitState

This method should set the actual, runtime high Watermark for this WorkUnitState. A high Watermark indicates that all data for the source has been pulled up to a specific point.

This method should be called inside the Extractor class, during the initialization of the class, before any calls to Extractor.readRecord(Object) are executed. This method keeps a local point to the given Watermark and expects the following invariant to always be upheld. The invariant for this Watermark is that it should cover all records up to and including the most recent record returned by Extractor.readRecord(Object).

The Watermark set in this method may be polled by the framework multiple times, in order to track the progress of how the Watermark changes. This is important for reporting percent completion of a WorkUnit.

TODO - Once we are ready to make a backwards incompatible change to the Extractor interface, this method should become part of the Extractor interface. For example, a method such as getCurrentHighWatermark() should be added.

Overrides:

setActualHighWatermark in class WorkUnitState

setHighWaterMark

@Deprecated
public void setHighWaterMark(long value)

Deprecated.

Description copied from class: WorkUnitState

Set the high watermark.

Overrides:

setHighWaterMark in class WorkUnitState

Parameters:

value - high watermark

setProp

public void setProp(String key,
                    Object value)

Description copied from class: State

Set a property.

Both key and value are stored as strings.

Overrides:

setProp in class State

Parameters:

key - property key

value - property value

addAll

public void addAll(Properties properties)

Description copied from class: State

Populates this instance with values of a Properties instance.

Overrides:

addAll in class State

Parameters:

properties - a Properties instance

addAllIfNotExist

public void addAllIfNotExist(Properties properties)

Description copied from class: State

Add properties in a Properties instance that are not in the current instance.

Overrides:

addAllIfNotExist in class State

Parameters:

properties - a Properties instance

overrideWith

public void overrideWith(Properties properties)

Description copied from class: State

Add properties in a Properties instance that are in the current instance.

Overrides:

overrideWith in class State

Parameters:

properties - a Properties instance

setId

public void setId(String id)

Description copied from class: State

Set the id used for state persistence and logging.

Overrides:

setId in class State

Parameters:

id - id of this instance

readFields

public void readFields(DataInput in)
                throws IOException

Specified by:

readFields in interface WritableShim

Overrides:

readFields in class WorkUnitState

Throws:

IOException

backoffActualHighWatermark

public void backoffActualHighWatermark()

Description copied from class: WorkUnitState

Backoff the actual high watermark to the low watermark returned by WorkUnit.getLowWatermark().

Overrides:

backoffActualHighWatermark in class WorkUnitState

appendToListProp

public void appendToListProp(String key,
                             String value)

Description copied from class: State

Appends the input value to a list property that can be retrieved with State.getPropAsList(java.lang.String).

List properties are internally stored as comma separated strings. Adding a value that contains commas (for example "a,b,c") will essentially add multiple values to the property ("a", "b", and "c"). This is similar to the way that Configuration works.

Overrides:

appendToListProp in class State

Parameters:

key - property key

value - property value (if it includes commas, it will be split by the commas).