WorkUnitState (gobblin-api 0.16.0 API)

java.lang.Object
- org.apache.gobblin.configuration.State
- - org.apache.gobblin.configuration.WorkUnitState

All Implemented Interfaces:

WritableShim

Direct Known Subclasses:

ImmutableWorkUnitState, WorkUnitState
```
public class WorkUnitState
extends State
```
This class encapsulates a WorkUnit instance and additionally holds all the task runtime state of that WorkUnit.
Properties set in the encapsulated WorkUnit can be overridden at runtime, with the original values available through the getWorkunit() method. Getters will return values set at task runtime if available, or the corresponding values from encapsulated WorkUnit if they are not set at task runtime.

Nested Class Summary

Nested Classes
Modifier and Type Class and Description

static class WorkUnitState.WorkingState
Runtime state of the WorkUnit.

Nested Classes
Modifier and Type	Class and Description
`static class`	`WorkUnitState.WorkingState` Runtime state of the `WorkUnit`.

Constructor Summary

Constructors
Constructor and Description
`WorkUnitState()` Default constructor used for deserialization.
`WorkUnitState(WorkUnit workUnit)` Deprecated. It is recommended to use `WorkUnitState(WorkUnit, State)` rather than combining properties in the job state into the workunit.
`WorkUnitState(WorkUnit workUnit, State jobState)` If creating a `WorkUnitState` for use by a task, use `WorkUnitState(WorkUnit, State, SharedResourcesBroker)` instead.
`WorkUnitState(WorkUnit workUnit, State jobState, SharedResourcesBroker<GobblinScopeTypes> taskBroker)`
`WorkUnitState(WorkUnit workUnit, State jobState, SubscopedBrokerBuilder<GobblinScopeTypes,?> taskBrokerBuilder)`

Method Summary

All Methods Instance Methods Concrete Methods Deprecated Methods
Modifier and Type	Method and Description
`void`	`addFinalConstructState(String infix, State finalConstructState)` Adds all properties from `State` to this `WorkUnitState`.
`void`	`backoffActualHighWatermark()` Backoff the actual high watermark to the low watermark returned by `WorkUnit.getLowWatermark()`.
`boolean`	`contains(String key)` Check if a property is set.
`boolean`	`equals(Object object)`
`com.google.gson.JsonElement`	`getActualHighWatermark()` Get the actual high `Watermark` as a `JsonElement`.
`<T extends Watermark> T`	`getActualHighWatermark(Class<T> watermarkClass)` Get the actual high `Watermark`.
`<T extends Watermark> T`	`getActualHighWatermark(Class<T> watermarkClass, com.google.gson.Gson gson)` Get the actual high `Watermark`.
`Extract`	`getExtract()` Get the `Extract` associated with the `WorkUnit`.
`State`	`getFinalConstructStates()` Builds a State containing all properties added with `addFinalConstructState(java.lang.String, org.apache.gobblin.configuration.State)` to this `WorkUnitState`.
`long`	`getHighWaterMark()` Deprecated. use `getActualHighWatermark()`.
`String`	`getOutputFilePath()`
`State`	`getPreviousTableState()` Get properties set in the previous run for the same table as the `WorkUnit`.
`String`	`getProp(String key)` Get the value of a property.
`String`	`getProp(String key, String def)` Get the value of a property, using the given default value if the property is not set.
`Properties`	`getProperties()` Return a copy of the underlying `Properties` object.
`protected String`	`getProperty(String key)` Deprecated. Use `getProp(String)`
`protected String`	`getProperty(String key, String def)` Deprecated. Use `getProp(String, String)`
`Set<String>`	`getPropertyNames()` Get the names of all the properties set in a `Set`.
`SharedResourcesBroker<GobblinScopeTypes>`	`getTaskBroker()` Get a `SharedResourcesBroker` scoped for this task.
`SharedResourcesBroker<GobblinScopeTypes>`	`getTaskBrokerNullable()` Get a `SharedResourcesBroker` scoped for this task or null if it doesn't exist.
`WorkUnitState.WorkingState`	`getWorkingState()` Get the current runtime state of the `WorkUnit`.
`WorkUnit`	`getWorkunit()` Get an `ImmutableWorkUnit` that wraps the internal `WorkUnit`.
`int`	`hashCode()`
`void`	`readFields(DataInput in)`
`void`	`removeProp(String key)` Remove a property if it exists.
`void`	`removePropsWithPrefix(String prefix)` Remove all properties with a certain keyPrefix
`void`	`setActualHighWatermark(Watermark watermark)` This method should set the actual, runtime high `Watermark` for this `WorkUnitState`.
`void`	`setHighWaterMark(long value)` Deprecated. use `setActualHighWatermark(Watermark)`.
`void`	`setJobState(State jobState)`
`void`	`setWorkingState(WorkUnitState.WorkingState state)` Set the current runtime state of the `WorkUnit`.
`void`	`setWuProperties(Properties commonProps, Properties specProps)` Override `workUnit`'s properties with new commonProps and specProps.
`String`	`toString()`
`void`	`write(DataOutput out)`

Methods inherited from class org.apache.gobblin.configuration.State
addAll, addAll, addAllIfNotExist, addAllIfNotExist, appendToListProp, appendToSetProp, getId, getPropAsBoolean, getPropAsBoolean, getPropAsCaseInsensitiveSet, getPropAsCaseInsensitiveSet, getPropAsDouble, getPropAsDouble, getPropAsInt, getPropAsInt, getPropAsJsonArray, getPropAsList, getPropAsList, getPropAsLong, getPropAsLong, getPropAsSet, getPropAsSet, getPropAsShort, getPropAsShort, getPropAsShortWithRadix, getPropAsShortWithRadix, overrideWith, overrideWith, setId, setProp, setProps

Methods inherited from class java.lang.Object
clone, finalize, getClass, notify, notifyAll, wait, wait, wait

- Constructor Detail
  - WorkUnitState
```
public WorkUnitState()
```
    Default constructor used for deserialization.
  - WorkUnitState
```
@Deprecated
public WorkUnitState(WorkUnit workUnit)
```
    Deprecated. It is recommended to use WorkUnitState(WorkUnit, State) rather than combining properties in the job state into the workunit.
    
    Constructor.
    
    Parameters:
    
    workUnit - a WorkUnit instance based on which a WorkUnitState instance is constructed
  - WorkUnitState
```
public WorkUnitState(WorkUnit workUnit,
                     State jobState)
```
    If creating a WorkUnitState for use by a task, use WorkUnitState(WorkUnit, State, SharedResourcesBroker) instead.
  - WorkUnitState
```
public WorkUnitState(WorkUnit workUnit,
                     State jobState,
                     SubscopedBrokerBuilder<GobblinScopeTypes,?> taskBrokerBuilder)
```
  - WorkUnitState
```
public WorkUnitState(WorkUnit workUnit,
                     State jobState,
                     SharedResourcesBroker<GobblinScopeTypes> taskBroker)
```
- Method Detail
  - getOutputFilePath
```
public String getOutputFilePath()
```
  - getTaskBroker
```
public SharedResourcesBroker<GobblinScopeTypes> getTaskBroker()
```
    Get a SharedResourcesBroker scoped for this task.
  - getTaskBrokerNullable
```
@Nullable
public SharedResourcesBroker<GobblinScopeTypes> getTaskBrokerNullable()
```
    Get a SharedResourcesBroker scoped for this task or null if it doesn't exist. This is used for internal calls.
  - getWorkunit
```
public WorkUnit getWorkunit()
```
    Get an ImmutableWorkUnit that wraps the internal WorkUnit.
    
    Returns:
    
    an ImmutableWorkUnit that wraps the internal WorkUnit
  - setWuProperties
```
public void setWuProperties(Properties commonProps,
                            Properties specProps)
```
    Override workUnit's properties with new commonProps and specProps.
  - getWorkingState
```
public WorkUnitState.WorkingState getWorkingState()
```
    Get the current runtime state of the WorkUnit.
    
    Returns:
    
    WorkUnitState.WorkingState of the WorkUnit
  - setWorkingState
```
public void setWorkingState(WorkUnitState.WorkingState state)
```
    Set the current runtime state of the WorkUnit.
    
    Parameters:
    
    state - WorkUnitState.WorkingState of the WorkUnit
  - getActualHighWatermark
```
public com.google.gson.JsonElement getActualHighWatermark()
```
    Get the actual high Watermark as a JsonElement.
    
    Returns:
    
    a JsonElement representing the actual high Watermark, or null if the actual high Watermark is not set.
  - getActualHighWatermark
```
public <T extends Watermark> T getActualHighWatermark(Class<T> watermarkClass,
                                                      com.google.gson.Gson gson)
```
    Get the actual high Watermark. If the WorkUnitState does not contain the actual high watermark (which may be caused by task failures), the low watermark in the corresponding WorkUnit will be returned.
    
    Parameters:
    
    watermarkClass - the watermark class for this WorkUnitState.
    
    gson - a Gson object used to deserialize the watermark.
    
    Returns:
    
    the actual high watermark in this WorkUnitState. null is returned if this WorkUnitState does not contain an actual high watermark, and the corresponding WorkUnit does not contain a low watermark.
  - getActualHighWatermark
```
public <T extends Watermark> T getActualHighWatermark(Class<T> watermarkClass)
```
    Get the actual high Watermark. If the WorkUnitState does not contain the actual high watermark (which may be caused by task failures), the low watermark in the corresponding WorkUnit will be returned.
    A default Gson object will be used to deserialize the watermark.
    
    Parameters:
    
    watermarkClass - the watermark class for this WorkUnitState.
    
    Returns:
    
    the actual high watermark in this WorkUnitState. null is returned if this WorkUnitState does not contain an actual high watermark, and the corresponding WorkUnit does not contain a low watermark.
  - setActualHighWatermark
```
public void setActualHighWatermark(Watermark watermark)
```
    This method should set the actual, runtime high Watermark for this WorkUnitState. A high Watermark indicates that all data for the source has been pulled up to a specific point.
    This method should be called inside the Extractor class, during the initialization of the class, before any calls to Extractor.readRecord(Object) are executed. This method keeps a local point to the given Watermark and expects the following invariant to always be upheld. The invariant for this Watermark is that it should cover all records up to and including the most recent record returned by Extractor.readRecord(Object).
    
    The Watermark set in this method may be polled by the framework multiple times, in order to track the progress of how the Watermark changes. This is important for reporting percent completion of a WorkUnit.
    TODO - Once we are ready to make a backwards incompatible change to the Extractor interface, this method should become part of the Extractor interface. For example, a method such as getCurrentHighWatermark() should be added.
  - backoffActualHighWatermark
```
public void backoffActualHighWatermark()
```
    Backoff the actual high watermark to the low watermark returned by WorkUnit.getLowWatermark().
  - getHighWaterMark
```
@Deprecated
public long getHighWaterMark()
```
    Deprecated. use getActualHighWatermark().
    
    Get the high watermark as set in Extractor.
    
    Returns:
    
    high watermark
  - setHighWaterMark
```
@Deprecated
public void setHighWaterMark(long value)
```
    Deprecated. use setActualHighWatermark(Watermark).
    
    Set the high watermark.
    
    Parameters:
    
    value - high watermark
  - getProperties
```
public Properties getProperties()
```
    Description copied from class: State
    
    Return a copy of the underlying Properties object.
    
    Overrides:
    
    getProperties in class State
    
    Returns:
    
    A copy of the underlying Properties object.
  - getProp
```
public String getProp(String key)
```
    Description copied from class: State
    
    Get the value of a property.
    
    Overrides:
    
    getProp in class State
    
    Parameters:
    
    key - property key
    
    Returns:
    
    value associated with the key as a string or null if the property is not set
  - getProp
```
public String getProp(String key,
                      String def)
```
    Description copied from class: State
    
    Get the value of a property, using the given default value if the property is not set.
    
    Overrides:
    
    getProp in class State
    
    Parameters:
    
    key - property key
    
    def - default value
    
    Returns:
    
    value associated with the key or the default value if the property is not set
  - getProperty
```
@Deprecated
protected String getProperty(String key)
```
    Deprecated. Use getProp(String)
    
    Overrides:
    
    getProperty in class State
  - getProperty
```
@Deprecated
protected String getProperty(String key,
                                         String def)
```
    Deprecated. Use getProp(String, String)
    
    Overrides:
    
    getProperty in class State
  - getPropertyNames
```
public Set<String> getPropertyNames()
```
    Description copied from class: State
    
    Get the names of all the properties set in a Set.
    
    Overrides:
    
    getPropertyNames in class State
    
    Returns:
    
    names of all the properties set in a Set
  - contains
```
public boolean contains(String key)
```
    Description copied from class: State
    
    Check if a property is set.
    
    Overrides:
    
    contains in class State
    
    Parameters:
    
    key - property key
    
    Returns:
    
    true if the property is set or false otherwise
  - removeProp
```
public void removeProp(String key)
```
    Description copied from class: State
    
    Remove a property if it exists.
    
    Overrides:
    
    removeProp in class State
    
    Parameters:
    
    key - property key
  - removePropsWithPrefix
```
public void removePropsWithPrefix(String prefix)
```
    Description copied from class: State
    
    Remove all properties with a certain keyPrefix
    
    Overrides:
    
    removePropsWithPrefix in class State
    
    Parameters:
    
    prefix - key prefix
  - getExtract
```
public Extract getExtract()
```
    Get the Extract associated with the WorkUnit.
    
    Returns:
    
    Extract associated with the WorkUnit
  - getPreviousTableState
```
public State getPreviousTableState()
```
    Get properties set in the previous run for the same table as the WorkUnit.
    
    Returns:
    
    properties as a State object
  - setJobState
```
public void setJobState(State jobState)
```
  - readFields
```
public void readFields(DataInput in)
                throws IOException
```
    Specified by:
    
    readFields in interface WritableShim
    
    Overrides:
    
    readFields in class State
    
    Throws:
    
    IOException
  - write
```
public void write(DataOutput out)
           throws IOException
```
    Specified by:
    
    write in interface WritableShim
    
    Overrides:
    
    write in class State
    
    Throws:
    
    IOException
  - equals
```
public boolean equals(Object object)
```
    Overrides:
    
    equals in class Object
  - hashCode
```
public int hashCode()
```
    Overrides:
    
    hashCode in class Object
  - toString
```
public String toString()
```
    Overrides:
    
    toString in class State
  - addFinalConstructState
```
public void addFinalConstructState(String infix,
                                   State finalConstructState)
```
    Adds all properties from State to this WorkUnitState.
    A property with name "property" will be added to this object with the key "FINAL_CONSTRUCT_STATE_PREFIX[.].property"
    
    Parameters:
    
    infix - Optional infix used for the name of the property in the WorkUnitState.
    
    finalConstructState - State for which all properties should be added to this object.
  - getFinalConstructStates
```
public State getFinalConstructStates()
```
    Builds a State containing all properties added with addFinalConstructState(java.lang.String, org.apache.gobblin.configuration.State) to this WorkUnitState. All such properties will be stripped of FINAL_CONSTRUCT_STATE_PREFIX but not of any infixes.
    For example, if state={sample.property: sampleValue} then
```
     
        this.addFinalConstructState("infix",state);
        this.getFinalConstructState();
      
   
```
    will return state={infix.sample.property: sampleValue}
    Returns:
    
    State containing all properties added with addFinalConstructState(java.lang.String, org.apache.gobblin.configuration.State).

Class WorkUnitState

Nested Class Summary

Constructor Summary

Method Summary

Methods inherited from class org.apache.gobblin.configuration.State

Methods inherited from class java.lang.Object

Constructor Detail

WorkUnitState

WorkUnitState

WorkUnitState

WorkUnitState

WorkUnitState

Method Detail

getOutputFilePath

getTaskBroker

getTaskBrokerNullable

getWorkunit

setWuProperties

getWorkingState

setWorkingState

getActualHighWatermark

getActualHighWatermark

getActualHighWatermark

setActualHighWatermark

backoffActualHighWatermark

getHighWaterMark

setHighWaterMark

getProperties

getProp

getProp

getProperty

getProperty

getPropertyNames

contains

removeProp

removePropsWithPrefix

getExtract

getPreviousTableState

setJobState

readFields

write

equals

hashCode

toString

addFinalConstructState

getFinalConstructStates