org.apache.gobblin.source.workunit

Class Extract

java.lang.Object

org.apache.gobblin.configuration.State

org.apache.gobblin.source.workunit.Extract

All Implemented Interfaces:

WritableShim

Direct Known Subclasses:

Extract, ImmutableExtract

public class Extract
extends State

A class representing all the base attributes required by all tables types. Subclasses will be expected to validate each table type for their respective required attributes.

The extract ID only needs to be unique for Extracts belonging to the same namespace/table. One or more WorkUnits can share the same extract ID. WorkUnits that do share an extract ID will be considered parts of a single Extract for the purpose of applying publishing policies.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Extract.TableType

Constructor Summary

Constructors

Constructor and Description
Extract(Extract.TableType type, String namespace, String table) Constructor.
Extract(Extract extract) Deep copy constructor.
Extract(SourceState state, Extract.TableType type, String namespace, String table) Deprecated. Extract does not use any property in SourceState. Use Extract(TableType, String, String)

Method Summary

Modifier and Type	Method and Description
void	addDeltaField(String... deltaFieldName) Deprecated. It is recommended to add delta fields in WorkUnit instead of Extract.
void	addPrimaryKey(String... primaryKeyFieldName) Deprecated. @deprecated It is recommended to add primary keys in WorkUnit instead of Extract.
boolean	equals(Object object)
List<String>	getDeltaFields() Deprecated. It is recommended to obtain delta fields from WorkUnit instead of Extract.
String	getExtractId() Get a (non-globally) unique ID for this Extract.
boolean	getIsFull() Deprecated. It is recommend to get this information from WorkUnit instead of Extract.
String	getNamespace() Get the dot-separated namespace of the table.
String	getOutputFilePath() Deprecated. As this.getIsFull is deprecated.
State	getPreviousTableState() Get the previous table State.
List<String>	getPrimaryKeys() Deprecated. It is recommended to obtain primary keys from WorkUnit instead of Extract.
String	getTable() Get the name of the table.
Extract.TableType	getType() Get the Extract.TableType of the table.
int	hashCode()
boolean	hasType() If this Extract has extract table type defined.
void	setDeltaFields(String... deltaFieldName) Deprecated. It is recommended to set delta fields in WorkUnit instead of Extract.
void	setExtractId(String extractId) Set a (non-globally) unique ID for this Extract.
void	setFullTrue(long extractFullRunTime) Deprecated. It is recommend to set this information in WorkUnit instead of Extract.
void	setPrimaryKeys(String... primaryKeyFieldName) Deprecated. It is recommended to set primary keys in WorkUnit instead of Extract.

Methods inherited from class org.apache.gobblin.configuration.State

addAll, addAll, addAllIfNotExist, addAllIfNotExist, appendToListProp, appendToSetProp, contains, getId, getProp, getProp, getPropAsBoolean, getPropAsBoolean, getPropAsCaseInsensitiveSet, getPropAsCaseInsensitiveSet, getPropAsDouble, getPropAsDouble, getPropAsInt, getPropAsInt, getPropAsJsonArray, getPropAsList, getPropAsList, getPropAsLong, getPropAsLong, getPropAsSet, getPropAsSet, getPropAsShort, getPropAsShort, getPropAsShortWithRadix, getPropAsShortWithRadix, getProperties, getProperty, getProperty, getPropertyNames, overrideWith, overrideWith, readFields, removeProp, removePropsWithPrefix, setId, setProp, setProps, toString, write

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, wait, wait, wait

Constructor Detail

Extract

@Deprecated
public Extract(SourceState state,
                           Extract.TableType type,
                           String namespace,
                           String table)

Deprecated. Extract does not use any property in SourceState. Use Extract(TableType, String, String)

Constructor.

Parameters:

state - a SourceState carrying properties needed to construct an Extract

namespace - dot separated namespace path

type - Extract.TableType

table - table name

Extract

public Extract(Extract.TableType type,
               String namespace,
               String table)

Constructor.

Parameters:

type - Extract.TableType

namespace - dot separated namespace path

table - table name

Extract

public Extract(Extract extract)

Deep copy constructor.

Parameters:

extract - the other Extract instance

Method Detail

equals

public boolean equals(Object object)

Overrides:

equals in class Object

hashCode

public int hashCode()

Overrides:

hashCode in class Object

getOutputFilePath

@Deprecated
public String getOutputFilePath()

Deprecated. As this.getIsFull is deprecated.

Get the writer output file path corresponding to this Extract.

Returns:

writer output file path corresponding to this Extract

hasType

public boolean hasType()

If this Extract has extract table type defined.

Returns:

true if it has, false otherwise.

getType

public Extract.TableType getType()

Get the Extract.TableType of the table.

Returns:

Extract.TableType of the table

getNamespace

public String getNamespace()

Get the dot-separated namespace of the table.

Returns:

dot-separated namespace of the table

getTable

public String getTable()

Get the name of the table.

Returns:

name of the table

getExtractId

public String getExtractId()

Get a (non-globally) unique ID for this Extract.

Returns:

unique ID for this Extract

setExtractId

public void setExtractId(String extractId)

Set a (non-globally) unique ID for this Extract.

Parameters:

extractId - unique ID for this Extract

getIsFull

@Deprecated
public boolean getIsFull()

Deprecated. It is recommend to get this information from WorkUnit instead of Extract.

Check if this Extract represents the full contents of the source table.

Returns:

true if this Extract represents the full contents of the source table and false otherwise

setFullTrue

@Deprecated
public void setFullTrue(long extractFullRunTime)

Deprecated. It is recommend to set this information in WorkUnit instead of Extract.

Set full drop date from the given time.

Parameters:

extractFullRunTime - full extract time

setPrimaryKeys

@Deprecated
public void setPrimaryKeys(String... primaryKeyFieldName)

Deprecated. It is recommended to set primary keys in WorkUnit instead of Extract.

Set primary keys.

The order of primary keys does not matter.

Parameters:

primaryKeyFieldName - primary key names

addPrimaryKey

@Deprecated
public void addPrimaryKey(String... primaryKeyFieldName)

Deprecated. @deprecated It is recommended to add primary keys in WorkUnit instead of Extract.

Add more primary keys to the existing set of primary keys.

Parameters:

primaryKeyFieldName - primary key names

getPrimaryKeys

@Deprecated
public List<String> getPrimaryKeys()

Deprecated. It is recommended to obtain primary keys from WorkUnit instead of Extract.

Get the list of primary keys.

Returns:

list of primary keys

setDeltaFields

@Deprecated
public void setDeltaFields(String... deltaFieldName)

Deprecated. It is recommended to set delta fields in WorkUnit instead of Extract.

Set delta fields.

The order of delta fields does not matter.

Parameters:

deltaFieldName - delta field names

addDeltaField

@Deprecated
public void addDeltaField(String... deltaFieldName)

Deprecated. It is recommended to add delta fields in WorkUnit instead of Extract.

Add more delta fields to the existing set of delta fields.

Parameters:

deltaFieldName - delta field names

getDeltaFields

@Deprecated
public List<String> getDeltaFields()

Deprecated. It is recommended to obtain delta fields from WorkUnit instead of Extract.

Get the list of delta fields.

Returns:

list of delta fields

getPreviousTableState

public State getPreviousTableState()

Get the previous table State.

Returns:

previous table State