org.apache.hadoop.hive.ql.plan

Class ReplCopyWork

java.lang.Object

org.apache.hadoop.hive.ql.plan.CopyWork

org.apache.hadoop.hive.ql.plan.ReplCopyWork

All Implemented Interfaces:

Serializable, BaseCopyWork

public class ReplCopyWork
extends CopyWork

Marker work for Replication - behaves similar to CopyWork, but maps to ReplCopyTask, which will have mechanics to list the files in source to write to the destination, instead of copying them, if specified, falling back to copying if needed.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
protected boolean	readSrcAsFilesList TODO : Refactor There is an upcoming patch that refactors this bit of code.

Constructor Summary

Constructors

Constructor and Description
ReplCopyWork(org.apache.hadoop.fs.Path srcPath, org.apache.hadoop.fs.Path destPath, boolean errorOnSrcEmpty)
ReplCopyWork(org.apache.hadoop.fs.Path srcPath, org.apache.hadoop.fs.Path destPath, boolean errorOnSrcEmpty, boolean overWrite)
ReplCopyWork(org.apache.hadoop.fs.Path srcPath, org.apache.hadoop.fs.Path destPath, boolean errorOnSrcEmpty, boolean overWrite, String dumpDirectory, ReplicationMetricCollector metricCollector)

Method Summary

Modifier and Type	Method and Description
String	distCpDoAsUser()
boolean	getDeleteDestIfExist()
String	getDumpDirectory()
boolean	getIsAutoPurge()
ReplicationMetricCollector	getMetricCollector()
boolean	getNeedRecycle()
void	initializeFromDeferredContext(DeferredWorkContext deferredContext)
boolean	isNeedCheckDuplicateCopy()
boolean	isOverWrite()
boolean	readSrcAsFilesList()
void	setAutoPurge(boolean isAutoPurge)
void	setCheckDuplicateCopy(boolean flag)
void	setDeleteDestIfExist(boolean deleteDestIfExist)
void	setDistCpDoAsUser(String distCpDoAsUser)
void	setNeedRecycle(boolean needRecycle)
void	setReadSrcAsFilesList(boolean readSrcAsFilesList)

Methods inherited from class org.apache.hadoop.hive.ql.plan.CopyWork

getFromPathExplain, getFromPaths, getFromPathsExplain, getToPathExplain, getToPaths, getToPathsExplain, isErrorOnSrcEmpty, isOverwrite, isReplication, setErrorOnSrcEmpty, setOverwrite, setToPath

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

readSrcAsFilesList

protected boolean readSrcAsFilesList

TODO : Refactor There is an upcoming patch that refactors this bit of code. Currently, the idea is the following: By default, ReplCopyWork will behave similarly to CopyWork, and simply copy along data from the source to destination. If the flag readSrcAsFilesList is set, changes the source behaviour of this CopyTask, and instead of copying explicit files, this will then fall back to a behaviour wherein an _files is read from the source, and the files specified by the _files are then copied to the destination. This allows us a lazy-copy-on-source and a pull-from destination semantic that we want to use from replication.

Constructor Detail

ReplCopyWork

public ReplCopyWork(org.apache.hadoop.fs.Path srcPath,
                    org.apache.hadoop.fs.Path destPath,
                    boolean errorOnSrcEmpty)

ReplCopyWork

public ReplCopyWork(org.apache.hadoop.fs.Path srcPath,
                    org.apache.hadoop.fs.Path destPath,
                    boolean errorOnSrcEmpty,
                    boolean overWrite)

ReplCopyWork

public ReplCopyWork(org.apache.hadoop.fs.Path srcPath,
                    org.apache.hadoop.fs.Path destPath,
                    boolean errorOnSrcEmpty,
                    boolean overWrite,
                    String dumpDirectory,
                    ReplicationMetricCollector metricCollector)

Method Detail

setReadSrcAsFilesList

public void setReadSrcAsFilesList(boolean readSrcAsFilesList)

readSrcAsFilesList

public boolean readSrcAsFilesList()

setDistCpDoAsUser

public void setDistCpDoAsUser(String distCpDoAsUser)

distCpDoAsUser

public String distCpDoAsUser()

getDeleteDestIfExist

public boolean getDeleteDestIfExist()

setDeleteDestIfExist

public void setDeleteDestIfExist(boolean deleteDestIfExist)

getNeedRecycle

public boolean getNeedRecycle()

setNeedRecycle

public void setNeedRecycle(boolean needRecycle)

getIsAutoPurge

public boolean getIsAutoPurge()

setAutoPurge

public void setAutoPurge(boolean isAutoPurge)

isNeedCheckDuplicateCopy

public boolean isNeedCheckDuplicateCopy()

setCheckDuplicateCopy

public void setCheckDuplicateCopy(boolean flag)

getMetricCollector

public ReplicationMetricCollector getMetricCollector()

Overrides:

getMetricCollector in class CopyWork

getDumpDirectory

public String getDumpDirectory()

Overrides:

getDumpDirectory in class CopyWork

isOverWrite

public boolean isOverWrite()

initializeFromDeferredContext

public void initializeFromDeferredContext(DeferredWorkContext deferredContext)
                                   throws HiveException

Specified by:

initializeFromDeferredContext in interface BaseCopyWork

Overrides:

initializeFromDeferredContext in class CopyWork

Throws:

HiveException