alluxio.master.job.tracker

Class DistLoadCliRunner

java.lang.Object

alluxio.master.job.tracker.AbstractCmdRunner

alluxio.master.job.tracker.DistLoadCliRunner

public class DistLoadCliRunner
extends AbstractCmdRunner

A config runner for a DistLoadCli job.

Field Summary

Fields inherited from class alluxio.master.job.tracker.AbstractCmdRunner

DEFAULT_ACTIVE_JOBS, DEFAULT_FILE_COUNT, mActiveJobs, mFileSystem, mFsContext, mJobMap, mJobMaster, mSubmitted

Constructor Summary

Constructors

Constructor and Description
DistLoadCliRunner(FileSystemContext fsContext, JobMaster jobMaster) constructor.

Method Summary

Modifier and Type	Method and Description
CmdInfo	runDistLoad(int batchSize, AlluxioURI filePath, int replication, java.util.Set<java.lang.String> workerSet, java.util.Set<java.lang.String> excludedWorkerSet, java.util.Set<java.lang.String> localityIds, java.util.Set<java.lang.String> excludedLocalityIds, boolean directCache, long jobControlId) Run a DistLoad command.
protected void	setJobConfigAndFileMetrics(java.util.List<URIStatus> filePath, int replication, java.util.Set<java.lang.String> workerSet, java.util.Set<java.lang.String> excludedWorkerSet, java.util.Set<java.lang.String> localityIds, java.util.Set<java.lang.String> excludedLocalityIds, boolean directCache, CmdRunAttempt attempt)

Methods inherited from class alluxio.master.job.tracker.AbstractCmdRunner

waitForCmdJob

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

DistLoadCliRunner

public DistLoadCliRunner(FileSystemContext fsContext,
                         JobMaster jobMaster)

constructor.

Parameters:

fsContext -

jobMaster -

Method Detail

runDistLoad

public CmdInfo runDistLoad(int batchSize,
                           AlluxioURI filePath,
                           int replication,
                           java.util.Set<java.lang.String> workerSet,
                           java.util.Set<java.lang.String> excludedWorkerSet,
                           java.util.Set<java.lang.String> localityIds,
                           java.util.Set<java.lang.String> excludedLocalityIds,
                           boolean directCache,
                           long jobControlId)
                    throws java.io.IOException

Run a DistLoad command.

Parameters:

batchSize - size for batched jobs

filePath - The AlluxioURI path to load into Alluxio memory

replication - Number of block replicas of each loaded file

workerSet - A set of worker hosts to load data

excludedWorkerSet - A set of worker hosts can not to load data

localityIds - The locality identify set

excludedLocalityIds - A set of worker locality identify can not to load data

directCache - Use direct cache request or cache through read

jobControlId - THe parent level job control ID

Returns:

CmdInfo

Throws:

java.io.IOException

setJobConfigAndFileMetrics

protected void setJobConfigAndFileMetrics(java.util.List<URIStatus> filePath,
                                          int replication,
                                          java.util.Set<java.lang.String> workerSet,
                                          java.util.Set<java.lang.String> excludedWorkerSet,
                                          java.util.Set<java.lang.String> localityIds,
                                          java.util.Set<java.lang.String> excludedLocalityIds,
                                          boolean directCache,
                                          CmdRunAttempt attempt)