org.apache.carbondata.hadoop.api

Class CarbonOutputCommitter

java.lang.Object

org.apache.hadoop.mapreduce.OutputCommitter

org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter

org.apache.carbondata.hadoop.api.CarbonOutputCommitter

public class CarbonOutputCommitter
extends org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter

Outputcommitter which manages the segments during loading.It commits segment information to the tablestatus file upon success or fail.

Field Summary

Fields inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter

FILEOUTPUTCOMMITTER_ALGORITHM_VERSION, FILEOUTPUTCOMMITTER_ALGORITHM_VERSION_DEFAULT, PENDING_DIR_NAME, SUCCEEDED_FILE_NAME, SUCCESSFUL_JOB_OUTPUT_DIR_MARKER, TEMP_DIR_NAME

Constructor Summary

Constructors

Constructor and Description
CarbonOutputCommitter(org.apache.hadoop.fs.Path outputPath, org.apache.hadoop.mapreduce.TaskAttemptContext context)

Method Summary

Modifier and Type	Method and Description
void	abortJob(org.apache.hadoop.mapreduce.JobContext context, org.apache.hadoop.mapreduce.JobStatus.State state) Update the tablestatus as fail if any fail happens.And also clean up the temp folders if any are existed.
void	commitJob(org.apache.hadoop.mapreduce.JobContext context) Update the tablestatus as success after job is success
void	setupJob(org.apache.hadoop.mapreduce.JobContext context) Update the tablestatus with inprogress while setup the job.
void	setupTask(org.apache.hadoop.mapreduce.TaskAttemptContext context)

Methods inherited from class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter

abortTask, abortTask, cleanupJob, commitTask, commitTask, getCommittedTaskPath, getCommittedTaskPath, getCommittedTaskPath, getJobAttemptPath, getJobAttemptPath, getJobAttemptPath, getTaskAttemptPath, getTaskAttemptPath, getWorkPath, isRecoverySupported, needsTaskCommit, needsTaskCommit, recoverTask

Methods inherited from class org.apache.hadoop.mapreduce.OutputCommitter

isRecoverySupported

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

CarbonOutputCommitter

public CarbonOutputCommitter(org.apache.hadoop.fs.Path outputPath,
                             org.apache.hadoop.mapreduce.TaskAttemptContext context)
                      throws IOException

Throws:

IOException

Method Detail

setupJob

public void setupJob(org.apache.hadoop.mapreduce.JobContext context)
              throws IOException

Update the tablestatus with inprogress while setup the job.

Overrides:

setupJob in class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter

Parameters:

context -

Throws:

IOException

setupTask

public void setupTask(org.apache.hadoop.mapreduce.TaskAttemptContext context)
               throws IOException

Overrides:

setupTask in class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter

Throws:

IOException

commitJob

public void commitJob(org.apache.hadoop.mapreduce.JobContext context)
               throws IOException

Update the tablestatus as success after job is success

Overrides:

commitJob in class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter

Parameters:

context -

Throws:

IOException

abortJob

public void abortJob(org.apache.hadoop.mapreduce.JobContext context,
                     org.apache.hadoop.mapreduce.JobStatus.State state)
              throws IOException

Update the tablestatus as fail if any fail happens.And also clean up the temp folders if any are existed.

Overrides:

abortJob in class org.apache.hadoop.mapreduce.lib.output.FileOutputCommitter

Parameters:

context -

state -

Throws:

IOException