org.apache.hadoop.fs.s3a.commit.staging

Class PartitionedStagingCommitterFactory

java.lang.Object

org.apache.hadoop.conf.Configured

org.apache.hadoop.mapreduce.lib.output.PathOutputCommitterFactory

org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitterFactory

org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitterFactory

All Implemented Interfaces:

org.apache.hadoop.conf.Configurable

public class PartitionedStagingCommitterFactory
extends AbstractS3ACommitterFactory

Factory for the PartitionedStagingCommitter.

Field Summary

Fields

Modifier and Type	Field and Description
static String	CLASSNAME Name of this class: "org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitterFactory".

Fields inherited from class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitterFactory

LOG

Fields inherited from class org.apache.hadoop.mapreduce.lib.output.PathOutputCommitterFactory

COMMITTER_FACTORY_CLASS, COMMITTER_FACTORY_DEFAULT, COMMITTER_FACTORY_SCHEME, COMMITTER_FACTORY_SCHEME_PATTERN, FILE_COMMITTER_FACTORY, NAMED_COMMITTER_CLASS, NAMED_COMMITTER_FACTORY

Constructor Summary

Constructors

Constructor and Description
PartitionedStagingCommitterFactory()

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.mapreduce.lib.output.PathOutputCommitter	createTaskCommitter(S3AFileSystem fileSystem, org.apache.hadoop.fs.Path outputPath, org.apache.hadoop.mapreduce.TaskAttemptContext context) Implementation point: create a task committer for a specific filesystem.

Methods inherited from class org.apache.hadoop.fs.s3a.commit.AbstractS3ACommitterFactory

createOutputCommitter, getDestinationFileSystem

Methods inherited from class org.apache.hadoop.mapreduce.lib.output.PathOutputCommitterFactory

createCommitter, createFileOutputCommitter, getCommitterFactory

Methods inherited from class org.apache.hadoop.conf.Configured

getConf, setConf

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

CLASSNAME

public static final String CLASSNAME

Name of this class: "org.apache.hadoop.fs.s3a.commit.staging.PartitionedStagingCommitterFactory".

See Also:

Constant Field Values

Constructor Detail

PartitionedStagingCommitterFactory

public PartitionedStagingCommitterFactory()

Method Detail

createTaskCommitter

public org.apache.hadoop.mapreduce.lib.output.PathOutputCommitter createTaskCommitter(S3AFileSystem fileSystem,
                                                                                      org.apache.hadoop.fs.Path outputPath,
                                                                                      org.apache.hadoop.mapreduce.TaskAttemptContext context)
                                                                               throws IOException

Description copied from class: AbstractS3ACommitterFactory

Implementation point: create a task committer for a specific filesystem.

Specified by:

createTaskCommitter in class AbstractS3ACommitterFactory

Parameters:

fileSystem - destination FS.

outputPath - final output path for work

context - task context

Returns:

a committer

Throws:

IOException - any problem, including the FS not supporting the desired committer