| Package | Description |
|---|---|
| org.apache.hadoop.fs.s3a.commit |
Support for committing the output of analytics jobs directly to S3.
|
| org.apache.hadoop.fs.s3a.commit.files |
Persistent data formats for the committers.
|
| org.apache.hadoop.fs.s3a.commit.impl |
Internal classes which make use of mapreduce code.
|
| Modifier and Type | Method and Description |
|---|---|
protected SuccessData |
AbstractS3ACommitter.maybeCreateSuccessMarker(org.apache.hadoop.mapreduce.JobContext context,
List<String> filenames,
org.apache.hadoop.fs.statistics.IOStatisticsSnapshot ioStatistics)
if the job requires a success marker on a successful job,
create the
_SUCCESS file. |
protected SuccessData |
AbstractS3ACommitter.maybeCreateSuccessMarkerFromCommits(CommitContext commitContext,
AbstractS3ACommitter.ActiveCommit pending)
if the job requires a success marker on a successful job,
create the file
CommitConstants._SUCCESS. |
| Modifier and Type | Method and Description |
|---|---|
static SuccessData |
SuccessData.load(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path)
Load an instance from a file, then validate it.
|
| Modifier and Type | Method and Description |
|---|---|
static org.apache.hadoop.util.JsonSerialization<SuccessData> |
SuccessData.serializer()
Get a JSON serializer for this class.
|
| Modifier and Type | Method and Description |
|---|---|
org.apache.hadoop.fs.statistics.IOStatistics |
SuccessData.save(org.apache.hadoop.fs.FileSystem fs,
org.apache.hadoop.fs.Path path,
org.apache.hadoop.util.JsonSerialization<SuccessData> serializer) |
byte[] |
SuccessData.toBytes(org.apache.hadoop.util.JsonSerialization<SuccessData> serializer) |
| Modifier and Type | Method and Description |
|---|---|
void |
CommitOperations.createSuccessMarker(org.apache.hadoop.fs.Path outputPath,
SuccessData successData,
boolean addMetrics)
Save the success data to the
_SUCCESS file. |
Copyright © 2008–2024 Apache Software Foundation. All rights reserved.