org.apache.hadoop.fs.s3a.commit.magic

Class MagicCommitTrackerUtils

java.lang.Object

org.apache.hadoop.fs.s3a.commit.magic.MagicCommitTrackerUtils

public final class MagicCommitTrackerUtils
extends Object

Utility class for the class MagicCommitTracker and its subclasses.

Method Summary

Modifier and Type	Method and Description
static String	extractTaskAttemptIdFromPath(org.apache.hadoop.fs.Path path) The magic path is of the following format.
static boolean	isCleanupMagicCommitterEnabled(org.apache.hadoop.conf.Configuration conf) Is cleanup of magic committer staging dirs enabled.
static boolean	isTrackMagicCommitsInMemoryEnabled(org.apache.hadoop.conf.Configuration conf) Is tracking of magic commit data in-memory enabled.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

extractTaskAttemptIdFromPath

public static String extractTaskAttemptIdFromPath(org.apache.hadoop.fs.Path path)

The magic path is of the following format. "s3://bucket-name/table-path/__magic_jobId/job-id/taskAttempt/id/tasks/taskAttemptId" So the third child from the "__magic" path will give the task attempt id.

Parameters:

path - Path

Returns:

taskAttemptId

isTrackMagicCommitsInMemoryEnabled

public static boolean isTrackMagicCommitsInMemoryEnabled(org.apache.hadoop.conf.Configuration conf)

Is tracking of magic commit data in-memory enabled.

Parameters:

conf - Configuration

Returns:

true if in memory tracking of commit data is enabled.

isCleanupMagicCommitterEnabled

public static boolean isCleanupMagicCommitterEnabled(org.apache.hadoop.conf.Configuration conf)

Is cleanup of magic committer staging dirs enabled.

Parameters:

conf - Configuration

Returns:

true if cleanup of staging dir is enabled.