org.janusgraph.hadoop.compat

Interface HadoopCompat

public interface HadoopCompat

This interface encapsulates both API and bytecode-level IncompatibleClassChanges in Hadoop core. In theory, janusgraph-hadoop-core should only touch parts of Hadoop core ABI that have remained stable since 1.0, and everything else should be hidden behind the compat layer.

This interface is public, but should be considered unstable and likely to change in the future as new Hadoop versions are released, or as janusgraph-hadoop-core uses additional Hadoop features, or as bugs are discovered. It's possible to write and use a third-party implementation, but be prepared to update it when upgrading to a newer JanusGraph release.

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.conf.Configuration	getContextConfiguration(org.apache.hadoop.mapreduce.TaskAttemptContext context) Get configuration from the supplied task attempt context and return it.
long	getContextCounter(org.apache.hadoop.mapreduce.TaskInputOutputContext context, String group, String name) Return the current value of counter designated by counter on context.
org.apache.hadoop.conf.Configuration	getJobContextConfiguration(org.apache.hadoop.mapreduce.JobContext context) Get configuration from the supplied job context and return it.
String	getJobFailureString(org.apache.hadoop.mapreduce.Job j)
String	getMapredJarConfigKey()
ScanMetrics	getMetrics(org.apache.hadoop.mapreduce.Counters c)
String	getSpeculativeMapConfigKey() Return the Hadoop configuration key which takes a boolean value and controls whether Hadoop will attempt speculative execution of mappers.
String	getSpeculativeReduceConfigKey() Return the Hadoop configuration key which takes a boolean value and controls whether Hadoop will attempt speculative execution of reducers.
void	incrementContextCounter(org.apache.hadoop.mapreduce.TaskInputOutputContext context, String group, String name, long increment) Add increment to the counter designated by counter on context.
JobClasspathConfigurer	newDistCacheConfigurer() Construct a JobClasspathConfigurer that walks the classpath and adds all jars its finds to the Hadoop Jobs's class paths via the Hadoop Distributed Cache.
org.apache.hadoop.conf.Configuration	newImmutableConfiguration(org.apache.hadoop.conf.Configuration base) Construct a Configuration instance which throws UnsupportedOperationException on any attempt to modify its state, but which forwards to its parameter all method calls that don't mutate state (i.e.
JobClasspathConfigurer	newMapredJarConfigurer(String mapReduceJarPath) Construct a JobClasspathConfigurer that sets the MapReduce job jar config key to the supplied value.
org.apache.hadoop.mapreduce.TaskAttemptContext	newTask(org.apache.hadoop.conf.Configuration c, org.apache.hadoop.mapreduce.TaskAttemptID t) Instantiate a new TaskAttemptContext using the given attempt ID and configuration.

Method Detail

newTask

org.apache.hadoop.mapreduce.TaskAttemptContext newTask(org.apache.hadoop.conf.Configuration c,
                                                       org.apache.hadoop.mapreduce.TaskAttemptID t)

Instantiate a new TaskAttemptContext using the given attempt ID and configuration.

Parameters:

c - configuration

t - task attempt ID

Returns:

new context object

getSpeculativeMapConfigKey

String getSpeculativeMapConfigKey()

Return the Hadoop configuration key which takes a boolean value and controls whether Hadoop will attempt speculative execution of mappers.

Returns:

string config key

getSpeculativeReduceConfigKey

String getSpeculativeReduceConfigKey()

Return the Hadoop configuration key which takes a boolean value and controls whether Hadoop will attempt speculative execution of reducers.

Returns:

string config key

getMapredJarConfigKey

String getMapredJarConfigKey()

incrementContextCounter

void incrementContextCounter(org.apache.hadoop.mapreduce.TaskInputOutputContext context,
                             String group,
                             String name,
                             long increment)

Add increment to the counter designated by counter on context.

Parameters:

context - Hadoop task IO context containing counter state

group - the Hadoop counter group (heading under which the counter is displayed)

name - the Hadoop counter name (the identifier for this counter within the group)

increment - amount to add to the counter's current value

getContextCounter

long getContextCounter(org.apache.hadoop.mapreduce.TaskInputOutputContext context,
                       String group,
                       String name)

Return the current value of counter designated by counter on context.

Parameters:

context - Hadoop task IO context containing counter state

group - the Hadoop counter group (heading under which the counter is displayed)

name - the Hadoop counter name (the identifier for this counter within the group)

Returns:

current counter value

getContextConfiguration

org.apache.hadoop.conf.Configuration getContextConfiguration(org.apache.hadoop.mapreduce.TaskAttemptContext context)

Get configuration from the supplied task attempt context and return it.

Parameters:

context - Hadoop task attempt context

Returns:

configuration on supplied context

getJobContextConfiguration

org.apache.hadoop.conf.Configuration getJobContextConfiguration(org.apache.hadoop.mapreduce.JobContext context)

Get configuration from the supplied job context and return it.

Parameters:

context - Hadoop job context

Returns:

configuration on supplied context

newMapredJarConfigurer

JobClasspathConfigurer newMapredJarConfigurer(String mapReduceJarPath)

Construct a JobClasspathConfigurer that sets the MapReduce job jar config key to the supplied value. The job jar should contain Faunus's classes plus its entire dependency tree ("fat" jar).

Parameters:

mapReduceJarPath - path to the map reduce job jar

Returns:

a configurer

newDistCacheConfigurer

JobClasspathConfigurer newDistCacheConfigurer()

Construct a JobClasspathConfigurer that walks the classpath and adds all jars its finds to the Hadoop Jobs's class paths via the Hadoop Distributed Cache.

Returns:

a configurer

newImmutableConfiguration

org.apache.hadoop.conf.Configuration newImmutableConfiguration(org.apache.hadoop.conf.Configuration base)

Construct a Configuration instance which throws UnsupportedOperationException on any attempt to modify its state, but which forwards to its parameter all method calls that don't mutate state (i.e. reads).

Parameters:

base - the configuration to encapsulate behind an immutable forwarder class

Returns:

an immutable forwarder class that encapsulates base

getMetrics

ScanMetrics getMetrics(org.apache.hadoop.mapreduce.Counters c)

getJobFailureString

String getJobFailureString(org.apache.hadoop.mapreduce.Job j)