com.thinkaurelius.titan.hadoop.compat

Interface HadoopCompat

public interface HadoopCompat

This interface encapsulates both API and bytecode-level IncompatibleClassChanges in Hadoop core. In theory, titan-hadoop-core should only touch parts of Hadoop core ABI that have remained stable since 1.0, and everything else should be hidden behind the compat layer.

This interface is public, but should be considered unstable and likely to change in the future as new Hadoop versions are released, or as titan-hadoop-core uses additional Hadoop features, or as bugs are discovered. It's possible to write and use a third-party implementation, but be prepared to update it when upgrading to a newer Titan release.

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.conf.Configuration	getContextConfiguration(org.apache.hadoop.mapreduce.TaskAttemptContext context) Get configuration from the supplied task attempt context and return it.
long	getCounter(org.apache.hadoop.mrunit.mapreduce.MapReduceDriver counters, Enum<?> e) Get the value of the counter specified by e on counters.
org.apache.hadoop.conf.Configuration	getJobContextConfiguration(org.apache.hadoop.mapreduce.JobContext context) Get configuration from the supplied job context and return it.
String	getMapredJarConfigKey()
String	getSpeculativeMapConfigKey() Return the Hadoop configuration key which takes a boolean value and controls whether Hadoop will attempt speculative execution of mappers.
String	getSpeculativeReduceConfigKey() Return the Hadoop configuration key which takes a boolean value and controls whether Hadoop will attempt speculative execution of reducers.
void	incrementContextCounter(org.apache.hadoop.mapreduce.TaskInputOutputContext context, Enum<?> counter, long incr) Add incr to the counter designated by counter on context.
HadoopCompiler	newCompiler(HadoopGraph g) Instantiate and return a HadoopCompiler instance that uses the supplied graph
JobClasspathConfigurer	newDistCacheConfigurer() Construct a JobClasspathConfigurer that walks the classpath and adds all jars its finds to the Hadoop Jobs's classpaths via the Hadoop Distributed Cache.
org.apache.hadoop.conf.Configuration	newImmutableConfiguration(org.apache.hadoop.conf.Configuration base) Construct a Configuration instance which throws UnsupportedOperationException on any attempt to modify its state, but which forwards to its parameter all method calls that don't mutate state (i.e.
JobClasspathConfigurer	newMapredJarConfigurer(String mapredJarPath) Construct a JobClasspathConfigurer that sets the Mapreduce job jar config key to the supplied value.
org.apache.hadoop.mapreduce.TaskAttemptContext	newTask(org.apache.hadoop.conf.Configuration c, org.apache.hadoop.mapreduce.TaskAttemptID t) Instantiate a new TaskAttemptContext using the given attempt ID and configuration.

Method Detail

newCompiler

HadoopCompiler newCompiler(HadoopGraph g)

Instantiate and return a HadoopCompiler instance that uses the supplied graph

Parameters:

g - data source/sink for the task compiler

Returns:

new compiler

newTask

org.apache.hadoop.mapreduce.TaskAttemptContext newTask(org.apache.hadoop.conf.Configuration c,
                                                       org.apache.hadoop.mapreduce.TaskAttemptID t)

Instantiate a new TaskAttemptContext using the given attempt ID and configuration.

Parameters:

c - configuration

t - task attempt ID

Returns:

new context object

getSpeculativeMapConfigKey

String getSpeculativeMapConfigKey()

Return the Hadoop configuration key which takes a boolean value and controls whether Hadoop will attempt speculative execution of mappers.

Returns:

string config key

getSpeculativeReduceConfigKey

String getSpeculativeReduceConfigKey()

Return the Hadoop configuration key which takes a boolean value and controls whether Hadoop will attempt speculative execution of reducers.

Returns:

string config key

getMapredJarConfigKey

String getMapredJarConfigKey()

incrementContextCounter

void incrementContextCounter(org.apache.hadoop.mapreduce.TaskInputOutputContext context,
                             Enum<?> counter,
                             long incr)

Add incr to the counter designated by counter on context.

Parameters:

context - Hadoop task IO context containing counter state

counter - name of the counter

incr - amount to add to the counter's current value

getContextConfiguration

org.apache.hadoop.conf.Configuration getContextConfiguration(org.apache.hadoop.mapreduce.TaskAttemptContext context)

Get configuration from the supplied task attempt context and return it.

Parameters:

context - Hadoop task attempt context

Returns:

configuration on supplied context

getJobContextConfiguration

org.apache.hadoop.conf.Configuration getJobContextConfiguration(org.apache.hadoop.mapreduce.JobContext context)

Get configuration from the supplied job context and return it.

Parameters:

context - Hadoop job context

Returns:

configuration on supplied context

getCounter

long getCounter(org.apache.hadoop.mrunit.mapreduce.MapReduceDriver counters,
                Enum<?> e)

Get the value of the counter specified by e on counters.

Parameters:

counters - MRUnit test driver containing counter state

e - the name of the counter whose value should be retrieved

Returns:

current value

newMapredJarConfigurer

JobClasspathConfigurer newMapredJarConfigurer(String mapredJarPath)

Construct a JobClasspathConfigurer that sets the Mapreduce job jar config key to the supplied value. The job jar should contain Faunus's classes plus its entire dependency tree ("fat" jar).

Parameters:

mapredJarPath - path to the mapreduce job jar

Returns:

a configurer

newDistCacheConfigurer

JobClasspathConfigurer newDistCacheConfigurer()

Construct a JobClasspathConfigurer that walks the classpath and adds all jars its finds to the Hadoop Jobs's classpaths via the Hadoop Distributed Cache.

Returns:

a configurer

newImmutableConfiguration

org.apache.hadoop.conf.Configuration newImmutableConfiguration(org.apache.hadoop.conf.Configuration base)

Construct a Configuration instance which throws UnsupportedOperationException on any attempt to modify its state, but which forwards to its parameter all method calls that don't mutate state (i.e. reads).

Parameters:

base - the configuration to encapsulate behind an immutable forwarder class

Returns:

an immutable forwarder class that encapsulates base