org.apache.tez.runtime.library.common.shuffle

Class Fetcher

java.lang.Object

org.apache.tez.common.CallableWithNdc<FetchResult>

org.apache.tez.runtime.library.common.shuffle.Fetcher

All Implemented Interfaces:

Callable<FetchResult>

public class Fetcher
extends org.apache.tez.common.CallableWithNdc<FetchResult>

Responsible for fetching inputs served by the ShuffleHandler for a single host. Construct using Fetcher.FetcherBuilder

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Fetcher.FetcherBuilder Builder for the construction of Fetchers
static class	Fetcher.PathPartition

Method Summary

Modifier and Type	Method and Description
FetchResult	callInternal()
protected org.apache.tez.runtime.library.common.shuffle.Fetcher.HostFetchResult	doHttpFetch()
protected org.apache.tez.runtime.library.common.shuffle.Fetcher.HostFetchResult	doHttpFetch(org.apache.tez.runtime.library.common.shuffle.Fetcher.CachingCallBack callback)
protected org.apache.tez.runtime.library.common.shuffle.Fetcher.HostFetchResult	doSharedFetch()
boolean	equals(Object obj)
String	getHost()
Map<Fetcher.PathPartition,InputAttemptIdentifier>	getPathToAttemptMap()
protected org.apache.hadoop.fs.Path	getShuffleInputFileName(String pathComponent, String suffix)
List<InputAttemptIdentifier>	getSrcAttempts()
protected TezIndexRecord	getTezIndexRecord(InputAttemptIdentifier srcAttemptId, int partition)
int	hashCode()
protected org.apache.tez.runtime.library.common.shuffle.Fetcher.HostFetchResult	setupLocalDiskFetch()
void	shutdown()

Methods inherited from class org.apache.tez.common.CallableWithNdc

call

Methods inherited from class java.lang.Object

clone, finalize, getClass, notify, notifyAll, toString, wait, wait, wait

Method Detail

getSrcAttempts

public List<InputAttemptIdentifier> getSrcAttempts()

getHost

public String getHost()

callInternal

public FetchResult callInternal()
                         throws Exception

Specified by:

callInternal in class org.apache.tez.common.CallableWithNdc<FetchResult>

Throws:

Exception

doSharedFetch

protected org.apache.tez.runtime.library.common.shuffle.Fetcher.HostFetchResult doSharedFetch()
                                                                                       throws IOException

Throws:

IOException

doHttpFetch

protected org.apache.tez.runtime.library.common.shuffle.Fetcher.HostFetchResult doHttpFetch()

doHttpFetch

protected org.apache.tez.runtime.library.common.shuffle.Fetcher.HostFetchResult doHttpFetch(org.apache.tez.runtime.library.common.shuffle.Fetcher.CachingCallBack callback)

setupLocalDiskFetch

protected org.apache.tez.runtime.library.common.shuffle.Fetcher.HostFetchResult setupLocalDiskFetch()

getTezIndexRecord

protected TezIndexRecord getTezIndexRecord(InputAttemptIdentifier srcAttemptId,
                                           int partition)
                                    throws IOException

Throws:

IOException

getShuffleInputFileName

protected org.apache.hadoop.fs.Path getShuffleInputFileName(String pathComponent,
                                                            String suffix)
                                                     throws IOException

Throws:

IOException

getPathToAttemptMap

public Map<Fetcher.PathPartition,InputAttemptIdentifier> getPathToAttemptMap()

shutdown

public void shutdown()

hashCode

public int hashCode()

Overrides:

hashCode in class Object

equals

public boolean equals(Object obj)

Overrides:

equals in class Object