Package org.apache.druid.data.input

Interface InputEntity

All Known Implementing Classes:

ByteEntity, BytesCountingInputEntity, FileEntity, HttpEntity, RetryingInputEntity

public interface InputEntity

InputEntity abstracts an input entity and knows how to read bytes from the given entity. Since the implementations of this interface assume that the given entity is not empty, the InputSources should not create InputEntities for empty entities.

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static interface	InputEntity.CleanableFile	CleanableFile is the result type of fetch(java.io.File, byte[]).

Field Summary

Fields

Modifier and Type	Field	Description
static int	DEFAULT_FETCH_BUFFER_SIZE
static int	DEFAULT_MAX_NUM_FETCH_TRIES
static Logger	LOG

Method Summary

Modifier and Type	Method	Description
default InputEntity.CleanableFile	fetch(File temporaryDirectory, byte[] fetchBuffer)	Fetches the input entity into the local storage.
default com.google.common.base.Predicate<Throwable>	getRetryCondition()	Returns a retry condition that the caller should retry on.
URI	getUri()	Returns an URI to identify the input entity.
InputStream	open()	Opens an InputStream on the input entity directly.

Field Detail

LOG

static final Logger LOG

DEFAULT_FETCH_BUFFER_SIZE

static final int DEFAULT_FETCH_BUFFER_SIZE

See Also:

Constant Field Values

DEFAULT_MAX_NUM_FETCH_TRIES

static final int DEFAULT_MAX_NUM_FETCH_TRIES

See Also:

Constant Field Values

Method Detail

getUri

@Nullable
URI getUri()

Returns an URI to identify the input entity. Implementations can return null if they don't have an unique URI.

open

InputStream open()
          throws IOException

Opens an InputStream on the input entity directly. This is the basic way to read the given entity. This method may be called multiple times to re-read the data from the entity.

Throws:

IOException

See Also:

fetch(java.io.File, byte[])

fetch

default InputEntity.CleanableFile fetch(File temporaryDirectory,
                                        byte[] fetchBuffer)
                                 throws IOException

Fetches the input entity into the local storage. This method might be preferred instead of open(), for example - InputFormat requires expensive random access on remote storage. - Holding a connection until you consume the entire InputStream is expensive.

Parameters:

temporaryDirectory - to store temp data. This directory will be removed automatically once the task finishes.

fetchBuffer - is used to fetch remote entity into local storage.

Throws:

IOException

See Also:

FileUtils.copyLarge(T, org.apache.druid.data.input.impl.prefetch.ObjectOpenFunction<T>, java.io.File, byte[], com.google.common.base.Predicate<java.lang.Throwable>, int, java.lang.String)

getRetryCondition

default com.google.common.base.Predicate<Throwable> getRetryCondition()

Returns a retry condition that the caller should retry on. The returned condition should be used when reading data from this InputEntity such as in fetch(java.io.File, byte[]) or RetryingInputEntity.