org.apache.druid.data.input

Interface FiniteFirehoseFactory<T extends InputRowParser,S>

Type Parameters:

T - parser type

S - input split type

All Superinterfaces:

FirehoseFactory<T>

All Known Implementing Classes:

AbstractTextFilesFirehoseFactory, PrefetchableTextFilesFirehoseFactory

public interface FiniteFirehoseFactory<T extends InputRowParser,S>
extends FirehoseFactory<T>

FiniteFirehoseFactory designed for batch processing. Its implementations assume that the amount of inputs is limited.

Method Summary

Modifier and Type	Method and Description
int	getNumSplits() Returns number of splits returned by getSplits().
Stream<InputSplit<S>>	getSplits() Returns a Stream for InputSplits.
default boolean	isSplittable() Returns true if this FiniteFirehoseFactory supports parallel batch indexing.
FiniteFirehoseFactory<T,S>	withSplit(InputSplit<S> split) Returns the same FiniteFirehoseFactory but with the given InputSplit.

Methods inherited from interface org.apache.druid.data.input.FirehoseFactory

connect, connect

Method Detail

isSplittable

default boolean isSplittable()

Returns true if this FiniteFirehoseFactory supports parallel batch indexing.

Specified by:

isSplittable in interface FirehoseFactory<T extends InputRowParser>

getSplits

Stream<InputSplit<S>> getSplits()
                         throws IOException

Returns a Stream for InputSplits. In parallel batch indexing, each InputSplit is processed by a sub task. Listing splits may cause high overhead in some implementations. In this case, InputSplits should be listed lazily so that the listing overhead could be amortized.

Throws:

IOException

getNumSplits

int getNumSplits()
          throws IOException

Returns number of splits returned by getSplits().

Throws:

IOException

withSplit

FiniteFirehoseFactory<T,S> withSplit(InputSplit<S> split)

Returns the same FiniteFirehoseFactory but with the given InputSplit. The returned FiniteFirehoseFactory is used by sub tasks in parallel batch indexing.