Package org.apache.druid.data.input
Interface InputSource
-
- All Known Subinterfaces:
SplittableInputSource<T>,SystemFieldInputSource
- All Known Implementing Classes:
AbstractInputSource,CloudObjectInputSource,CombiningInputSource,HttpInputSource,InlineInputSource,LocalInputSource
public interface InputSourceInputSource abstracts the storage system where input data is stored. It creates anInputSourceReaderto read data from the given input source. The most common use case would be:InputSourceReader reader = inputSource.reader(); try (CloseableIterator<InputRow> iterator = reader.read()) { while (iterator.hasNext()) { InputRow row = iterator.next(); processRow(row); } }
-
-
Field Summary
Fields Modifier and Type Field Description static StringTYPE_PROPERTY
-
Method Summary
All Methods Instance Methods Abstract Methods Default Methods Modifier and Type Method Description default Set<String>getTypes()The types of input sources uses.booleanisSplittable()Returns true if this inputSource can be processed in parallel using ParallelIndexSupervisorTask.booleanneedsFormat()Returns true if this inputSource supports differentInputFormats.InputSourceReaderreader(InputRowSchema inputRowSchema, InputFormat inputFormat, File temporaryDirectory)Creates anInputSourceReader.
-
-
-
Field Detail
-
TYPE_PROPERTY
static final String TYPE_PROPERTY
- See Also:
- Constant Field Values
-
-
Method Detail
-
isSplittable
boolean isSplittable()
Returns true if this inputSource can be processed in parallel using ParallelIndexSupervisorTask. It must be castable to SplittableInputSource and the various SplittableInputSource methods must work as documented.
-
needsFormat
boolean needsFormat()
Returns true if this inputSource supports differentInputFormats. Some inputSources such asLocalInputSourcecan store files of any format. These storage types require anInputFormatto be passed so thatInputSourceReadercan parse data properly. However, some storage types have a fixed format. For example, druid inputSource always reads segments. These inputSources should return false for this method.
-
reader
InputSourceReader reader(InputRowSchema inputRowSchema, @Nullable InputFormat inputFormat, File temporaryDirectory)
Creates anInputSourceReader.- Parameters:
inputRowSchema- forInputRowinputFormat- to parse data. It can be null ifneedsFormat()= truetemporaryDirectory- to store temp data. It will be cleaned up automatically once the task is finished.
-
-