Package org.apache.druid.data.input

Class TextReader<T>

java.lang.Object

org.apache.druid.data.input.IntermediateRowParsingReader<T>

org.apache.druid.data.input.TextReader<T>

All Implemented Interfaces:

InputEntityReader

Direct Known Subclasses:

TextReader.Bytes, TextReader.Strings

public abstract class TextReader<T>
extends IntermediateRowParsingReader<T>

Abstract InputEntityReader for text format readers such as CSV or JSON.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	TextReader.Bytes
static class	TextReader.Strings

Constructor Summary

Constructors

Modifier	Constructor	Description
protected	TextReader(InputRowSchema inputRowSchema, InputEntity source)

Method Summary

Modifier and Type	Method	Description
static RowSignature	findOrCreateInputRowSignature(List<String> parsedLine)
InputRowSchema	getInputRowSchema()
abstract int	getNumHeaderLinesToSkip()	Returns the number of header lines to skip.
CloseableIteratorWithMetadata<T>	intermediateRowIteratorWithMetadata()	Same as intermediateRowIterator, but it also contains the metadata such as the line number to generate more informative ParseException.
protected abstract CloseableIterator<T>	makeSourceIterator(InputStream in)
abstract boolean	needsToProcessHeaderLine()	Returns true if the file format needs to process a header line.
abstract List<InputRow>	parseInputRows(T intermediateRow)	Parses the given line into a list of InputRows.
abstract void	processHeaderLine(T line)	Processes a header line.
protected InputEntity	source()

Methods inherited from class org.apache.druid.data.input.IntermediateRowParsingReader

intermediateRowAsString, intermediateRowIterator, read, sample, toMap

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

TextReader

protected TextReader(InputRowSchema inputRowSchema,
                     InputEntity source)

Method Detail

getInputRowSchema

public InputRowSchema getInputRowSchema()

intermediateRowIteratorWithMetadata

public CloseableIteratorWithMetadata<T> intermediateRowIteratorWithMetadata()
                                                                     throws IOException

Description copied from class: IntermediateRowParsingReader

Same as intermediateRowIterator, but it also contains the metadata such as the line number to generate more informative ParseException.

Overrides:

intermediateRowIteratorWithMetadata in class IntermediateRowParsingReader<T>

Throws:

IOException

source

protected InputEntity source()

Overrides:

source in class IntermediateRowParsingReader<T>

Returns:

InputEntity which the implementation is reading from. Useful in generating informative ParseExceptions. For example, in case of FileEntity, file name containing erroneous records or in case of HttpEntity, the endpoint containing the erroneous data can be attached to the error message

parseInputRows

public abstract List<InputRow> parseInputRows(T intermediateRow)
                                       throws IOException,
                                              ParseException

Parses the given line into a list of InputRows. Note that some file formats can explode a single line of input into multiple inputRows. This method will be called after getNumHeaderLinesToSkip() and processHeaderLine(T).

Specified by:

parseInputRows in class IntermediateRowParsingReader<T>

Throws:

ParseException - if it cannot parse the given intermediateRow properly

IOException

getNumHeaderLinesToSkip

public abstract int getNumHeaderLinesToSkip()

Returns the number of header lines to skip. processHeaderLine(T) will be called as many times as the returned number.

needsToProcessHeaderLine

public abstract boolean needsToProcessHeaderLine()

Returns true if the file format needs to process a header line. This method will be called after skipping lines as many as getNumHeaderLinesToSkip().

processHeaderLine

public abstract void processHeaderLine(T line)
                                throws IOException

Processes a header line. This will be called if needsToProcessHeaderLine() = true.

Throws:

IOException

makeSourceIterator

protected abstract CloseableIterator<T> makeSourceIterator(InputStream in)

findOrCreateInputRowSignature

public static RowSignature findOrCreateInputRowSignature(List<String> parsedLine)