Package org.apache.druid.data.input
Class TextReader<T>
- java.lang.Object
-
- org.apache.druid.data.input.IntermediateRowParsingReader<T>
-
- org.apache.druid.data.input.TextReader<T>
-
- All Implemented Interfaces:
InputEntityReader
- Direct Known Subclasses:
TextReader.Bytes,TextReader.Strings
public abstract class TextReader<T> extends IntermediateRowParsingReader<T>
AbstractInputEntityReaderfor text format readers such as CSV or JSON.
-
-
Nested Class Summary
Nested Classes Modifier and Type Class Description static classTextReader.Bytesstatic classTextReader.Strings
-
Constructor Summary
Constructors Modifier Constructor Description protectedTextReader(InputRowSchema inputRowSchema, InputEntity source)
-
Method Summary
All Methods Static Methods Instance Methods Abstract Methods Concrete Methods Modifier and Type Method Description static RowSignaturefindOrCreateInputRowSignature(List<String> parsedLine)InputRowSchemagetInputRowSchema()abstract intgetNumHeaderLinesToSkip()Returns the number of header lines to skip.CloseableIteratorWithMetadata<T>intermediateRowIteratorWithMetadata()Same asintermediateRowIterator, but it also contains the metadata such as the line number to generate more informativeParseException.protected abstract CloseableIterator<T>makeSourceIterator(InputStream in)abstract booleanneedsToProcessHeaderLine()Returns true if the file format needs to process a header line.abstract List<InputRow>parseInputRows(T intermediateRow)Parses the given line into a list ofInputRows.abstract voidprocessHeaderLine(T line)Processes a header line.protected InputEntitysource()-
Methods inherited from class org.apache.druid.data.input.IntermediateRowParsingReader
intermediateRowAsString, intermediateRowIterator, read, sample, toMap
-
-
-
-
Constructor Detail
-
TextReader
protected TextReader(InputRowSchema inputRowSchema, InputEntity source)
-
-
Method Detail
-
getInputRowSchema
public InputRowSchema getInputRowSchema()
-
intermediateRowIteratorWithMetadata
public CloseableIteratorWithMetadata<T> intermediateRowIteratorWithMetadata() throws IOException
Description copied from class:IntermediateRowParsingReaderSame asintermediateRowIterator, but it also contains the metadata such as the line number to generate more informativeParseException.- Overrides:
intermediateRowIteratorWithMetadatain classIntermediateRowParsingReader<T>- Throws:
IOException
-
source
protected InputEntity source()
- Overrides:
sourcein classIntermediateRowParsingReader<T>- Returns:
- InputEntity which the implementation is reading from. Useful in generating informative
ParseExceptions. For example, in case ofFileEntity, file name containing erroneous records or in case ofHttpEntity, the endpoint containing the erroneous data can be attached to the error message
-
parseInputRows
public abstract List<InputRow> parseInputRows(T intermediateRow) throws IOException, ParseException
Parses the given line into a list ofInputRows. Note that some file formats can explode a single line of input into multiple inputRows. This method will be called aftergetNumHeaderLinesToSkip()andprocessHeaderLine(T).- Specified by:
parseInputRowsin classIntermediateRowParsingReader<T>- Throws:
ParseException- if it cannot parse the given intermediateRow properlyIOException
-
getNumHeaderLinesToSkip
public abstract int getNumHeaderLinesToSkip()
Returns the number of header lines to skip.processHeaderLine(T)will be called as many times as the returned number.
-
needsToProcessHeaderLine
public abstract boolean needsToProcessHeaderLine()
Returns true if the file format needs to process a header line. This method will be called after skipping lines as many asgetNumHeaderLinesToSkip().
-
processHeaderLine
public abstract void processHeaderLine(T line) throws IOException
Processes a header line. This will be called ifneedsToProcessHeaderLine()= true.- Throws:
IOException
-
makeSourceIterator
protected abstract CloseableIterator<T> makeSourceIterator(InputStream in)
-
findOrCreateInputRowSignature
public static RowSignature findOrCreateInputRowSignature(List<String> parsedLine)
-
-