Package org.apache.pinot.segment.local.segment.readers

Class MultiplePinotSegmentRecordReader

java.lang.Object

org.apache.pinot.segment.local.segment.readers.MultiplePinotSegmentRecordReader

All Implemented Interfaces:

Closeable, Serializable, AutoCloseable, RecordReader

public class MultiplePinotSegmentRecordReader
extends Object
implements RecordReader

Record reader for multiple pinot segments.

See Also:

Serialized Form

Constructor Summary

Constructors

Constructor	Description
MultiplePinotSegmentRecordReader(List<File> indexDirs)	Read records using the passed in schema from multiple pinot segments.
MultiplePinotSegmentRecordReader(List<File> indexDirs, Set<String> fieldsToRead, List<String> sortOrder)	Read records using the passed in schema and in the order of sorted column from multiple pinot segments.

Method Summary

Modifier and Type	Method	Description
void	close()
boolean	hasNext()
void	init(File dataFile, Set<String> fieldsToRead, RecordReaderConfig recordReaderConfig)
GenericRow	next()
GenericRow	next(GenericRow reuse)
void	rewind()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MultiplePinotSegmentRecordReader

public MultiplePinotSegmentRecordReader(List<File> indexDirs)
                                 throws Exception

Read records using the passed in schema from multiple pinot segments.

Passed in schema must be a subset of the segment schema.

Parameters:

indexDirs - a list of input segment directory paths

Throws:

Exception

MultiplePinotSegmentRecordReader

public MultiplePinotSegmentRecordReader(List<File> indexDirs,
                                        @Nullable
                                        Set<String> fieldsToRead,
                                        @Nullable
                                        List<String> sortOrder)
                                 throws Exception

Read records using the passed in schema and in the order of sorted column from multiple pinot segments.

Passed in schema must be a subset of the segment schema.

If sort order is not specified, it will not attempt to sort the segments and simply concatenate the records from muiltiple segments.

Parameters:

indexDirs - a list of input paths for the segment indices

fieldsToRead - if null or empty, reads all fields

sortOrder - a list of column names that represent the sorting order

Throws:

Exception

Method Detail

init

public void init(File dataFile,
                 Set<String> fieldsToRead,
                 @Nullable
                 RecordReaderConfig recordReaderConfig)

Specified by:

init in interface RecordReader

hasNext

public boolean hasNext()

Specified by:

hasNext in interface RecordReader

next

public GenericRow next()

Specified by:

next in interface RecordReader

next

public GenericRow next(GenericRow reuse)

Specified by:

next in interface RecordReader

rewind

public void rewind()

Specified by:

rewind in interface RecordReader

close

public void close()
           throws IOException

Specified by:

close in interface AutoCloseable

Specified by:

close in interface Closeable

Throws:

IOException