com.facebook.presto.hive.s3select

Class S3SelectLineRecordReader

java.lang.Object

com.facebook.presto.hive.s3select.S3SelectLineRecordReader

All Implemented Interfaces:

org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Direct Known Subclasses:

S3SelectCsvRecordReader, S3SelectJsonRecordReader

@ThreadSafe
public abstract class S3SelectLineRecordReader
extends Object
implements org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	S3SelectLineRecordReader.UnrecoverableS3OperationException This exception is for stopping retries for S3 Select calls that shouldn't be retried.

Field Summary

Fields

Modifier and Type	Field and Description
protected org.apache.hadoop.io.compress.CompressionCodecFactory	compressionCodecFactory
protected String	lineDelimiter

Method Summary

Modifier and Type	Method and Description
protected abstract com.amazonaws.services.s3.model.InputSerialization	buildInputSerialization()
protected abstract com.amazonaws.services.s3.model.OutputSerialization	buildOutputSerialization()
com.amazonaws.services.s3.model.SelectObjectContentRequest	buildSelectObjectRequest(String query, org.apache.hadoop.fs.Path path)
void	close()
org.apache.hadoop.io.LongWritable	createKey()
org.apache.hadoop.io.Text	createValue()
protected com.amazonaws.services.s3.model.CompressionType	getCompressionType()
protected com.amazonaws.services.s3.model.CompressionType	getCompressionType(org.apache.hadoop.fs.Path path)
long	getPos()
float	getProgress()
protected Properties	getSchema()
boolean	next(org.apache.hadoop.io.LongWritable key, org.apache.hadoop.io.Text value)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

compressionCodecFactory

protected final org.apache.hadoop.io.compress.CompressionCodecFactory compressionCodecFactory

lineDelimiter

protected final String lineDelimiter

Method Detail

buildInputSerialization

protected abstract com.amazonaws.services.s3.model.InputSerialization buildInputSerialization()

buildOutputSerialization

protected abstract com.amazonaws.services.s3.model.OutputSerialization buildOutputSerialization()

getSchema

protected Properties getSchema()

getCompressionType

protected com.amazonaws.services.s3.model.CompressionType getCompressionType()

buildSelectObjectRequest

public com.amazonaws.services.s3.model.SelectObjectContentRequest buildSelectObjectRequest(String query,
                                                                                           org.apache.hadoop.fs.Path path)

getCompressionType

protected com.amazonaws.services.s3.model.CompressionType getCompressionType(org.apache.hadoop.fs.Path path)

next

public boolean next(org.apache.hadoop.io.LongWritable key,
                    org.apache.hadoop.io.Text value)
             throws IOException

Specified by:

next in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Throws:

IOException

createKey

public org.apache.hadoop.io.LongWritable createKey()

Specified by:

createKey in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

createValue

public org.apache.hadoop.io.Text createValue()

Specified by:

createValue in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

getPos

public long getPos()

Specified by:

getPos in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

close

public void close()
           throws IOException

Specified by:

close in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Throws:

IOException

getProgress

public float getProgress()

Specified by:

getProgress in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>