org.apache.hadoop.hive.ql.io.esriJson

Class UnenclosedBaseJsonRecordReader

java.lang.Object

org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

org.apache.hadoop.hive.ql.io.esriJson.UnenclosedBaseJsonRecordReader

All Implemented Interfaces:

Closeable, AutoCloseable, org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Direct Known Subclasses:

UnenclosedEsriJsonRecordReader, UnenclosedGeoJsonRecordReader

public abstract class UnenclosedBaseJsonRecordReader
extends org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>
implements org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Enumerates records from an Unenclosed JSON file - use either Esri JSON or GeoJSON subclass

Field Summary

Fields

Modifier and Type	Field and Description
protected long	end
protected boolean	firstBraceConsumed
protected BufferedReader	inputReader
protected org.apache.hadoop.io.LongWritable	mkey
protected org.apache.hadoop.io.Text	mval
protected long	readerPosition
protected long	start

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	UnenclosedBaseJsonRecordReader()
protected	UnenclosedBaseJsonRecordReader(org.apache.hadoop.mapred.InputSplit split, org.apache.hadoop.conf.Configuration conf)

Method Summary

Modifier and Type	Method and Description
void	close()
org.apache.hadoop.io.LongWritable	createKey()
org.apache.hadoop.io.Text	createValue()
protected int	getChar()
org.apache.hadoop.io.LongWritable	getCurrentKey()
org.apache.hadoop.io.Text	getCurrentValue()
protected int	getNonWhite()
long	getPos()
float	getProgress()
void	initialize(org.apache.hadoop.mapreduce.InputSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext taskContext)
protected abstract boolean	moveToRecordStart() Given an arbitrary byte offset into an unenclosed JSON document, find the start of the next record in the document.
boolean	next(org.apache.hadoop.io.LongWritable key, org.apache.hadoop.io.Text value)
boolean	nextKeyValue()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

inputReader

protected BufferedReader inputReader

mkey

protected org.apache.hadoop.io.LongWritable mkey

mval

protected org.apache.hadoop.io.Text mval

readerPosition

protected long readerPosition

start

protected long start

end

protected long end

firstBraceConsumed

protected boolean firstBraceConsumed

Constructor Detail

UnenclosedBaseJsonRecordReader

protected UnenclosedBaseJsonRecordReader()
                                  throws IOException

Throws:

IOException

UnenclosedBaseJsonRecordReader

protected UnenclosedBaseJsonRecordReader(org.apache.hadoop.mapred.InputSplit split,
                                         org.apache.hadoop.conf.Configuration conf)
                                  throws IOException

Throws:

IOException

Method Detail

close

public void close()
           throws IOException

Specified by:

close in interface Closeable

Specified by:

close in interface AutoCloseable

Specified by:

close in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Specified by:

close in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Throws:

IOException

createKey

public org.apache.hadoop.io.LongWritable createKey()

Specified by:

createKey in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

createValue

public org.apache.hadoop.io.Text createValue()

Specified by:

createValue in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

getCurrentKey

public org.apache.hadoop.io.LongWritable getCurrentKey()
                                                throws IOException,
                                                       InterruptedException

Specified by:

getCurrentKey in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Throws:

IOException

InterruptedException

getCurrentValue

public org.apache.hadoop.io.Text getCurrentValue()
                                          throws IOException,
                                                 InterruptedException

Specified by:

getCurrentValue in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Throws:

IOException

InterruptedException

getPos

public long getPos()
            throws IOException

Specified by:

getPos in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Throws:

IOException

getProgress

public float getProgress()
                  throws IOException

Specified by:

getProgress in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Specified by:

getProgress in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Throws:

IOException

initialize

public void initialize(org.apache.hadoop.mapreduce.InputSplit split,
                       org.apache.hadoop.mapreduce.TaskAttemptContext taskContext)
                throws IOException,
                       InterruptedException

Specified by:

initialize in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Throws:

IOException

InterruptedException

next

public boolean next(org.apache.hadoop.io.LongWritable key,
                    org.apache.hadoop.io.Text value)
             throws IOException

Specified by:

next in interface org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Throws:

IOException

nextKeyValue

public boolean nextKeyValue()
                     throws IOException,
                            InterruptedException

Specified by:

nextKeyValue in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.LongWritable,org.apache.hadoop.io.Text>

Throws:

IOException

InterruptedException

getChar

protected int getChar()
               throws IOException

Throws:

IOException

getNonWhite

protected int getNonWhite()
                   throws IOException

Throws:

IOException

moveToRecordStart

protected abstract boolean moveToRecordStart()
                                      throws IOException

Given an arbitrary byte offset into an unenclosed JSON document, find the start of the next record in the document. Discard trailing bytes from the previous record if we happened to seek to the middle of it.

Throws:

IOException