org.apache.hadoop.streaming.mapreduce

Class StreamXmlRecordReader

java.lang.Object

org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

org.apache.hadoop.streaming.mapreduce.StreamBaseRecordReader

org.apache.hadoop.streaming.mapreduce.StreamXmlRecordReader

All Implemented Interfaces:

Closeable, AutoCloseable

public class StreamXmlRecordReader
extends StreamBaseRecordReader

A way to interpret XML fragments as Mapper input records. Values are XML subtrees delimited by configurable tags. Keys could be the value of a certain attribute in the XML subtree, but this is left to the stream processor application. The name-value properties that StreamXmlRecordReader understands are: String begin (chars marking beginning of record) String end (chars marking end of record) int maxrec (maximum record size) int lookahead(maximum lookahead to sync CDATA) boolean slowmatch

Field Summary

Fields inherited from class org.apache.hadoop.streaming.mapreduce.StreamBaseRecordReader

LOG

Constructor Summary

Constructors

Constructor and Description
StreamXmlRecordReader(org.apache.hadoop.fs.FSDataInputStream in, org.apache.hadoop.mapreduce.lib.input.FileSplit split, org.apache.hadoop.mapreduce.TaskAttemptContext context, org.apache.hadoop.conf.Configuration conf, org.apache.hadoop.fs.FileSystem fs)

Method Summary

Modifier and Type	Method and Description
org.apache.hadoop.io.Text	getCurrentKey()
org.apache.hadoop.io.Text	getCurrentValue()
void	init()
void	initialize(org.apache.hadoop.mapreduce.InputSplit arg0, org.apache.hadoop.mapreduce.TaskAttemptContext arg1)
boolean	next(org.apache.hadoop.io.Text key, org.apache.hadoop.io.Text value) Read a record.
boolean	nextKeyValue()
void	seekNextRecordBoundary() Implementation should seek forward in_ to the first byte of the next record.

Methods inherited from class org.apache.hadoop.streaming.mapreduce.StreamBaseRecordReader

close, createKey, createValue, getPos, getProgress

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

StreamXmlRecordReader

public StreamXmlRecordReader(org.apache.hadoop.fs.FSDataInputStream in,
                             org.apache.hadoop.mapreduce.lib.input.FileSplit split,
                             org.apache.hadoop.mapreduce.TaskAttemptContext context,
                             org.apache.hadoop.conf.Configuration conf,
                             org.apache.hadoop.fs.FileSystem fs)
                      throws IOException

Throws:

IOException

Method Detail

init

public final void init()
                throws IOException

Throws:

IOException

next

public boolean next(org.apache.hadoop.io.Text key,
                    org.apache.hadoop.io.Text value)
             throws IOException

Description copied from class: StreamBaseRecordReader

Read a record. Implementation should call numRecStats at the end

Specified by:

next in class StreamBaseRecordReader

Throws:

IOException

seekNextRecordBoundary

public void seekNextRecordBoundary()
                            throws IOException

Description copied from class: StreamBaseRecordReader

Implementation should seek forward in_ to the first byte of the next record. The initial byte offset in the stream is arbitrary.

Specified by:

seekNextRecordBoundary in class StreamBaseRecordReader

Throws:

IOException

getCurrentKey

public org.apache.hadoop.io.Text getCurrentKey()
                                        throws IOException,
                                               InterruptedException

Specified by:

getCurrentKey in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Throws:

IOException

InterruptedException

getCurrentValue

public org.apache.hadoop.io.Text getCurrentValue()
                                          throws IOException,
                                                 InterruptedException

Specified by:

getCurrentValue in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Throws:

IOException

InterruptedException

initialize

public void initialize(org.apache.hadoop.mapreduce.InputSplit arg0,
                       org.apache.hadoop.mapreduce.TaskAttemptContext arg1)
                throws IOException,
                       InterruptedException

Specified by:

initialize in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Throws:

IOException

InterruptedException

nextKeyValue

public boolean nextKeyValue()
                     throws IOException,
                            InterruptedException

Specified by:

nextKeyValue in class org.apache.hadoop.mapreduce.RecordReader<org.apache.hadoop.io.Text,org.apache.hadoop.io.Text>

Throws:

IOException

InterruptedException