edu.umd.cloud9.collection.clue

Class ClueWarcInputFormat

java.lang.Object

org.apache.hadoop.mapred.FileInputFormat<org.apache.hadoop.io.LongWritable,ClueWarcRecord>

edu.umd.cloud9.collection.clue.ClueWarcInputFormat

All Implemented Interfaces:

org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.LongWritable,ClueWarcRecord>

public class ClueWarcInputFormat
extends org.apache.hadoop.mapred.FileInputFormat<org.apache.hadoop.io.LongWritable,ClueWarcRecord>

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ClueWarcInputFormat.ClueWarcRecordReader

Field Summary

Fields inherited from class org.apache.hadoop.mapred.FileInputFormat

LOG

Constructor Summary

Constructors

Constructor and Description
ClueWarcInputFormat()

Method Summary

Methods

Modifier and Type	Method and Description
org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,ClueWarcRecord>	getRecordReader(org.apache.hadoop.mapred.InputSplit split, org.apache.hadoop.mapred.JobConf conf, org.apache.hadoop.mapred.Reporter reporter) Just return the record reader
protected boolean	isSplitable(org.apache.hadoop.fs.FileSystem fs, org.apache.hadoop.fs.Path filename) Don't allow the files to be split!

Methods inherited from class org.apache.hadoop.mapred.FileInputFormat

addInputPath, addInputPaths, computeSplitSize, getBlockIndex, getInputPathFilter, getInputPaths, getSplitHosts, getSplits, listStatus, setInputPathFilter, setInputPaths, setInputPaths, setMinSplitSize

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ClueWarcInputFormat

public ClueWarcInputFormat()

Method Detail

isSplitable

protected boolean isSplitable(org.apache.hadoop.fs.FileSystem fs,
                  org.apache.hadoop.fs.Path filename)

Don't allow the files to be split!

Overrides:

isSplitable in class org.apache.hadoop.mapred.FileInputFormat<org.apache.hadoop.io.LongWritable,ClueWarcRecord>

getRecordReader

public org.apache.hadoop.mapred.RecordReader<org.apache.hadoop.io.LongWritable,ClueWarcRecord> getRecordReader(org.apache.hadoop.mapred.InputSplit split,
                                                                                                      org.apache.hadoop.mapred.JobConf conf,
                                                                                                      org.apache.hadoop.mapred.Reporter reporter)
                                                                                                        throws IOException

Just return the record reader

Specified by:

getRecordReader in interface org.apache.hadoop.mapred.InputFormat<org.apache.hadoop.io.LongWritable,ClueWarcRecord>

Specified by:

getRecordReader in class org.apache.hadoop.mapred.FileInputFormat<org.apache.hadoop.io.LongWritable,ClueWarcRecord>

Throws:

IOException