edu.umd.cloud9.collection.wikipedia

Class WikipediaDocnoMapping

java.lang.Object

edu.umd.cloud9.collection.wikipedia.WikipediaDocnoMapping

All Implemented Interfaces:

DocnoMapping

public class WikipediaDocnoMapping
extends Object
implements DocnoMapping

Provides a mapping between Wikipedia internal ids (docids) and sequentially-numbered ints (docnos).

The main of this class provides a simple program for accessing docno mappings. Command-line arguments are as follows:

• getDocno, getDocid: get docno from docid or, get docid from docno
• [mappings-file]: the mappings file
• [docid/docno]: the docid or docno (optional)

Author:

Jimmy Lin

Nested Class Summary

Nested classes/interfaces inherited from interface edu.umd.cloud9.collection.DocnoMapping

DocnoMapping.Builder, DocnoMapping.BuilderUtils, DocnoMapping.DefaultBuilderOptions

Constructor Summary

Constructors

Constructor and Description
WikipediaDocnoMapping() Creates a WikipediaDocnoMapping object

Method Summary

Methods

Modifier and Type	Method and Description
DocnoMapping.Builder	getBuilder() Returns the builder for this mapping.
String	getDocid(int docno) Returns the docid for a particular docno.
int	getDocno(String docid) Returns the docno for a particular docid.
void	loadMapping(org.apache.hadoop.fs.Path p, org.apache.hadoop.fs.FileSystem fs) Loads a mapping file.
static void	main(String[] args) Simple program the provides access to the docno/docid mappings.
static int[]	readDocnoMappingData(org.apache.hadoop.fs.Path p, org.apache.hadoop.fs.FileSystem fs) Reads a mappings file into memory.
static void	writeDocnoMappingData(org.apache.hadoop.fs.FileSystem fs, String inputFile, int n, String outputFile) Creates a mappings file from the contents of a flat text file containing docid to docno mappings.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

WikipediaDocnoMapping

public WikipediaDocnoMapping()

Creates a WikipediaDocnoMapping object

Method Detail

getDocno

public int getDocno(String docid)

Description copied from interface: DocnoMapping

Returns the docno for a particular docid.

Specified by:

getDocno in interface DocnoMapping

Parameters:

docid - the docid

Returns:

the docno for the docid

getDocid

public String getDocid(int docno)

Description copied from interface: DocnoMapping

Returns the docid for a particular docno.

Specified by:

getDocid in interface DocnoMapping

Parameters:

docno - the docno

Returns:

the docid for the docno

loadMapping

public void loadMapping(org.apache.hadoop.fs.Path p,
               org.apache.hadoop.fs.FileSystem fs)
                 throws IOException

Description copied from interface: DocnoMapping

Loads a mapping file.

Specified by:

loadMapping in interface DocnoMapping

Parameters:

p - path to the mappings file

fs - reference to the FileSystem

Throws:

IOException

writeDocnoMappingData

public static void writeDocnoMappingData(org.apache.hadoop.fs.FileSystem fs,
                         String inputFile,
                         int n,
                         String outputFile)
                                  throws IOException

Creates a mappings file from the contents of a flat text file containing docid to docno mappings. This method is used by WikipediaDocnoMappingBuilder internally.

Parameters:

inputFile - flat text file containing docid to docno mappings

outputFile - output mappings file

Throws:

IOException

readDocnoMappingData

public static int[] readDocnoMappingData(org.apache.hadoop.fs.Path p,
                         org.apache.hadoop.fs.FileSystem fs)
                                  throws IOException

Reads a mappings file into memory.

Parameters:

p - path to the mappings file

fs - appropriate FileSystem

Returns:

an array of docids; the index position of each docid is its docno

Throws:

IOException

getBuilder

public DocnoMapping.Builder getBuilder()

Description copied from interface: DocnoMapping

Returns the builder for this mapping.

Specified by:

getBuilder in interface DocnoMapping

Returns:

builder for this mapping

main

public static void main(String[] args)
                 throws IOException

Simple program the provides access to the docno/docid mappings.

Parameters:

args - command-line arguments

Throws:

IOException