parquet.hadoop

Class ParquetInputSplit

java.lang.Object

org.apache.hadoop.mapreduce.InputSplit

org.apache.hadoop.mapreduce.lib.input.FileSplit

parquet.hadoop.ParquetInputSplit

All Implemented Interfaces:

org.apache.hadoop.io.Writable

public class ParquetInputSplit
extends org.apache.hadoop.mapreduce.lib.input.FileSplit
implements org.apache.hadoop.io.Writable

An input split for the Parquet format It contains the information to read one block of the file.

Author:

Julien Le Dem

Constructor Summary

Constructors

Constructor and Description
ParquetInputSplit() Writables must have a parameterless constructor
ParquetInputSplit(org.apache.hadoop.fs.Path path, long start, long length, String[] hosts, List<BlockMetaData> blocks, String requestedSchema, String fileSchema, Map<String,String> extraMetadata, Map<String,String> readSupportMetadata) Used by ParquetInputFormat.getSplits(org.apache.hadoop.mapreduce.JobContext)

Method Summary

Methods

Modifier and Type	Method and Description
List<BlockMetaData>	getBlocks()
Map<String,String>	getExtraMetadata()
String	getFileSchema()
Map<String,String>	getReadSupportMetadata()
String	getRequestedSchema()
void	readFields(DataInput in)
String	toString()
void	write(DataOutput out)

Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileSplit

getLength, getLocations, getPath, getStart

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ParquetInputSplit

public ParquetInputSplit()

Writables must have a parameterless constructor

ParquetInputSplit

public ParquetInputSplit(org.apache.hadoop.fs.Path path,
                 long start,
                 long length,
                 String[] hosts,
                 List<BlockMetaData> blocks,
                 String requestedSchema,
                 String fileSchema,
                 Map<String,String> extraMetadata,
                 Map<String,String> readSupportMetadata)

Used by ParquetInputFormat.getSplits(org.apache.hadoop.mapreduce.JobContext)

Parameters:

path - the path to the file

start - the offset of the block in the file

length - the size of the block in the file

hosts - the hosts where this block can be found

blocks - the block meta data (Columns locations)

schema - the file schema

readSupportClass - the class used to materialize records

requestedSchema - the requested schema for materialization

fileSchema - the schema of the file

extraMetadata - the app specific meta data in the file

readSupportMetadata - the read support specific metadata

Method Detail

getBlocks

public List<BlockMetaData> getBlocks()

Returns:

the block meta data

getRequestedSchema

public String getRequestedSchema()

Returns:

the requested schema

getFileSchema

public String getFileSchema()

Returns:

the file schema

getExtraMetadata

public Map<String,String> getExtraMetadata()

Returns:

app specific metadata from the file

getReadSupportMetadata

public Map<String,String> getReadSupportMetadata()

Returns:

app specific metadata provided by the read support in the init phase

readFields

public void readFields(DataInput in)
                throws IOException

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Overrides:

readFields in class org.apache.hadoop.mapreduce.lib.input.FileSplit

Throws:

IOException

write

public void write(DataOutput out)
           throws IOException

Specified by:

write in interface org.apache.hadoop.io.Writable

Overrides:

write in class org.apache.hadoop.mapreduce.lib.input.FileSplit

Throws:

IOException

toString

public String toString()

Overrides:

toString in class org.apache.hadoop.mapreduce.lib.input.FileSplit