parquet.hadoop

Class ParquetInputSplit

java.lang.Object

org.apache.hadoop.mapreduce.InputSplit

org.apache.hadoop.mapreduce.lib.input.FileSplit

parquet.hadoop.ParquetInputSplit

All Implemented Interfaces:

org.apache.hadoop.io.Writable

@InterfaceAudience.Private
public class ParquetInputSplit
extends org.apache.hadoop.mapreduce.lib.input.FileSplit
implements org.apache.hadoop.io.Writable

An input split for the Parquet format It contains the information to read one block of the file. This class is private to the ParquetInputFormat. Backward compatibility is not maintained.

Author:

Julien Le Dem

Constructor Summary

Constructors

Constructor and Description
ParquetInputSplit() Writables must have a parameterless constructor
ParquetInputSplit(org.apache.hadoop.fs.Path file, long start, long end, long length, String[] hosts, long[] rowGroupOffsets, String requestedSchema, Map<String,String> readSupportMetadata)
ParquetInputSplit(org.apache.hadoop.fs.Path path, long start, long length, String[] hosts, List<BlockMetaData> blocks, String requestedSchema, String fileSchema, Map<String,String> extraMetadata, Map<String,String> readSupportMetadata) Deprecated.

Method Summary

Methods

Modifier and Type	Method and Description
void	readFields(DataInput hin)
String	toString()
void	write(DataOutput hout)

Methods inherited from class org.apache.hadoop.mapreduce.lib.input.FileSplit

getLength, getLocations, getPath, getStart

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Constructor Detail

ParquetInputSplit

public ParquetInputSplit()

Writables must have a parameterless constructor

ParquetInputSplit

@Deprecated
public ParquetInputSplit(org.apache.hadoop.fs.Path path,
                            long start,
                            long length,
                            String[] hosts,
                            List<BlockMetaData> blocks,
                            String requestedSchema,
                            String fileSchema,
                            Map<String,String> extraMetadata,
                            Map<String,String> readSupportMetadata)

Deprecated.

For compatibility only use ParquetInputSplit(Path, long, long, long, String[], long[], String, Map)

Parameters:

path -

start -

length -

hosts -

blocks -

requestedSchema -

fileSchema -

extraMetadata -

readSupportMetadata -

ParquetInputSplit

public ParquetInputSplit(org.apache.hadoop.fs.Path file,
                 long start,
                 long end,
                 long length,
                 String[] hosts,
                 long[] rowGroupOffsets,
                 String requestedSchema,
                 Map<String,String> readSupportMetadata)

Parameters:

file - the path of the file for that split

start - the start offset in the file

end - the end offset in the file

length - the actual size in bytes that we expect to read

hosts - the hosts with the replicas of this data

rowGroupOffsets - the offsets of the rowgroups selected if loaded on the client

requestedSchema - the user requested schema

readSupportMetadata - metadata from the read support

Method Detail

toString

public String toString()

Overrides:

toString in class org.apache.hadoop.mapreduce.lib.input.FileSplit

readFields

public final void readFields(DataInput hin)
                      throws IOException

Specified by:

readFields in interface org.apache.hadoop.io.Writable

Overrides:

readFields in class org.apache.hadoop.mapreduce.lib.input.FileSplit

Throws:

IOException

write

public final void write(DataOutput hout)
                 throws IOException

Specified by:

write in interface org.apache.hadoop.io.Writable

Overrides:

write in class org.apache.hadoop.mapreduce.lib.input.FileSplit

Throws:

IOException