water.parser.parquet

Class ParquetParser

java.lang.Object

water.Iced

water.parser.Parser

water.parser.parquet.ParquetParser

All Implemented Interfaces:

java.io.Externalizable, java.io.Serializable, java.lang.Cloneable, water.Freezable

public class ParquetParser
extends water.parser.Parser

Parquet parser for H2O distributed parsing subsystem.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	ParquetParser.ParquetParseSetup

Field Summary

Fields inherited from class water.parser.Parser

_jobKey, _keepColumns, _setup, CHAR_DECIMAL_SEP, CHAR_ESCAPE, CHAR_SEPARATOR, COND_QUOTE, COND_QUOTED_NUMBER_END, COND_QUOTED_TOKEN, EOL, EXPECT_COND_LF, HASHTAG, LARGEST_DIGIT_NUMBER, NUMBER, NUMBER_END, NUMBER_EXP, NUMBER_EXP_START, NUMBER_FRACTION, NUMBER_SKIP, NUMBER_SKIP_NO_DOT, POSSIBLE_CURRENCY, POSSIBLE_EMPTY_LINE, POSSIBLE_ESCAPED_QUOTE, SEPARATOR_OR_EOL, SKIP_LINE, STRING, STRING_END, TOKEN, WHITESPACE_BEFORE_TOKEN

Method Summary

Modifier and Type	Method and Description
static byte[]	correctTypeConversions(water.fvec.ByteVec vec, byte[] requestedTypes) Overrides unsupported type conversions/mappings specified by the user.
static ParquetParser.ParquetParseSetup	guessDataSetup(water.fvec.ByteVec vec, ParquetParser.ParquetParseSetup ps, boolean[] keepcolumns)
static ParquetParser.ParquetParseSetup	guessFormatSetup(water.fvec.ByteVec vec, byte[] bits)
protected water.parser.ParseWriter	parseChunk(int cidx, water.parser.ParseReader din, water.parser.ParseWriter dout)
protected water.parser.StreamParseWriter	sequentialParse(water.fvec.Vec vec, water.parser.StreamParseWriter dout)

Methods inherited from class water.parser.Parser

fileHasHeader, isEOL, streamParse, streamParseZip

Methods inherited from class water.Iced

asBytes, clone, copyOver, frozenType, read, readExternal, readJSON, reloadFromBytes, toJsonString, write, writeExternal, writeJSON

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Method Detail

sequentialParse

protected final water.parser.StreamParseWriter sequentialParse(water.fvec.Vec vec,
                                                               water.parser.StreamParseWriter dout)

Overrides:

sequentialParse in class water.parser.Parser

parseChunk

protected final water.parser.ParseWriter parseChunk(int cidx,
                                                    water.parser.ParseReader din,
                                                    water.parser.ParseWriter dout)

Specified by:

parseChunk in class water.parser.Parser

guessFormatSetup

public static ParquetParser.ParquetParseSetup guessFormatSetup(water.fvec.ByteVec vec,
                                                               byte[] bits)

guessDataSetup

public static ParquetParser.ParquetParseSetup guessDataSetup(water.fvec.ByteVec vec,
                                                             ParquetParser.ParquetParseSetup ps,
                                                             boolean[] keepcolumns)

correctTypeConversions

public static byte[] correctTypeConversions(water.fvec.ByteVec vec,
                                            byte[] requestedTypes)

Overrides unsupported type conversions/mappings specified by the user.

Parameters:

vec - byte vec holding bin\ary parquet data

requestedTypes - user-specified target types

Returns:

corrected types