Package net.snowflake.ingest.streaming.internal

Class ParquetFlusher

java.lang.Object

net.snowflake.ingest.streaming.internal.ParquetFlusher

All Implemented Interfaces:

Flusher<ParquetChunkData>

public class ParquetFlusher
extends Object
implements Flusher<ParquetChunkData>

Converts ChannelData buffered in RowBuffer to the Parquet format for faster processing.

Nested Class Summary

Nested classes/interfaces inherited from interface net.snowflake.ingest.streaming.internal.Flusher

Flusher.SerializationResult

Constructor Summary

Constructors

Constructor	Description
ParquetFlusher(org.apache.parquet.schema.MessageType schema, boolean enableParquetInternalBuffering, long maxChunkSizeInBytes)	Construct parquet flusher from its schema and set flag that indicates whether Parquet memory optimization is enabled, i.e.

Method Summary

Modifier and Type	Method	Description
Flusher.SerializationResult	serialize(List<net.snowflake.ingest.streaming.internal.ChannelData<ParquetChunkData>> channelsDataPerTable, String filePath)	Serialize buffered rows into the underlying format.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

ParquetFlusher

public ParquetFlusher(org.apache.parquet.schema.MessageType schema,
                      boolean enableParquetInternalBuffering,
                      long maxChunkSizeInBytes)

Construct parquet flusher from its schema and set flag that indicates whether Parquet memory optimization is enabled, i.e. rows will be buffered in internal Parquet buffer.

Method Detail

serialize

public Flusher.SerializationResult serialize(List<net.snowflake.ingest.streaming.internal.ChannelData<ParquetChunkData>> channelsDataPerTable,
                                             String filePath)
                                      throws IOException

Description copied from interface: Flusher

Serialize buffered rows into the underlying format.

Specified by:

serialize in interface Flusher<ParquetChunkData>

Parameters:

channelsDataPerTable - buffered rows

filePath - file path

Returns:

Flusher.SerializationResult

Throws:

IOException