Package org.apache.pinot.spi.ingestion.segment.writer

Interface SegmentWriter

All Superinterfaces:

AutoCloseable, Closeable

public interface SegmentWriter
extends Closeable

An interface to collect records and create a Pinot segment. This interface helps abstract out details regarding segment generation from the caller.

Method Summary

Modifier and Type	Method	Description
void	collect(GenericRow row)	Collects a single GenericRow into a buffer.
default void	collect(GenericRow[] rowBatch)	Collects a batch of GenericRows into a buffer.
URI	flush()	Creates one Pinot segment using the GenericRows collected in the buffer, at the outputDirUri as specified in the tableConfig->batchConfigs.
void	init(TableConfig tableConfig, Schema schema)
void	init(TableConfig tableConfig, Schema schema, Map<String,String> batchConfigOverride)	Initializes the SegmentWriter with provided tableConfig and Pinot schema.

Methods inherited from interface java.io.Closeable

close

Method Detail

init

void init(TableConfig tableConfig,
          Schema schema)
   throws Exception

Throws:

Exception

See Also:

init(TableConfig, Schema, Map)

init

void init(TableConfig tableConfig,
          Schema schema,
          Map<String,String> batchConfigOverride)
   throws Exception

Initializes the SegmentWriter with provided tableConfig and Pinot schema.

Parameters:

tableConfig - The table config for the segment

schema - The Pinot schema for the table

batchConfigOverride - The config override on top of tableConfig

Throws:

Exception

collect

void collect(GenericRow row)
      throws Exception

Collects a single GenericRow into a buffer. This row is not available in the segment until a flush() is invoked.

Throws:

Exception

collect

default void collect(GenericRow[] rowBatch)
              throws Exception

Collects a batch of GenericRows into a buffer. These rows are not available in the segment until a flush() is invoked.

Throws:

Exception

flush

URI flush()
   throws Exception

Creates one Pinot segment using the GenericRows collected in the buffer, at the outputDirUri as specified in the tableConfig->batchConfigs. Successful invocation of this method means that the GenericRows collected so far, are now available in the Pinot segment and not available in the buffer anymore.

Returns:

URI of the generated segment

Throws:

Exception