Package org.apache.pinot.spi.stream

Interface StreamDataProducer

public interface StreamDataProducer

StreamDataProducer is the interface for stream data sources. E.g. KafkaDataProducer.

Nested Class Summary

Nested Classes

Modifier and Type	Interface	Description
static class	StreamDataProducer.RowWithKey	Helper class so the key and payload can be easily tied together instead of using a pair The class is intended for StreamDataProducer only

Method Summary

Modifier and Type	Method	Description
void	close()
void	init(Properties props)
void	produce(String topic, byte[] payload)
void	produce(String topic, byte[] key, byte[] payload)
default void	produce(String topic, byte[] key, byte[] payload, GenericRow headers)
default void	produceBatch(String topic, List<byte[]> rows)	Allows the producer to optimize for a batched write.
default void	produceKeyedBatch(String topic, List<StreamDataProducer.RowWithKey> payloadWithKey)
default void	produceKeyedBatch(String topic, List<StreamDataProducer.RowWithKey> payloadWithKey, boolean includeHeaders)	Allows the producer to optimize for a batched write.

Method Detail

init

void init(Properties props)

produce

void produce(String topic,
             byte[] payload)

produce

void produce(String topic,
             byte[] key,
             byte[] payload)

produce

default void produce(String topic,
                     byte[] key,
                     byte[] payload,
                     GenericRow headers)

close

void close()

produceBatch

default void produceBatch(String topic,
                          List<byte[]> rows)

Allows the producer to optimize for a batched write. This will help increase throughput in some cases

Parameters:

topic - the topic of the output

rows - the rows

produceKeyedBatch

default void produceKeyedBatch(String topic,
                               List<StreamDataProducer.RowWithKey> payloadWithKey,
                               boolean includeHeaders)

Allows the producer to optimize for a batched write. This will help increase throughput in some cases

Parameters:

topic - the topic of the output

payloadWithKey - the payload rows with key

produceKeyedBatch

default void produceKeyedBatch(String topic,
                               List<StreamDataProducer.RowWithKey> payloadWithKey)