Package org.apache.pinot.spi.stream

Interface PartitionLevelConsumer

All Superinterfaces:

AutoCloseable, Closeable, PartitionGroupConsumer

@Public
@Stable
public interface PartitionLevelConsumer
extends Closeable, PartitionGroupConsumer

Interface for a consumer which fetches messages at the partition level of a stream, for given offsets

Method Summary

Modifier and Type	Method	Description
MessageBatch	fetchMessages(long startOffset, long endOffset, int timeoutMillis)	Deprecated.
default MessageBatch	fetchMessages(StreamPartitionMsgOffset startOffset, StreamPartitionMsgOffset endOffset, int timeoutMillis)	Fetch messages and the per-partition high watermark from Kafka between the specified offsets.

Methods inherited from interface java.io.Closeable

close

Methods inherited from interface org.apache.pinot.spi.stream.PartitionGroupConsumer

checkpoint, start

Method Detail

fetchMessages

@Deprecated
MessageBatch fetchMessages(long startOffset,
                           long endOffset,
                           int timeoutMillis)
                    throws TimeoutException

Deprecated.

Is here for backward compatibility for a short time. TODO Issue 5359 remove this API once external kafka consumers implements return of StreamPartitionMsgOffset Fetch messages from the stream between the specified offsets

Parameters:

startOffset -

endOffset -

timeoutMillis -

Returns:

Throws:

TimeoutException

fetchMessages

default MessageBatch fetchMessages(StreamPartitionMsgOffset startOffset,
                                   StreamPartitionMsgOffset endOffset,
                                   int timeoutMillis)
                            throws TimeoutException

Fetch messages and the per-partition high watermark from Kafka between the specified offsets.

Specified by:

fetchMessages in interface PartitionGroupConsumer

Parameters:

startOffset - The offset of the first message desired, inclusive

endOffset - The offset of the last message desired, exclusive, or null

timeoutMillis - Timeout in milliseconds

Returns:

An iterable containing messages fetched from the stream partition and their offsets, as well as the high watermark for this partition.

Throws:

TimeoutException - If the operation could not be completed within timeoutMillis milliseconds