Package io.trino.plugin.kinesis

Class KinesisSplitManager

java.lang.Object

io.trino.plugin.kinesis.KinesisSplitManager

All Implemented Interfaces:

ConnectorSplitManager

public class KinesisSplitManager
extends Object
implements ConnectorSplitManager

Split data chunk from kinesis Stream to multiple small chunks for parallelization and distribution to multiple Trino workers. By default, each shard of Kinesis Stream forms one Kinesis Split

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	KinesisSplitManager.InternalStreamDescription	Cache the result of a Kinesis describe stream call so we don't need to retrieve the shards for every single query.

Field Summary

Fields

Modifier and Type	Field	Description
static final long	MAX_CACHE_AGE_MILLIS

Constructor Summary

Constructors

Constructor	Description
KinesisSplitManager(KinesisClientProvider clientManager)

Method Summary

Modifier and Type	Method	Description
ConnectorSplitSource	getSplits(ConnectorTransactionHandle transactionHandle, ConnectorSession session, ConnectorTableHandle table, DynamicFilter dynamicFilter, Constraint constraint)
protected KinesisSplitManager.InternalStreamDescription	getStreamDescription(String streamName)	Internal method to retrieve the stream description and get the shards from AWS.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Methods inherited from interface io.trino.spi.connector.ConnectorSplitManager

getSplits

Field Details

MAX_CACHE_AGE_MILLIS

public static final long MAX_CACHE_AGE_MILLIS

Constructor Details

KinesisSplitManager

@Inject
public KinesisSplitManager(KinesisClientProvider clientManager)

Method Details

getSplits

public ConnectorSplitSource getSplits(ConnectorTransactionHandle transactionHandle,
 ConnectorSession session,
 ConnectorTableHandle table,
 DynamicFilter dynamicFilter,
 Constraint constraint)

Specified by:

getSplits in interface ConnectorSplitManager

getStreamDescription

protected KinesisSplitManager.InternalStreamDescription getStreamDescription(String streamName)

Internal method to retrieve the stream description and get the shards from AWS.

Gets from the internal cache unless not yet created or too old.