object KCLConsumerFS2
- Source
- KCLConsumerFS2.scala
- Alphabetic
- By Inheritance
- KCLConsumerFS2
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Type Members
-
final
case class
Config[F[_]](underlying: KCLConsumer.Config[F], queue: Queue[F, CommittableRecord[F]], fs2Config: FS2Config) extends Product with Serializable
Configuration for the KCLConsumerFS2
Configuration for the KCLConsumerFS2
- underlying
- queue
-
final
case class
FS2Config(queueSize: Int, commitMaxChunk: Int, commitMaxWait: FiniteDuration, commitMaxRetries: Int, commitRetryInterval: FiniteDuration) extends Product with Serializable
Configuration for the FS2 implementation
Configuration for the FS2 implementation
- queueSize
Size of the underlying queue for the FS2 stream. If the queue fills up, backpressure on the processors will occur. Default 100
- commitMaxChunk
Max records to be received in the commitRecords Pipe before a commit is run. Default is 1000
- commitMaxWait
Max duration to wait in commitRecords Pipe before a commit is run. Default is 10 seconds
- commitMaxRetries
Max number of retries for a commit operation
- commitRetryInterval
Interval to wait between commit retries
-
final
case class
StreamAndDeferred[F[_]](stream: Stream[F, CommittableRecord[F]], deferred: Deferred[F, Unit]) extends Product with Serializable
Helper class that holds both an fs2.Stream and a Deferred
Helper class that holds both an fs2.Stream and a Deferred
- stream
fs2.Stream of kinesis4cats.kcl.CommittableRecord values to process
- deferred
Deferred which will complete when a defined WorkerState is recognized.
-
final
case class
StreamAndRef[F[_]](stream: Stream[F, CommittableRecord[F]], ref: Ref[F, WorkerState]) extends Product with Serializable
Helper class that holds both an fs2.Stream and a Ref of the WorkerState
Helper class that holds both an fs2.Stream and a Ref of the WorkerState
- stream
fs2.Stream of kinesis4cats.kcl.CommittableRecord values to process
- ref
Ref that contains the current WorkerState
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
apply[F[_]](checkpointConfig: CheckpointConfig, coordinatorConfig: CoordinatorConfig, leaseManagementConfig: LeaseManagementConfig, lifecycleConfig: LifecycleConfig, metricsConfig: MetricsConfig, retrievalConfig: RetrievalConfig, fs2Config: FS2Config = FS2Config.default, processConfig: ProcessConfig = defaultProcessConfig)(implicit F: Async[F], P: Parallel[F], encoders: LogEncoders): Resource[F, KCLConsumerFS2[F]]
Low-level constructor for KCLConsumerFS2.
Low-level constructor for KCLConsumerFS2.
- checkpointConfig
- coordinatorConfig
- leaseManagementConfig
- lifecycleConfig
- metricsConfig
- retrievalConfig
- processConfig
- F
Async instance
- encoders
RecordProcessor.LogEncoders for encoding structured logs
- returns
Resource containing the KCLConsumerFS2
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native() @IntrinsicCandidate()
-
def
configsBuilder[F[_]](kinesisClient: KinesisAsyncClient, dynamoClient: DynamoDbAsyncClient, cloudWatchClient: CloudWatchAsyncClient, streamName: String, appName: String, fs2Config: FS2Config = FS2Config.default, workerId: String = Utils.randomUUIDString, processConfig: ProcessConfig = defaultProcessConfig)(tfn: (KCLConsumer.Config[F]) ⇒ KCLConsumer.Config[F] = ...)(implicit F: Async[F], P: Parallel[F], encoders: LogEncoders): Resource[F, KCLConsumerFS2[F]]
Constructor for the KCLConsumerFS2 that leverages the ConfigsBuilder from the KCL.
Constructor for the KCLConsumerFS2 that leverages the ConfigsBuilder from the KCL. This is a simpler entry-point for creating the configuration, and provides a transform function to add any custom configuration that was not covered by the default
- kinesisClient
- dynamoClient
- cloudWatchClient
- streamName
Name of the Kinesis stream to consume from
- appName
Name of the application. Usually also the dynamo table name for checkpoints
- workerId
Unique identifier for a single instance of this consumer. Default is a random UUID.
- processConfig
- tfn
Function to update the KCLConsumer.Config. Useful for overriding defaults.
- F
Async instance
- encoders
RecordProcessor.LogEncoders for encoding structured logs
- returns
Resource containing the KCLConsumerFS2.Config
-
def
configsBuilderMultiStream[F[_]](kinesisClient: KinesisAsyncClient, dynamoClient: DynamoDbAsyncClient, cloudWatchClient: CloudWatchAsyncClient, tracker: MultiStreamTracker, appName: String, fs2Config: FS2Config = FS2Config.default, workerId: String = Utils.randomUUIDString, processConfig: ProcessConfig = defaultProcessConfig)(tfn: (KCLConsumer.Config[F]) ⇒ KCLConsumer.Config[F] = ...)(implicit F: Async[F], P: Parallel[F], encoders: LogEncoders): Resource[F, KCLConsumerFS2[F]]
Constructor for the KCLConsumerFS2 that leverages the ConfigsBuilder from the KCL.
Constructor for the KCLConsumerFS2 that leverages the ConfigsBuilder from the KCL. This is a simpler entry-point for creating the configuration, and provides a transform function to add any custom configuration that was not covered by the default. This constructor specifically leverages the MultiStreamTracker to allow for consumption from multiple streams.
- kinesisClient
- dynamoClient
- cloudWatchClient
- tracker
- appName
Name of the application. Usually also the dynamo table name for checkpoints
- workerId
Unique identifier for a single instance of this consumer. Default is a random UUID.
- processConfig
- tfn
Function to update the KCLConsumer.Config. Useful for overriding defaults.
- F
Async instance
- encoders
RecordProcessor.LogEncoders for encoding structured logs
- returns
Resource containing the KCLConsumerFS2.Config
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @IntrinsicCandidate()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @IntrinsicCandidate()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @IntrinsicCandidate()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @IntrinsicCandidate()
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
- object Config extends Serializable
- object FS2Config extends Serializable