object KCLConsumer
- Source
- KCLConsumer.scala
- Alphabetic
- By Inheritance
- KCLConsumer
- AnyRef
- Any
- Hide All
- Show All
- Public
- All
Type Members
-
final
case class
Config[F[_]] extends Product with Serializable
Config class for the KCLConsumer
-
final
case class
ProcessConfig(raiseOnError: Boolean, recordProcessorConfig: RecordProcessor.Config, callProcessRecordsEvenForEmptyRecordList: Option[Boolean]) extends Product with Serializable
Helper class to hold configuration for the ProcessorConfig construction
Helper class to hold configuration for the ProcessorConfig construction
- raiseOnError
Whether the RecordProcessor should raise exceptions or simply log them. It is recommended to set this to true. See this issue for more information.
- recordProcessorConfig
- callProcessRecordsEvenForEmptyRecordList
Determines if processRecords() should run on the record processor for empty record lists. Default None.
Value Members
-
final
def
!=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
##(): Int
- Definition Classes
- AnyRef → Any
-
final
def
==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
def
apply[F[_]](checkpointConfig: CheckpointConfig, coordinatorConfig: CoordinatorConfig, leaseManagementConfig: LeaseManagementConfig, lifecycleConfig: LifecycleConfig, metricsConfig: MetricsConfig, retrievalConfig: RetrievalConfig, processConfig: ProcessConfig = ProcessConfig.default)(cb: (List[CommittableRecord[F]]) ⇒ F[Unit])(implicit F: Async[F], encoders: LogEncoders): Resource[F, KCLConsumer[F]]
Low-level constructor for the KCLConsumer.
Low-level constructor for the KCLConsumer.
- checkpointConfig
- coordinatorConfig
- leaseManagementConfig
- lifecycleConfig
- metricsConfig
- retrievalConfig
- processConfig
- cb
Function to process CommittableRecords received from Kinesis
- F
Async instance
- encoders
RecordProcessor.LogEncoders for encoding structured logs
- returns
Resource containing the KCLConsumer
-
final
def
asInstanceOf[T0]: T0
- Definition Classes
- Any
-
def
clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native() @IntrinsicCandidate()
-
def
configsBuilder[F[_]](kinesisClient: KinesisAsyncClient, dynamoClient: DynamoDbAsyncClient, cloudWatchClient: CloudWatchAsyncClient, streamName: String, appName: String, workerId: String = Utils.randomUUIDString, processConfig: ProcessConfig = ProcessConfig.default)(cb: (List[CommittableRecord[F]]) ⇒ F[Unit])(tfn: (Config[F]) ⇒ Config[F] = (x: Config[F]) => x)(implicit F: Async[F], encoders: LogEncoders): Resource[F, KCLConsumer[F]]
Constructor for the KCLConsumer that leverages the ConfigsBuilder from the KCL.
Constructor for the KCLConsumer that leverages the ConfigsBuilder from the KCL. This is a simpler entry-point for creating the configuration, and provides a transform function to add any custom configuration that was not covered by the default
- kinesisClient
- dynamoClient
- cloudWatchClient
- streamName
Name of the Kinesis stream to consume from
- appName
Name of the application. Usually also the dynamo table name for checkpoints
- workerId
Unique identifier for a single instance of this consumer. Default is a random UUID.
- processConfig
- cb
Function to process CommittableRecords received from Kinesis
- tfn
Function to update the KCLConsumer.Config. Useful for overriding defaults.
- F
Async instance
- encoders
RecordProcessor.LogEncoders for encoding structured logs
-
def
configsBuilderMultiStream[F[_]](kinesisClient: KinesisAsyncClient, dynamoClient: DynamoDbAsyncClient, cloudWatchClient: CloudWatchAsyncClient, tracker: MultiStreamTracker, appName: String, workerId: String = Utils.randomUUIDString, processConfig: ProcessConfig = ProcessConfig.default)(cb: (List[CommittableRecord[F]]) ⇒ F[Unit])(tfn: (Config[F]) ⇒ Config[F] = (x: Config[F]) => x)(implicit F: Async[F], encoders: LogEncoders): Resource[F, KCLConsumer[F]]
Constructor for the KCLConsumer that leverages the ConfigsBuilder from the KCL.
Constructor for the KCLConsumer that leverages the ConfigsBuilder from the KCL. This is a simpler entry-point for creating the configuration, and provides a transform function to add any custom configuration that was not covered by the default. This constructor specifically leverages the MultiStreamTracker to allow for consumption from multiple streams.
- kinesisClient
- dynamoClient
- cloudWatchClient
- tracker
- appName
Name of the application. Usually also the dynamo table name for checkpoints
- workerId
Unique identifier for a single instance of this consumer. Default is a random UUID.
- processConfig
- cb
Function to process CommittableRecords received from Kinesis
- tfn
Function to update the KCLConsumer.Config. Useful for overriding defaults.
- F
Async instance
- encoders
RecordProcessor.LogEncoders for encoding structured logs
-
final
def
eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
def
equals(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
-
final
def
getClass(): Class[_]
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @IntrinsicCandidate()
-
def
hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @IntrinsicCandidate()
-
final
def
isInstanceOf[T0]: Boolean
- Definition Classes
- Any
-
final
def
ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
-
final
def
notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @IntrinsicCandidate()
-
final
def
notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @IntrinsicCandidate()
-
final
def
synchronized[T0](arg0: ⇒ T0): T0
- Definition Classes
- AnyRef
-
def
toString(): String
- Definition Classes
- AnyRef → Any
-
final
def
wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
-
final
def
wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... ) @native()
-
final
def
wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws( ... )
- object Config extends Serializable
- object ProcessConfig extends Serializable