trait Consumer extends AnyRef
- Alphabetic
- By Inheritance
- Consumer
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Abstract Value Members
- abstract def assignment: Task[Set[TopicPartition]]
Returns the topic-partitions that this consumer is currently assigned.
Returns the topic-partitions that this consumer is currently assigned.
This is subject to consumer rebalancing, unless using a manual subscription.
- abstract def beginningOffsets(partitions: Set[TopicPartition], timeout: zio.Duration = Duration.Infinity): Task[Map[TopicPartition, Long]]
- abstract def committed(partitions: Set[TopicPartition], timeout: zio.Duration = Duration.Infinity): Task[Map[TopicPartition, Option[OffsetAndMetadata]]]
Retrieve the last committed offset for the given topic-partitions
- abstract def consumeWith[R, R1, K, V](subscription: Subscription, keyDeserializer: Deserializer[R, K], valueDeserializer: Deserializer[R, V], commitRetryPolicy: Schedule[Any, Any, Any] = Schedule.exponential(1.second) && Schedule.recurs(3))(f: (ConsumerRecord[K, V]) => URIO[R1, Unit])(implicit arg0: zio.EnvironmentTag[R], arg1: zio.EnvironmentTag[R1]): ZIO[zio.&[R, R1], Throwable, Unit]
See Consumer.consumeWith.
- abstract val consumerSettings: ConsumerSettings
The settings used to create this consumer.
- abstract def endOffsets(partitions: Set[TopicPartition], timeout: zio.Duration = Duration.Infinity): Task[Map[TopicPartition, Long]]
- abstract def listTopics(timeout: zio.Duration = Duration.Infinity): Task[Map[String, List[PartitionInfo]]]
- abstract def metrics: Task[Map[MetricName, Metric]]
Expose internal consumer metrics
- abstract def offsetsForTimes(timestamps: Map[TopicPartition, Long], timeout: zio.Duration = Duration.Infinity): Task[Map[TopicPartition, OffsetAndTimestamp]]
Look up the offsets for the given partitions by timestamp.
Look up the offsets for the given partitions by timestamp. The returned offset for each partition is the earliest offset whose timestamp is greater than or equal to the given timestamp in the corresponding partition.
The consumer does not have to be assigned the partitions. If no messages exist yet for a partition, it will not exist in the returned map.
- abstract def partitionedAssignmentStream[R, K, V](subscription: Subscription, keyDeserializer: Deserializer[R, K], valueDeserializer: Deserializer[R, V]): Stream[Throwable, Chunk[(TopicPartition, ZStream[R, Throwable, CommittableRecord[K, V]])]]
Create a stream that emits chunks whenever new partitions are assigned to this consumer.
Create a stream that emits chunks whenever new partitions are assigned to this consumer.
The top-level stream will emit chunks whenever the consumer rebalances, unless a manual subscription was made. When rebalancing occurs, new topic-partition streams may be emitted and existing streams may be completed.
All streams can be completed by calling stopConsumption.
Multiple subscriptions on one Consumer are supported, as long as the subscriptions are of the same type (topics, patterns, manual). Each subscription will only receive messages from the topic-partitions that match the subscription. When subscriptions overlap, kafka records will be divided over the overlapping subscriptions non-deterministically.
On completion of the stream, the consumer is unsubscribed. In case of multiple subscriptions, the total consumer subscription is changed to exclude this subscription.
- abstract def partitionedAssignmentStreamWithControl[R, K, V](subscription: Subscription, keyDeserializer: Deserializer[R, K], valueDeserializer: Deserializer[R, V]): ZIO[Scope, Throwable, StreamControl[R, Throwable, Chunk[(TopicPartition, ZStream[R, Throwable, CommittableRecord[K, V]])]]]
Like partitionedAssignmentStream but returns an object to control the stream.
Like partitionedAssignmentStream but returns an object to control the stream.
The Scope provided to the returned effect controls the lifetime of the subscription. The subscription is unsubscribed when the scope is ended. Calling StreamControl.end stops fetching data for the subscription partitions but will not unsubscribe until the Scope is ended.
See the docs at https://zio.dev/zio-kafka/consuming-kafka-topics-using-zio-streams for more information.
WARNING: this is an EXPERIMENTAL API and may disappear or change in an incompatible way without notice in any zio-kafka version.
- abstract def partitionedStream[R, K, V](subscription: Subscription, keyDeserializer: Deserializer[R, K], valueDeserializer: Deserializer[R, V]): Stream[Throwable, (TopicPartition, ZStream[R, Throwable, CommittableRecord[K, V]])]
Create a stream with messages on the subscribed topic-partitions by topic-partition
Create a stream with messages on the subscribed topic-partitions by topic-partition
The top-level stream will emit new topic-partition streams for each topic-partition that is assigned to this consumer. This is subject to consumer rebalancing, unless a manual subscription was made. When rebalancing occurs, new topic-partition streams may be emitted and existing streams may be completed.
All streams can be completed by calling stopConsumption.
Multiple subscriptions on one Consumer are supported, as long as the subscriptions are of the same type (topics, patterns, manual). Each subscription will only receive messages from the topic-partitions that match the subscription. When subscriptions overlap, kafka records will be divided over the overlapping subscriptions non-deterministically.
On completion of the stream, the consumer is unsubscribed. In case of multiple subscriptions, the total consumer subscription is changed to exclude this subscription.
- abstract def partitionedStreamWithControl[R, K, V](subscription: Subscription, keyDeserializer: Deserializer[R, K], valueDeserializer: Deserializer[R, V]): ZIO[Scope with R, Throwable, StreamControl[R, Throwable, (TopicPartition, ZStream[R, Throwable, CommittableRecord[K, V]])]]
Like partitionedStream but returns an object to control the stream.
Like partitionedStream but returns an object to control the stream.
The Scope provided to the returned effect controls the lifetime of the subscription. The subscription is unsubscribed when the scope is ended. Calling StreamControl.end stops fetching data for the subscription partitions but will not unsubscribe until the Scope is ended.
See the docs at https://zio.dev/zio-kafka/consuming-kafka-topics-using-zio-streams for more information.
WARNING: this is an EXPERIMENTAL API and may disappear or change in an incompatible way without notice in any zio-kafka version.
- abstract def partitionsFor(topic: String, timeout: zio.Duration = Duration.Infinity): Task[List[PartitionInfo]]
- abstract def plainStream[R, K, V](subscription: Subscription, keyDeserializer: Deserializer[R, K], valueDeserializer: Deserializer[R, V], bufferSize: Int = 4): ZStream[R, Throwable, CommittableRecord[K, V]]
Create a stream with all messages on the subscribed topic-partitions
Create a stream with all messages on the subscribed topic-partitions
The stream will emit messages from all topic-partitions interleaved. Per-partition record order is guaranteed, but the topic-partition interleaving is non-deterministic.
Up to
bufferSizechunks may be buffered in memory by this operator.The stream can be completed by calling stopConsumption.
Multiple subscriptions on one Consumer are supported, as long as the subscriptions are of the same type (topics, patterns, manual). Each subscription will only receive messages from the topic-partitions that match the subscription. When subscriptions overlap, kafka records will be divided over the overlapping subscriptions non-deterministically.
On completion of the stream, the consumer is unsubscribed. In case of multiple subscriptions, the total consumer subscription is changed to exclude this subscription.
- abstract def plainStreamWithControl[R, K, V](subscription: Subscription, keyDeserializer: Deserializer[R, K], valueDeserializer: Deserializer[R, V], bufferSize: Int = 4): ZIO[Scope with R, Throwable, StreamControl[R, Throwable, CommittableRecord[K, V]]]
Like plainStream but returns an object to control the stream.
Like plainStream but returns an object to control the stream.
The Scope provided to the returned effect controls the lifetime of the subscription. The subscription is unsubscribed when the scope is ended. Calling StreamControl.end stops fetching data for the subscription partitions but will not unsubscribe until the Scope is ended.
See the docs at https://zio.dev/zio-kafka/consuming-kafka-topics-using-zio-streams for more information.
WARNING: this is an EXPERIMENTAL API and may disappear or change in an incompatible way without notice in any zio-kafka version.
- abstract def position(partition: TopicPartition, timeout: zio.Duration = Duration.Infinity): Task[Long]
- abstract def registerExternalCommits(offsetBatch: OffsetBatch): Task[Unit]
Register a commit that was done externally, that is, not by this consumer.
Register a commit that was done externally, that is, not by this consumer.
This method is useful when you want to use rebalance-safe-commits, but you are not committing to the Kafka brokers, but to some external system, for example a relational database.
When this consumer is used in combination with a zio.kafka.producer.TransactionalProducer, the transactional producer calls this method when the transaction is committed.
See also zio.kafka.consumer.ConsumerSettings.withRebalanceSafeCommits.
- abstract def stopConsumption: UIO[Unit]
Stops consumption of data, drains buffered records, and ends the attached streams while still serving commit requests.
- abstract def subscription: Task[Set[String]]
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @IntrinsicCandidate() @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @IntrinsicCandidate() @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @IntrinsicCandidate() @native()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])