trait KafkaConsume[F[_], K, V] extends AnyRef
- Alphabetic
- By Inheritance
- KafkaConsume
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Abstract Value Members
- abstract def partitionedStream: Stream[F, Stream[F, CommittableConsumerRecord[F, K, V]]]
Streamwhere the elements themselves areStreams which continually request records for a single partition.Streamwhere the elements themselves areStreams which continually request records for a single partition. TheseStreams will have to be processed in parallel, usingparJoinorparJoinUnbounded. Note that when usingparJoin(n)andnis smaller than the number of currently assigned partitions, then there will be assigned partitions which won't be processed. For that reason, preferparJoinUnboundedand the actual limit will be the number of assigned partitions.
If you do not want to process all partitions in parallel, then you can use records instead, where records for all partitions are in a singleStream.- Note
you have to first use
subscribeto subscribe the consumer before using thisStream. If you forgot to subscribe, there will be a NotSubscribedException raised in theStream.
- abstract def partitionsMapStream: Stream[F, Map[TopicPartition, Stream[F, CommittableConsumerRecord[F, K, V]]]]
Streamwhere each element contains an assignment.Streamwhere each element contains an assignment. Each assignment assignment isMap, where keys are aTopicPartitions, and values are record streams for theTopicPartition.
With the default assignor, previous partition assignments are revoked at once, and a new set of partitions assigned to a consumer on each rebalance. In this case, each returned map contains the full partition assignment for the consumer.partitionsMapStreamreflects the assignment process in a streaming manner.
This may not be the case when a custom assignor is configured in the consumer. When using theCooperativeStickyAssignor, for instance, partition assignments may be revoked individually. In this case, each element in the stream will contain only streams for newly assigned partitions. Streams returned previously will remain active until the assignment is revoked.
Note, that partition streams for revoked partitions will be closed after the new assignment comes.
This is the most genericStreammethod. If you don't need such control, consider usingpartitionedStreamorstreammethods. They are both based on apartitionsMapStream.- Note
you have to first use
subscribeto subscribe the consumer before using thisStream. If you forgot to subscribe, there will be a NotSubscribedException raised in theStream.- See also
- abstract def stopConsuming: F[Unit]
Stops consuming new messages from Kafka.
Stops consuming new messages from Kafka. This method could be used to implement a graceful shutdown.
This method has a few effects: 1. After this call no more data will be fetched from Kafka through thepollmethod. 2. All currently running streams will continue to run until all in-flight messages will be processed. It means that streams will be completed when all fetched messages will be processed.
If some of the records methods will be called after stopConsuming call, these methods will return empty streams.
More than one call of stopConsuming will have no effect. - abstract def stream: Stream[F, CommittableConsumerRecord[F, K, V]]
Alias for
partitionedStream.parJoinUnbounded.Alias for
partitionedStream.parJoinUnbounded. See partitionedRecords for more information.- Note
you have to first use
subscribeto subscribe the consumer before using thisStream. If you forgot to subscribe, there will be a NotSubscribedException raised in theStream.
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def partitionedRecords: Stream[F, Stream[F, CommittableConsumerRecord[F, K, V]]]
Alias for partitionedStream
- final def records: Stream[F, CommittableConsumerRecord[F, K, V]]
Consume from all assigned partitions, producing a stream of CommittableConsumerRecords.
Consume from all assigned partitions, producing a stream of CommittableConsumerRecords. Alias for stream.
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()