trait KafkaConsume[F[_], K, V] extends AnyRef
- Alphabetic
- By Inheritance
- KafkaConsume
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Abstract Value Members
- abstract def partitionedStream: Stream[F, Stream[F, CommittableConsumerRecord[F, K, V]]]
Streamwhere the elements themselves areStreams which continually request records for a single partition.Streamwhere the elements themselves areStreams which continually request records for a single partition. TheseStreams will have to be processed in parallel, usingparJoinorparJoinUnbounded. Note that when usingparJoin(n)andnis smaller than the number of currently assigned partitions, then there will be assigned partitions which won't be processed. For that reason, preferparJoinUnboundedand the actual limit will be the number of assigned partitions.
If you do not want to process all partitions in parallel, then you can use stream instead, where records for all partitions are in a singleStream.- Note
you have to first use
subscribeto subscribe the consumer before using thisStream. If you forgot to subscribe, there will be a NotSubscribedException raised in theStream.
- abstract def partitionsMapStream: Stream[F, Map[TopicPartition, Stream[F, CommittableConsumerRecord[F, K, V]]]]
Streamwhere each element contains a current assignment.Streamwhere each element contains a current assignment. The current assignment is theMap, where keys is aTopicPartition, and values are streams with records for a particularTopicPartition.
New assignments will be received on each rebalance. On rebalance, Kafka revoke all previously assigned partitions, and after that assigned new partitions all at once.partitionsMapStreamreflects this process in a streaming manner.
Note, that partition streams for revoked partitions will be closed after the new assignment comes.
This is the most genericStreammethod. If you don't need such control, consider usingpartitionedStreamorstreammethods. They are both based on apartitionsMapStream.- Note
you have to first use
subscribeto subscribe the consumer before using thisStream. If you forgot to subscribe, there will be a NotSubscribedException raised in theStream.- See also
- abstract def stopConsuming: F[Unit]
Stops consuming new messages from Kafka.
Stops consuming new messages from Kafka. This method could be used to implement a graceful shutdown.
This method has a few effects: 1. After this call no more data will be fetched from Kafka through thepollmethod. 2. All currently running streams will continue to run until all in-flight messages will be processed. It means that streams will be completed when all fetched messages will be processed.
If some of the stream methods will be called after stopConsuming call, these methods will return empty streams.
More than one call of stopConsuming will have no effect. - abstract def stream: Stream[F, CommittableConsumerRecord[F, K, V]]
Alias for
partitionedStream.parJoinUnbounded.Alias for
partitionedStream.parJoinUnbounded. See partitionedStream for more information.- Note
you have to first use
subscribeto subscribe the consumer before using thisStream. If you forgot to subscribe, there will be a NotSubscribedException raised in theStream.
Concrete Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable])
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native()
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native()
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()