Package org.apache.pulsar.broker.service.persistent

Class MessageDeduplication

java.lang.Object

org.apache.pulsar.broker.service.persistent.MessageDeduplication

public class MessageDeduplication
extends Object

Class that contains all the logic to control and perform the deduplication on the broker side.

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static enum	MessageDeduplication.MessageDupStatus
static class	MessageDeduplication.MessageDupUnknownException

Constructor Summary

Constructors

Constructor	Description
MessageDeduplication(PulsarService pulsar, PersistentTopic topic, org.apache.bookkeeper.mledger.ManagedLedger managedLedger)

Method Summary

Modifier and Type	Method	Description
CompletableFuture<Void>	checkStatus()	Check the status of deduplication.
long	getLastPublishedSequenceId(String producerName)
org.apache.pulsar.broker.service.persistent.MessageDeduplication.Status	getStatus()
MessageDeduplication.MessageDupStatus	isDuplicate(Topic.PublishContext publishContext, io.netty.buffer.ByteBuf headersAndPayload)	Assess whether the message was already stored in the topic.
boolean	isEnabled()
void	producerAdded(String producerName)	Topic will call this method whenever a producer connects.
void	producerRemoved(String producerName)	Topic will call this method whenever a producer disconnects.
void	purgeInactiveProducers()	Remove from hash maps all the producers that were inactive for more than the configured amount of time.
void	recordMessagePersisted(Topic.PublishContext publishContext, org.apache.bookkeeper.mledger.impl.PositionImpl position)	Call this method whenever a message is persisted to get the chance to trigger a snapshot.
void	resetHighestSequenceIdPushed()
void	takeSnapshot()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

MessageDeduplication

public MessageDeduplication(PulsarService pulsar,
 PersistentTopic topic,
 org.apache.bookkeeper.mledger.ManagedLedger managedLedger)

Method Details

getStatus

public org.apache.pulsar.broker.service.persistent.MessageDeduplication.Status getStatus()

checkStatus

public CompletableFuture<Void> checkStatus()

Check the status of deduplication. If the configuration has changed, it will enable/disable deduplication, returning a future to track the completion of the task

isEnabled

public boolean isEnabled()

isDuplicate

public MessageDeduplication.MessageDupStatus isDuplicate(Topic.PublishContext publishContext,
 io.netty.buffer.ByteBuf headersAndPayload)

Assess whether the message was already stored in the topic.

Returns:

true if the message should be published or false if it was recognized as a duplicate

recordMessagePersisted

public void recordMessagePersisted(Topic.PublishContext publishContext,
 org.apache.bookkeeper.mledger.impl.PositionImpl position)

Call this method whenever a message is persisted to get the chance to trigger a snapshot.

resetHighestSequenceIdPushed

public void resetHighestSequenceIdPushed()

producerAdded

public void producerAdded(String producerName)

Topic will call this method whenever a producer connects.

producerRemoved

public void producerRemoved(String producerName)

Topic will call this method whenever a producer disconnects.

purgeInactiveProducers

public void purgeInactiveProducers()

Remove from hash maps all the producers that were inactive for more than the configured amount of time.

getLastPublishedSequenceId

public long getLastPublishedSequenceId(String producerName)

takeSnapshot

public void takeSnapshot()