org.apache.pulsar.broker.service

Class AbstractTopic

java.lang.Object

org.apache.pulsar.broker.service.AbstractTopic

All Implemented Interfaces:

Topic, TopicPolicyListener<org.apache.pulsar.common.policies.data.TopicPolicies>

Direct Known Subclasses:

NonPersistentTopic, PersistentTopic

public abstract class AbstractTopic
extends Object
implements Topic, TopicPolicyListener<org.apache.pulsar.common.policies.data.TopicPolicies>

Nested Class Summary

Nested classes/interfaces inherited from interface org.apache.pulsar.broker.service.Topic

Topic.PublishContext

Field Summary

Fields

Modifier and Type	Field and Description
protected StatsBuckets	addEntryLatencyStatsUsec
protected BrokerService	brokerService
protected boolean	hasBatchMessagePublished
protected Boolean	isAllowAutoUpdateSchema
protected boolean	isEncryptionRequired
protected boolean	isFenced
protected long	lastActive
protected ReentrantReadWriteLock	lock
protected static long	POLICY_UPDATE_FAILURE_RETRY_TIME_SECONDS
protected boolean	preciseTopicPublishRateLimitingEnable
protected ConcurrentHashMap<String,Producer>	producers
protected long	publishRateLimitedTimes
protected String	replicatorPrefix
protected ResourceGroupPublishLimiter	resourceGroupPublishLimiter
protected boolean	resourceGroupRateLimitingEnabled
protected boolean	schemaValidationEnforced
protected String	topic
protected Optional<Long>	topicEpoch
protected org.apache.pulsar.common.policies.data.HierarchyTopicPolicies	topicPolicies
protected PublishRateLimiter	topicPublishRateLimiter

Constructor Summary

Constructors

Constructor and Description
AbstractTopic(String topic, BrokerService brokerService)

Method Summary

Modifier and Type	Method and Description
protected CompletableFuture<Void>	addConsumerToSubscription(Subscription subscription, Consumer consumer)
void	addFilteredEntriesCount(int filtered)
CompletableFuture<Optional<Long>>	addProducer(Producer producer, CompletableFuture<Void> producerQueuedFuture) Tries to add a producer to the topic.
CompletableFuture<org.apache.pulsar.common.protocol.schema.SchemaVersion>	addSchema(org.apache.pulsar.common.protocol.schema.SchemaData schema) Add a schema to the topic.
CompletableFuture<Void>	checkSchemaCompatibleForConsumer(org.apache.pulsar.common.protocol.schema.SchemaData schema) Check if schema is compatible with current topic schema.
protected void	checkTopicFenced()
void	checkTopicPublishThrottlingRate()
long	currentUsageCount()
void	decrementUsageCount()
boolean	deletePartitionedTopicMetadataWhileInactive()
CompletableFuture<org.apache.pulsar.common.protocol.schema.SchemaVersion>	deleteSchema() Delete the schema if this topic has a schema defined for it.
CompletableFuture<Void>	deleteTopicPolicies()
void	disableCnxAutoRead()
protected void	disableProducerRead()
void	enableCnxAutoRead()
protected void	enableProducerReadForPublishBufferLimiting()
protected void	enableProducerReadForPublishRateLimiting() it sets cnx auto-readable if producer's cnx is disabled due to publish-throttling.
protected Consumer	getActiveConsumer(Subscription subscription)
PublishRateLimiter	getBrokerPublishRateLimiter()
BrokerService	getBrokerService() Get BrokerService.
long	getBytesInCounter()
long	getBytesOutCounter()
org.apache.pulsar.common.policies.data.impl.DispatchRateImpl	getDispatchRate()
long	getFilteredEntriesCount()
org.apache.pulsar.common.policies.data.HierarchyTopicPolicies	getHierarchyTopicPolicies() Get HierarchyTopicPolicies.
org.apache.pulsar.common.policies.data.InactiveTopicPolicies	getInactiveTopicPolicies()
long	getMsgInCounter()
long	getMsgOutCounter()
String	getName()
abstract int	getNumberOfConsumers()
abstract int	getNumberOfSameAddressConsumers(String clientAddress)
protected int	getNumberOfSameAddressConsumers(String clientAddress, List<? extends Subscription> subscriptions)
int	getNumberOfSameAddressProducers(String clientAddress)
Map<String,Producer>	getProducers()
org.apache.pulsar.common.policies.data.impl.DispatchRateImpl	getReplicatorDispatchRate()
String	getReplicatorPrefix()
org.apache.pulsar.common.policies.data.SchemaCompatibilityStrategy	getSchemaCompatibilityStrategy()
boolean	getSchemaValidationEnforced()
org.apache.pulsar.common.policies.data.SubscribeRate	getSubscribeRate()
org.apache.pulsar.common.policies.data.impl.DispatchRateImpl	getSubscriptionDispatchRate()
Optional<org.apache.pulsar.common.policies.data.TopicPolicies>	getTopicPolicies() Get TopicPolicies for this topic.
PublishRateLimiter	getTopicPublishRateLimiter()
protected int	getWaitingProducersCount()
void	handleConsumerAdded(String subscriptionName, String consumerName)
protected void	handleProducerRemoved(Producer producer)
protected boolean	hasLocalProducers()
CompletableFuture<Boolean>	hasSchema() Whether a topic has had a schema defined for it.
long	increasePublishLimitedTimes() increase the publishing limited times.
void	incrementPublishCount(int numOfMessages, long msgSizeInBytes)
protected abstract CompletableFuture<Long>	incrementTopicEpoch(Optional<Long> currentEpoch)
protected CompletableFuture<Optional<Long>>	incrementTopicEpochIfNeeded(Producer producer, CompletableFuture<Void> producerQueuedFuture)
protected void	internalAddProducer(Producer producer)
boolean	isBrokerPublishRateExceeded()
protected boolean	isConsumersExceededOnTopic()
boolean	isDeleteWhileInactive()
boolean	isEncryptionRequired()
protected boolean	isExceedMaximumMessageSize(int size, Topic.PublishContext publishContext)
protected boolean	isProducersExceeded()
boolean	isPublishRateExceeded()
boolean	isResourceGroupPublishRateExceeded(int numMessages, int bytes)
boolean	isResourceGroupRateLimitingEnabled()
protected boolean	isSameAddressConsumersExceededOnTopic(Consumer consumer)
protected boolean	isSameAddressProducersExceeded(Producer producer)
protected abstract boolean	isTerminated()
boolean	isTopicPublishRateExceeded(int numberMessages, int bytes)
void	markBatchMessagePublished()
void	recordAddLatency(long latency, TimeUnit unit) record add-latency.
protected void	registerTopicPolicyListener()
void	removeProducer(Producer producer)
void	resetBrokerPublishCountAndEnableReadIfRequired(boolean doneBrokerReset)
void	resetTopicPublishCountAndEnableReadIfRequired()
protected abstract CompletableFuture<Long>	setTopicEpoch(long newEpoch)
String	toString()
protected void	unregisterTopicPolicyListener()
void	updateBrokerDispatchRate()
void	updateBrokerPublishRate()
void	updateBrokerReplicatorDispatchRate()
void	updateBrokerSubscribeRate()
void	updateBrokerSubscriptionDispatchRate()
void	updateBrokerSubscriptionTypesEnabled()
void	updatePublishDispatcher() update topic publish dispatcher for this topic.
void	updateResourceGroupLimiter(Optional<org.apache.pulsar.common.policies.data.Policies> optPolicies)
protected void	updateTopicPolicy(org.apache.pulsar.common.policies.data.TopicPolicies data)
protected void	updateTopicPolicyByNamespacePolicy(org.apache.pulsar.common.policies.data.Policies namespacePolicies)

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Methods inherited from interface org.apache.pulsar.broker.service.Topic

addSchemaIfIdleOrCheckCompatible, asyncGetStats, checkBackloggedCursors, checkBacklogQuotaExceeded, checkDeduplicationSnapshot, checkGC, checkIfTransactionBufferRecoverCompletely, checkInactiveSubscriptions, checkMessageDeduplicationInfo, checkMessageExpiry, checkReplication, close, createSubscription, delete, deleteForcefully, endTxn, getBacklogQuota, getBrokerDispatchRateLimiter, getDispatchRateLimiter, getInternalStats, getLastMessageId, getLastPosition, getReplicators, getStats, getSubscribeRateLimiter, getSubscription, getSubscriptions, initialize, isPersistent, isReplicated, isSystemTopic, onPoliciesUpdate, publishMessage, publishTxnMessage, subscribe, subscribe, truncate, unsubscribe, updateRates

Methods inherited from interface org.apache.pulsar.broker.service.TopicPolicyListener

onUpdate

Field Detail

POLICY_UPDATE_FAILURE_RETRY_TIME_SECONDS

protected static final long POLICY_UPDATE_FAILURE_RETRY_TIME_SECONDS

See Also:

Constant Field Values

topic

protected final String topic

producers

protected final ConcurrentHashMap<String,Producer> producers

brokerService

protected final BrokerService brokerService

replicatorPrefix

protected final String replicatorPrefix

lock

protected final ReentrantReadWriteLock lock

isFenced

protected volatile boolean isFenced

topicPolicies

protected final org.apache.pulsar.common.policies.data.HierarchyTopicPolicies topicPolicies

lastActive

protected volatile long lastActive

hasBatchMessagePublished

protected volatile boolean hasBatchMessagePublished

addEntryLatencyStatsUsec

protected StatsBuckets addEntryLatencyStatsUsec

isEncryptionRequired

protected volatile boolean isEncryptionRequired

isAllowAutoUpdateSchema

protected volatile Boolean isAllowAutoUpdateSchema

schemaValidationEnforced

protected volatile boolean schemaValidationEnforced

topicPublishRateLimiter

protected volatile PublishRateLimiter topicPublishRateLimiter

resourceGroupPublishLimiter

protected volatile ResourceGroupPublishLimiter resourceGroupPublishLimiter

preciseTopicPublishRateLimitingEnable

protected boolean preciseTopicPublishRateLimitingEnable

resourceGroupRateLimitingEnabled

protected boolean resourceGroupRateLimitingEnabled

publishRateLimitedTimes

protected volatile long publishRateLimitedTimes

topicEpoch

protected volatile Optional<Long> topicEpoch

Constructor Detail

AbstractTopic

public AbstractTopic(String topic,
                     BrokerService brokerService)

Method Detail

getSubscribeRate

public org.apache.pulsar.common.policies.data.SubscribeRate getSubscribeRate()

getSubscriptionDispatchRate

public org.apache.pulsar.common.policies.data.impl.DispatchRateImpl getSubscriptionDispatchRate()

getSchemaCompatibilityStrategy

public org.apache.pulsar.common.policies.data.SchemaCompatibilityStrategy getSchemaCompatibilityStrategy()

getDispatchRate

public org.apache.pulsar.common.policies.data.impl.DispatchRateImpl getDispatchRate()

getReplicatorDispatchRate

public org.apache.pulsar.common.policies.data.impl.DispatchRateImpl getReplicatorDispatchRate()

updateTopicPolicy

protected void updateTopicPolicy(org.apache.pulsar.common.policies.data.TopicPolicies data)

updateTopicPolicyByNamespacePolicy

protected void updateTopicPolicyByNamespacePolicy(org.apache.pulsar.common.policies.data.Policies namespacePolicies)

isProducersExceeded

protected boolean isProducersExceeded()

registerTopicPolicyListener

protected void registerTopicPolicyListener()

unregisterTopicPolicyListener

protected void unregisterTopicPolicyListener()

isSameAddressProducersExceeded

protected boolean isSameAddressProducersExceeded(Producer producer)

getNumberOfSameAddressProducers

public int getNumberOfSameAddressProducers(String clientAddress)

isConsumersExceededOnTopic

protected boolean isConsumersExceededOnTopic()

isSameAddressConsumersExceededOnTopic

protected boolean isSameAddressConsumersExceededOnTopic(Consumer consumer)

getNumberOfConsumers

public abstract int getNumberOfConsumers()

getNumberOfSameAddressConsumers

public abstract int getNumberOfSameAddressConsumers(String clientAddress)

getNumberOfSameAddressConsumers

protected int getNumberOfSameAddressConsumers(String clientAddress,
                                              List<? extends Subscription> subscriptions)

addConsumerToSubscription

protected CompletableFuture<Void> addConsumerToSubscription(Subscription subscription,
                                                            Consumer consumer)

getActiveConsumer

protected Consumer getActiveConsumer(Subscription subscription)

disableCnxAutoRead

public void disableCnxAutoRead()

Specified by:

disableCnxAutoRead in interface Topic

enableCnxAutoRead

public void enableCnxAutoRead()

Specified by:

enableCnxAutoRead in interface Topic

hasLocalProducers

protected boolean hasLocalProducers()

toString

public String toString()

Overrides:

toString in class Object

getProducers

public Map<String,Producer> getProducers()

Specified by:

getProducers in interface Topic

getBrokerService

public BrokerService getBrokerService()

Description copied from interface: Topic

Get BrokerService.

Specified by:

getBrokerService in interface Topic

Returns:

getName

public String getName()

Specified by:

getName in interface Topic

isEncryptionRequired

public boolean isEncryptionRequired()

Specified by:

isEncryptionRequired in interface Topic

getSchemaValidationEnforced

public boolean getSchemaValidationEnforced()

Specified by:

getSchemaValidationEnforced in interface Topic

markBatchMessagePublished

public void markBatchMessagePublished()

getReplicatorPrefix

public String getReplicatorPrefix()

hasSchema

public CompletableFuture<Boolean> hasSchema()

Description copied from interface: Topic

Whether a topic has had a schema defined for it.

Specified by:

hasSchema in interface Topic

addSchema

public CompletableFuture<org.apache.pulsar.common.protocol.schema.SchemaVersion> addSchema(org.apache.pulsar.common.protocol.schema.SchemaData schema)

Description copied from interface: Topic

Add a schema to the topic. This will fail if the new schema is incompatible with the current schema.

Specified by:

addSchema in interface Topic

deleteSchema

public CompletableFuture<org.apache.pulsar.common.protocol.schema.SchemaVersion> deleteSchema()

Description copied from interface: Topic

Delete the schema if this topic has a schema defined for it.

Specified by:

deleteSchema in interface Topic

checkSchemaCompatibleForConsumer

public CompletableFuture<Void> checkSchemaCompatibleForConsumer(org.apache.pulsar.common.protocol.schema.SchemaData schema)

Description copied from interface: Topic

Check if schema is compatible with current topic schema.

Specified by:

checkSchemaCompatibleForConsumer in interface Topic

addProducer

public CompletableFuture<Optional<Long>> addProducer(Producer producer,
                                                     CompletableFuture<Void> producerQueuedFuture)

Description copied from interface: Topic

Tries to add a producer to the topic. Several validations will be performed.

Specified by:

addProducer in interface Topic

producerQueuedFuture - a future that will be triggered if the producer is being queued up prior of getting established

Returns:

the "topic epoch" if there is one or empty

incrementTopicEpochIfNeeded

protected CompletableFuture<Optional<Long>> incrementTopicEpochIfNeeded(Producer producer,
                                                                        CompletableFuture<Void> producerQueuedFuture)

setTopicEpoch

protected abstract CompletableFuture<Long> setTopicEpoch(long newEpoch)

incrementTopicEpoch

protected abstract CompletableFuture<Long> incrementTopicEpoch(Optional<Long> currentEpoch)

recordAddLatency

public void recordAddLatency(long latency,
                             TimeUnit unit)

Description copied from interface: Topic

record add-latency.

Specified by:

recordAddLatency in interface Topic

increasePublishLimitedTimes

public long increasePublishLimitedTimes()

Description copied from interface: Topic

increase the publishing limited times.

Specified by:

increasePublishLimitedTimes in interface Topic

checkTopicPublishThrottlingRate

public void checkTopicPublishThrottlingRate()

Specified by:

checkTopicPublishThrottlingRate in interface Topic

incrementPublishCount

public void incrementPublishCount(int numOfMessages,
                                  long msgSizeInBytes)

Specified by:

incrementPublishCount in interface Topic

resetTopicPublishCountAndEnableReadIfRequired

public void resetTopicPublishCountAndEnableReadIfRequired()

Specified by:

resetTopicPublishCountAndEnableReadIfRequired in interface Topic

resetBrokerPublishCountAndEnableReadIfRequired

public void resetBrokerPublishCountAndEnableReadIfRequired(boolean doneBrokerReset)

Specified by:

resetBrokerPublishCountAndEnableReadIfRequired in interface Topic

enableProducerReadForPublishRateLimiting

protected void enableProducerReadForPublishRateLimiting()

it sets cnx auto-readable if producer's cnx is disabled due to publish-throttling.

enableProducerReadForPublishBufferLimiting

protected void enableProducerReadForPublishBufferLimiting()

disableProducerRead

protected void disableProducerRead()

checkTopicFenced

protected void checkTopicFenced()
                         throws BrokerServiceException

Throws:

BrokerServiceException

internalAddProducer

protected void internalAddProducer(Producer producer)
                            throws BrokerServiceException

Throws:

BrokerServiceException

removeProducer

public void removeProducer(Producer producer)

Specified by:

removeProducer in interface Topic

handleProducerRemoved

protected void handleProducerRemoved(Producer producer)

handleConsumerAdded

public void handleConsumerAdded(String subscriptionName,
                                String consumerName)

decrementUsageCount

public void decrementUsageCount()

currentUsageCount

public long currentUsageCount()

isPublishRateExceeded

public boolean isPublishRateExceeded()

Specified by:

isPublishRateExceeded in interface Topic

isResourceGroupPublishRateExceeded

public boolean isResourceGroupPublishRateExceeded(int numMessages,
                                                  int bytes)

Specified by:

isResourceGroupPublishRateExceeded in interface Topic

isResourceGroupRateLimitingEnabled

public boolean isResourceGroupRateLimitingEnabled()

Specified by:

isResourceGroupRateLimitingEnabled in interface Topic

isTopicPublishRateExceeded

public boolean isTopicPublishRateExceeded(int numberMessages,
                                          int bytes)

Specified by:

isTopicPublishRateExceeded in interface Topic

isBrokerPublishRateExceeded

public boolean isBrokerPublishRateExceeded()

Specified by:

isBrokerPublishRateExceeded in interface Topic

getTopicPublishRateLimiter

public PublishRateLimiter getTopicPublishRateLimiter()

getBrokerPublishRateLimiter

public PublishRateLimiter getBrokerPublishRateLimiter()

updateResourceGroupLimiter

public void updateResourceGroupLimiter(Optional<org.apache.pulsar.common.policies.data.Policies> optPolicies)

getMsgInCounter

public long getMsgInCounter()

getBytesInCounter

public long getBytesInCounter()

getMsgOutCounter

public long getMsgOutCounter()

getBytesOutCounter

public long getBytesOutCounter()

isDeleteWhileInactive

public boolean isDeleteWhileInactive()

deletePartitionedTopicMetadataWhileInactive

public boolean deletePartitionedTopicMetadataWhileInactive()

isTerminated

protected abstract boolean isTerminated()

getInactiveTopicPolicies

public org.apache.pulsar.common.policies.data.InactiveTopicPolicies getInactiveTopicPolicies()

getTopicPolicies

public Optional<org.apache.pulsar.common.policies.data.TopicPolicies> getTopicPolicies()

Get TopicPolicies for this topic.

Returns:

TopicPolicies, if they exist. Otherwise, the value will not be present.

deleteTopicPolicies

public CompletableFuture<Void> deleteTopicPolicies()

getWaitingProducersCount

protected int getWaitingProducersCount()

isExceedMaximumMessageSize

protected boolean isExceedMaximumMessageSize(int size,
                                             Topic.PublishContext publishContext)

updatePublishDispatcher

public void updatePublishDispatcher()

update topic publish dispatcher for this topic.

updateBrokerSubscriptionTypesEnabled

public void updateBrokerSubscriptionTypesEnabled()

getHierarchyTopicPolicies

public org.apache.pulsar.common.policies.data.HierarchyTopicPolicies getHierarchyTopicPolicies()

Description copied from interface: Topic

Get HierarchyTopicPolicies.

Specified by:

getHierarchyTopicPolicies in interface Topic

Returns:

updateBrokerSubscriptionDispatchRate

public void updateBrokerSubscriptionDispatchRate()

updateBrokerReplicatorDispatchRate

public void updateBrokerReplicatorDispatchRate()

updateBrokerDispatchRate

public void updateBrokerDispatchRate()

addFilteredEntriesCount

public void addFilteredEntriesCount(int filtered)

getFilteredEntriesCount

public long getFilteredEntriesCount()

updateBrokerPublishRate

public void updateBrokerPublishRate()

updateBrokerSubscribeRate

public void updateBrokerSubscribeRate()