Package org.apache.pulsar.broker.service

Class BrokerService

java.lang.Object
org.apache.pulsar.broker.service.BrokerService
All Implemented Interfaces:
Closeable, AutoCloseable
public class BrokerService extends Object implements Closeable

  • Field Details

  • Constructor Details

  • Method Details

    • startProtocolHandlers

      public void startProtocolHandlers(Map<String,Map<InetSocketAddress,io.netty.channel.ChannelInitializer<io.netty.channel.socket.SocketChannel>>> protocolHandlers)

    • start

      public void start() throws Exception
      Throws:
      Exception

    • startStatsUpdater

      protected void startStatsUpdater(int statsUpdateInitialDelayInSecs, int statsUpdateFrequencyInSecs)

    • startDeduplicationSnapshotMonitor

      protected void startDeduplicationSnapshotMonitor()

    • startInactivityMonitor

      protected void startInactivityMonitor()

    • startMessageExpiryMonitor

      protected void startMessageExpiryMonitor()

    • startCheckReplicationPolicies

      protected void startCheckReplicationPolicies()

    • startCompactionMonitor

      protected void startCompactionMonitor()

    • startConsumedLedgersMonitor

      protected void startConsumedLedgersMonitor()

    • startBacklogQuotaChecker

      protected void startBacklogQuotaChecker()

    • setupTopicPublishRateLimiterMonitor

      public void setupTopicPublishRateLimiterMonitor()
      Schedules and monitors publish-throttling for all owned topics that has publish-throttling configured. It also disables and shutdowns publish-rate-limiter monitor task if broker disables it.

    • setupBrokerPublishRateLimiterMonitor

      public void setupBrokerPublishRateLimiterMonitor()
      Schedules and monitors publish-throttling for broker that has publish-throttling configured. It also disables and shutdowns publish-rate-limiter monitor for broker task if broker disables it.

    • close

      public void close() throws IOException
      Specified by:
      close in interface AutoCloseable
      Specified by:
      close in interface Closeable
      Throws:
      IOException

    • closeAndRemoveReplicationClient

      public CompletableFuture<Void> closeAndRemoveReplicationClient(String clusterName)

    • closeAsync

      public CompletableFuture<Void> closeAsync()

    • unloadNamespaceBundlesGracefully

      public void unloadNamespaceBundlesGracefully()
      It unloads all owned namespacebundles gracefully.
      • First it makes current broker unavailable and isolates from the clusters so, it will not serve any new requests.
      • Second it starts unloading namespace bundle one by one without closing the connection in order to avoid disruption for other namespacebundles which are sharing the same connection from the same client.

    • unloadNamespaceBundlesGracefully

      public void unloadNamespaceBundlesGracefully(int maxConcurrentUnload, boolean closeWithoutWaitingClientDisconnect)

    • getTopicIfExists

      public CompletableFuture<Optional<Topic>> getTopicIfExists(String topic)

    • getOrCreateTopic

      public CompletableFuture<Topic> getOrCreateTopic(String topic)

    • getTopic

      public CompletableFuture<Optional<Topic>> getTopic(String topic, boolean createIfMissing)

    • getTopic

      public CompletableFuture<Optional<Topic>> getTopic(String topic, boolean createIfMissing, Map<String,String> properties)

    • getTopic

      public CompletableFuture<Optional<Topic>> getTopic(org.apache.pulsar.common.naming.TopicName topicName, boolean createIfMissing, Map<String,String> properties)

    • deleteTopic

      public CompletableFuture<Void> deleteTopic(String topic, boolean forceDelete)

    • deleteTopicAuthenticationWithRetry

      public void deleteTopicAuthenticationWithRetry(String topic, CompletableFuture<Void> future, int count)

    • getReplicationClient

      public org.apache.pulsar.client.api.PulsarClient getReplicationClient(String cluster, Optional<org.apache.pulsar.common.policies.data.ClusterData> clusterDataOp)

    • getClusterPulsarAdmin

      public org.apache.pulsar.client.admin.PulsarAdmin getClusterPulsarAdmin(String cluster, Optional<org.apache.pulsar.common.policies.data.ClusterData> clusterDataOp)

    • loadOrCreatePersistentTopic

      protected CompletableFuture<Optional<Topic>> loadOrCreatePersistentTopic(String topic, boolean createIfMissing, Map<String,String> properties) throws RuntimeException
      It creates a topic async and returns CompletableFuture. It also throttles down configured max-concurrent topic loading and puts them into queue once in-process topics are created.
      Parameters:
      topic - persistent-topic name
      Returns:
      CompletableFuture
      Throws:
      RuntimeException

    • createPersistentTopic0

      public void createPersistentTopic0(String topic, boolean createIfMissing, CompletableFuture<Optional<Topic>> topicFuture, Map<String,String> properties)

    • getManagedLedgerConfig

      public CompletableFuture<org.apache.bookkeeper.mledger.ManagedLedgerConfig> getManagedLedgerConfig(org.apache.pulsar.common.naming.TopicName topicName)

    • refreshTopicToStatsMaps

      public void refreshTopicToStatsMaps(NamespaceBundle oldBundle)

    • getOfflineTopicStat

      public org.apache.pulsar.common.policies.data.PersistentOfflineTopicStats getOfflineTopicStat(org.apache.pulsar.common.naming.TopicName topicName)

    • cacheOfflineTopicStats

      public void cacheOfflineTopicStats(org.apache.pulsar.common.naming.TopicName topicName, org.apache.pulsar.common.policies.data.PersistentOfflineTopicStats offlineTopicStats)

    • invalidateOfflineTopicStatCache

      public void invalidateOfflineTopicStatCache(org.apache.pulsar.common.naming.TopicName topicName)

    • getTopicReference

      public Optional<Topic> getTopicReference(String topic)
      Get a reference to a topic that is currently loaded in the broker. This method will not make the broker attempt to load the topic if it's not already.

    • updateRates

      public void updateRates()

    • getDimensionMetrics

      public void getDimensionMetrics(Consumer<io.netty.buffer.ByteBuf> consumer)

    • getTopicMetrics

      public List<org.apache.pulsar.common.stats.Metrics> getTopicMetrics()

    • getBundleStats

      public Map<String,org.apache.pulsar.policies.data.loadbalancer.NamespaceBundleStats> getBundleStats()

    • getLookupRequestSemaphore

      public Semaphore getLookupRequestSemaphore()

    • checkGC

      public void checkGC()

    • checkMessageExpiry

      public void checkMessageExpiry()

    • checkReplicationPolicies

      public void checkReplicationPolicies()

    • checkCompaction

      public void checkCompaction()

    • checkMessageDeduplicationInfo

      public void checkMessageDeduplicationInfo()

    • checkInactiveSubscriptions

      public void checkInactiveSubscriptions()

    • checkTopicPublishThrottlingRate

      public void checkTopicPublishThrottlingRate()

    • checkBrokerPublishThrottlingRate

      public void checkBrokerPublishThrottlingRate()

    • forEachTopic

      public void forEachTopic(Consumer<Topic> consumer)
      Iterates over all loaded topics in the broker.

    • forEachPersistentTopic

      public void forEachPersistentTopic(Consumer<PersistentTopic> consumer)

    • getBacklogQuotaManager

      public BacklogQuotaManager getBacklogQuotaManager()

    • monitorBacklogQuota

      public void monitorBacklogQuota()

    • isTopicNsOwnedByBroker

      public boolean isTopicNsOwnedByBroker(org.apache.pulsar.common.naming.TopicName topicName)

    • checkTopicNsOwnership

      public CompletableFuture<Void> checkTopicNsOwnership(String topic)

    • unloadServiceUnit

      public CompletableFuture<Integer> unloadServiceUnit(NamespaceBundle serviceUnit, boolean closeWithoutWaitingClientDisconnect, long timeout, TimeUnit unit)

    • cleanUnloadedTopicFromCache

      public void cleanUnloadedTopicFromCache(NamespaceBundle serviceUnit)

    • getAuthorizationService

      public org.apache.pulsar.broker.authorization.AuthorizationService getAuthorizationService()

    • removeTopicFromCache

      public CompletableFuture<Void> removeTopicFromCache(String topicName)

    • removeTopicFromCache

      public CompletableFuture<Void> removeTopicFromCache(Topic topic)

    • getNumberOfNamespaceBundles

      public int getNumberOfNamespaceBundles()

    • getTopics

      public org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<String,CompletableFuture<Optional<Topic>>> getTopics()

    • pulsar

      public PulsarService pulsar()

    • executor

      public io.netty.channel.EventLoopGroup executor()

    • getReplicationClients

      public org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<String,org.apache.pulsar.client.api.PulsarClient> getReplicationClients()

    • isAuthenticationEnabled

      public boolean isAuthenticationEnabled()

    • isAuthorizationEnabled

      public boolean isAuthorizationEnabled()

    • getKeepAliveIntervalSeconds

      public int getKeepAliveIntervalSeconds()

    • generateUniqueProducerName

      public String generateUniqueProducerName()

    • getTopicStats

      public Map<String,org.apache.pulsar.common.policies.data.stats.TopicStatsImpl> getTopicStats()

    • getAuthenticationService

      public org.apache.pulsar.broker.authentication.AuthenticationService getAuthenticationService()

    • getAllTopicsFromNamespaceBundle

      public List<Topic> getAllTopicsFromNamespaceBundle(String namespace, String bundle)

    • registerConfigurationListener

      public <T> void registerConfigurationListener(String configKey, Consumer<T> listener)
      Allows a listener to listen on update of ServiceConfiguration change, so listener can take appropriate action if any specific config-field value has been changed. On notification, listener should first check if config value has been changed and after taking appropriate action, listener should update config value with new value if it has been changed (so, next time listener can compare values on configMap change).
      Type Parameters:
      T -
      Parameters:
      configKey - : configuration field name
      listener - : listener which takes appropriate action on config-value change

    • getDelayedDeliveryTrackerFactory

      public DelayedDeliveryTrackerFactory getDelayedDeliveryTrackerFactory()

    • getDynamicConfiguration

      public static List<String> getDynamicConfiguration()

    • getRuntimeConfiguration

      public Map<String,String> getRuntimeConfiguration()

    • isDynamicConfiguration

      public static boolean isDynamicConfiguration(String key)

    • validateDynamicConfiguration

      public static boolean validateDynamicConfiguration(String key, String value)

    • fetchPartitionedTopicMetadataCheckAllowAutoCreationAsync

      public CompletableFuture<org.apache.pulsar.common.partition.PartitionedTopicMetadata> fetchPartitionedTopicMetadataCheckAllowAutoCreationAsync(org.apache.pulsar.common.naming.TopicName topicName)

    • fetchPartitionedTopicMetadataAsync

      public CompletableFuture<org.apache.pulsar.common.partition.PartitionedTopicMetadata> fetchPartitionedTopicMetadataAsync(org.apache.pulsar.common.naming.TopicName topicName)

    • getTopicOrderedExecutor

      public org.apache.bookkeeper.common.util.OrderedExecutor getTopicOrderedExecutor()

    • getMultiLayerTopicMap

      public org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<String,org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<String,org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<String,Topic>>> getMultiLayerTopicMap()

    • addUnAckedMessages

      public void addUnAckedMessages(PersistentDispatcherMultipleConsumers dispatcher, int numberOfMessages)
      If per-broker unacked message reached to limit then it blocks dispatcher if its unacked message limit has been reached to maxUnackedMsgsPerDispatcher.
      Parameters:
      dispatcher -
      numberOfMessages -

    • checkUnAckMessageDispatching

      public void checkUnAckMessageDispatching()
      Adds given dispatcher's unackMessage count to broker-unack message count and if it reaches to the maxUnackedMessages then it blocks all the dispatchers which has unack-messages higher than maxUnackedMsgsPerDispatcher. It unblocks all dispatchers once broker-unack message counts decreased to (maxUnackedMessages/2)

    • isBrokerDispatchingBlocked

      public boolean isBrokerDispatchingBlocked()

    • unblockDispatchersOnUnAckMessages

      public void unblockDispatchersOnUnAckMessages(List<PersistentDispatcherMultipleConsumers> dispatcherList)
      Unblocks the dispatchers and removes it from the blockedDispatchers list.
      Parameters:
      dispatcherList -

    • extractTopic

      public static Optional<Topic> extractTopic(CompletableFuture<Optional<Topic>> topicFuture)
      Safely extract optional topic instance from a future, in a way to avoid unchecked exceptions and race conditions.

    • getListenPort

      public Optional<Integer> getListenPort()

    • getListenPortTls

      public Optional<Integer> getListenPortTls()

    • isAllowAutoTopicCreation

      public boolean isAllowAutoTopicCreation(String topic)

    • isAllowAutoTopicCreation

      public boolean isAllowAutoTopicCreation(org.apache.pulsar.common.naming.TopicName topicName)

    • isAllowAutoTopicCreation

      public boolean isAllowAutoTopicCreation(org.apache.pulsar.common.naming.TopicName topicName, Optional<org.apache.pulsar.common.policies.data.Policies> policies)

    • isDefaultTopicTypePartitioned

      public boolean isDefaultTopicTypePartitioned(org.apache.pulsar.common.naming.TopicName topicName, Optional<org.apache.pulsar.common.policies.data.Policies> policies)

    • getDefaultNumPartitions

      public int getDefaultNumPartitions(org.apache.pulsar.common.naming.TopicName topicName, Optional<org.apache.pulsar.common.policies.data.Policies> policies)

    • isAllowAutoSubscriptionCreation

      public boolean isAllowAutoSubscriptionCreation(String topic)

    • isAllowAutoSubscriptionCreation

      public boolean isAllowAutoSubscriptionCreation(org.apache.pulsar.common.naming.TopicName topicName)

    • isSystemTopic

      public boolean isSystemTopic(String topic)

    • isSystemTopic

      public boolean isSystemTopic(org.apache.pulsar.common.naming.TopicName topicName)

    • getTopicPolicies

      public Optional<org.apache.pulsar.common.policies.data.TopicPolicies> getTopicPolicies(org.apache.pulsar.common.naming.TopicName topicName)
      Get TopicPolicies for the parameterized topic.
      Parameters:
      topicName -
      Returns:
      TopicPolicies, if they exist. Otherwise, the value will not be present.

    • deleteTopicPolicies

      public CompletableFuture<Void> deleteTopicPolicies(org.apache.pulsar.common.naming.TopicName topicName)

    • setInterceptor

      public void setInterceptor(BrokerInterceptor interceptor)

    • getBrokerEntryMetadataInterceptors

      public Set<org.apache.pulsar.common.intercept.BrokerEntryMetadataInterceptor> getBrokerEntryMetadataInterceptors()

    • isBrokerEntryMetadataEnabled

      public boolean isBrokerEntryMetadataEnabled()

    • isBrokerPayloadProcessorEnabled

      public boolean isBrokerPayloadProcessorEnabled()

    • pausedConnections

      public void pausedConnections(int numberOfConnections)

    • resumedConnections

      public void resumedConnections(int numberOfConnections)

    • getPausedConnections

      public long getPausedConnections()

    • newTopic

      public <T extends Topic> T newTopic(String topic, org.apache.bookkeeper.mledger.ManagedLedger ledger, BrokerService brokerService, Class<T> topicClazz) throws org.apache.pulsar.broker.PulsarServerException
      Throws:
      org.apache.pulsar.broker.PulsarServerException

    • setPulsarChannelInitializerFactory

      public void setPulsarChannelInitializerFactory(PulsarChannelInitializer.Factory factory)