Package org.apache.pulsar.broker.loadbalance.impl

Class LoadManagerShared

  • public class LoadManagerShared
extends java.lang.Object
    This class contains code which in shared between the two load manager implementations.

    • Field Summary

      Fields 
      Modifier and Type Field Description
      static org.slf4j.Logger LOG  
      static int MIBI  

    • Method Summary

      All Methods Static Methods Concrete Methods 
      Modifier and Type Method Description
      static void applyNamespacePolicies​(org.apache.pulsar.common.naming.ServiceUnitId serviceUnit, SimpleResourceAllocationPolicies policies, java.util.Set<java.lang.String> brokerCandidateCache, java.util.Set<java.lang.String> availableBrokers, LoadManagerShared.BrokerTopicLoadingPredicate brokerTopicLoadingPredicate)  
      static void fillNamespaceToBundlesMap​(java.util.Set<java.lang.String> bundles, org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashSet<java.lang.String>> target)
      Using the given bundles, populate the namespace to bundle range map.
      static void filterAntiAffinityGroupOwnedBrokers​(PulsarService pulsar, java.lang.String assignedBundleName, java.util.Set<java.lang.String> candidates, org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashSet<java.lang.String>>> brokerToNamespaceToBundleRange, java.util.Map<java.lang.String,​java.lang.String> brokerToDomainMap)
      It tries to filter out brokers which own namespace with same anti-affinity-group as given namespace.
      static void filterBrokersWithLargeTopicCount​(java.util.Set<java.lang.String> brokerCandidateCache, LoadData loadData, int loadBalancerBrokerMaxTopics)
      It filters out brokers which owns topic higher than configured threshold at ServiceConfiguration.loadBalancerBrokerMaxTopics.
      static java.util.concurrent.CompletableFuture<java.util.Map<java.lang.String,​java.lang.Integer>> getAntiAffinityNamespaceOwnedBrokers​(PulsarService pulsar, java.lang.String namespaceName, org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashSet<java.lang.String>>> brokerToNamespaceToBundleRange)
      It returns map of broker and count of namespace that are belong to the same anti-affinity group as given.
      static java.lang.String getBundleRangeFromBundleName​(java.lang.String bundleName)  
      static java.lang.String getNamespaceNameFromBundleName​(java.lang.String bundleName)  
      static org.apache.pulsar.policies.data.loadbalancer.SystemResourceUsage getSystemResourceUsage​(BrokerHostUsage brokerHostUsage)  
      static boolean isLoadSheddingEnabled​(PulsarService pulsar)
      If load balancing is enabled, load shedding is enabled by default unless forced off by dynamic configuration.
      static void removeMostServicingBrokersForNamespace​(java.lang.String assignedBundleName, java.util.Set<java.lang.String> candidates, org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashSet<java.lang.String>>> brokerToNamespaceToBundleRange)
      Removes the brokers which have more bundles assigned to them in the same namespace as the incoming bundle than at least one other available broker from consideration.
      static boolean shouldAntiAffinityNamespaceUnload​(java.lang.String namespace, java.lang.String bundle, java.lang.String currentBroker, PulsarService pulsar, org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashSet<java.lang.String>>> brokerToNamespaceToBundleRange, java.util.Set<java.lang.String> candidateBrokers)
      It checks if given anti-affinity namespace should be unloaded by broker due to load-shedding.

      • Methods inherited from class java.lang.Object

        clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

    • Field Detail

      • LOG

        public static final org.slf4j.Logger LOG

    • Method Detail

      • fillNamespaceToBundlesMap

        public static void fillNamespaceToBundlesMap​(java.util.Set<java.lang.String> bundles,
                                             org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashSet<java.lang.String>> target)
        Using the given bundles, populate the namespace to bundle range map.
        Parameters:
        bundles - Bundles with which to populate.
        target - Map to fill.

      • getBundleRangeFromBundleName

        public static java.lang.String getBundleRangeFromBundleName​(java.lang.String bundleName)

      • getNamespaceNameFromBundleName

        public static java.lang.String getNamespaceNameFromBundleName​(java.lang.String bundleName)

      • getSystemResourceUsage

        public static org.apache.pulsar.policies.data.loadbalancer.SystemResourceUsage getSystemResourceUsage​(BrokerHostUsage brokerHostUsage)

      • isLoadSheddingEnabled

        public static boolean isLoadSheddingEnabled​(PulsarService pulsar)
        If load balancing is enabled, load shedding is enabled by default unless forced off by dynamic configuration.
        Returns:
        true by default

      • removeMostServicingBrokersForNamespace

        public static void removeMostServicingBrokersForNamespace​(java.lang.String assignedBundleName,
                                                          java.util.Set<java.lang.String> candidates,
                                                          org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashSet<java.lang.String>>> brokerToNamespaceToBundleRange)
        Removes the brokers which have more bundles assigned to them in the same namespace as the incoming bundle than at least one other available broker from consideration.
        Parameters:
        assignedBundleName - Name of bundle to be assigned.
        candidates - BrokersBase available for placement.
        brokerToNamespaceToBundleRange - Map from brokers to namespaces to bundle ranges.

      • filterAntiAffinityGroupOwnedBrokers

        public static void filterAntiAffinityGroupOwnedBrokers​(PulsarService pulsar,
                                                       java.lang.String assignedBundleName,
                                                       java.util.Set<java.lang.String> candidates,
                                                       org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashSet<java.lang.String>>> brokerToNamespaceToBundleRange,
                                                       java.util.Map<java.lang.String,​java.lang.String> brokerToDomainMap)
        It tries to filter out brokers which own namespace with same anti-affinity-group as given namespace. If all the domains own namespace with same anti-affinity group then it will try to keep brokers with domain that has least number of namespaces. It also tries to keep brokers which has least number of namespace with in domain. eg. 
         Before:
 Domain-count  BrokersBase-count
 ____________  ____________
 d1-3          b1-2,b2-1
 d2-3          b3-2,b4-1
 d3-4          b5-2,b6-2

 After filtering: "candidates" brokers
 Domain-count  BrokersBase-count
 ____________  ____________
 d1-3          b2-1
 d2-3          b4-1

 "candidate" broker to own anti-affinity-namespace = b2 or b4
        Parameters:
        pulsar -
        assignedBundleName -
        candidates -
        brokerToNamespaceToBundleRange -

      • getAntiAffinityNamespaceOwnedBrokers

        public static java.util.concurrent.CompletableFuture<java.util.Map<java.lang.String,​java.lang.Integer>> getAntiAffinityNamespaceOwnedBrokers​(PulsarService pulsar,
                                                                                                                                                   java.lang.String namespaceName,
                                                                                                                                                   org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashSet<java.lang.String>>> brokerToNamespaceToBundleRange)
        It returns map of broker and count of namespace that are belong to the same anti-affinity group as given. {@param namespaceName}
        Parameters:
        pulsar -
        namespaceName -
        brokerToNamespaceToBundleRange -
        Returns:

      • shouldAntiAffinityNamespaceUnload

        public static boolean shouldAntiAffinityNamespaceUnload​(java.lang.String namespace,
                                                        java.lang.String bundle,
                                                        java.lang.String currentBroker,
                                                        PulsarService pulsar,
                                                        org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<java.lang.String,​org.apache.pulsar.common.util.collections.ConcurrentOpenHashSet<java.lang.String>>> brokerToNamespaceToBundleRange,
                                                        java.util.Set<java.lang.String> candidateBrokers)
                                                 throws java.lang.Exception
        It checks if given anti-affinity namespace should be unloaded by broker due to load-shedding. If all the brokers are owning same number of anti-affinity namespaces then unloading this namespace again ends up at the same broker from which it was unloaded. So, this util checks that given namespace should be unloaded only if it can be loaded by different broker.
        Parameters:
        namespace -
        bundle -
        currentBroker -
        pulsar -
        brokerToNamespaceToBundleRange -
        candidateBrokers -
        Returns:
        Throws:
        java.lang.Exception

      • filterBrokersWithLargeTopicCount

        public static void filterBrokersWithLargeTopicCount​(java.util.Set<java.lang.String> brokerCandidateCache,
                                                    LoadData loadData,
                                                    int loadBalancerBrokerMaxTopics)
        It filters out brokers which owns topic higher than configured threshold at ServiceConfiguration.loadBalancerBrokerMaxTopics.
        if all the brokers own topic higher than threshold then it resets the list with original broker candidates
        Parameters:
        brokerCandidateCache -
        loadData -
        loadBalancerBrokerMaxTopics -