Package org.apache.pulsar.broker.loadbalance.impl

Class LoadManagerShared

java.lang.Object
org.apache.pulsar.broker.loadbalance.impl.LoadManagerShared
public class LoadManagerShared extends Object
This class contains code which in shared between the two load manager implementations.

  • Field Details

    • LOG

      public static final org.slf4j.Logger LOG

    • MIBI

      public static final int MIBI
      See Also:

  • Method Details

    • applyNamespacePolicies

      public static void applyNamespacePolicies(org.apache.pulsar.common.naming.ServiceUnitId serviceUnit, SimpleResourceAllocationPolicies policies, Set<String> brokerCandidateCache, Set<String> availableBrokers, LoadManagerShared.BrokerTopicLoadingPredicate brokerTopicLoadingPredicate)

    • fillNamespaceToBundlesMap

      public static void fillNamespaceToBundlesMap(Set<String> bundles, org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<String,org.apache.pulsar.common.util.collections.ConcurrentOpenHashSet<String>> target)
      Using the given bundles, populate the namespace to bundle range map.
      Parameters:
      bundles - Bundles with which to populate.
      target - Map to fill.

    • getBundleRangeFromBundleName

      public static String getBundleRangeFromBundleName(String bundleName)

    • getNamespaceNameFromBundleName

      public static String getNamespaceNameFromBundleName(String bundleName)

    • getSystemResourceUsage

      public static org.apache.pulsar.policies.data.loadbalancer.SystemResourceUsage getSystemResourceUsage(BrokerHostUsage brokerHostUsage)

    • isLoadSheddingEnabled

      public static boolean isLoadSheddingEnabled(PulsarService pulsar)
      If load balancing is enabled, load shedding is enabled by default unless forced off by dynamic configuration.
      Returns:
      true by default

    • removeMostServicingBrokersForNamespace

      public static void removeMostServicingBrokersForNamespace(String assignedBundleName, Set<String> candidates, org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<String,org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<String,org.apache.pulsar.common.util.collections.ConcurrentOpenHashSet<String>>> brokerToNamespaceToBundleRange)
      Removes the brokers which have more bundles assigned to them in the same namespace as the incoming bundle than at least one other available broker from consideration.
      Parameters:
      assignedBundleName - Name of bundle to be assigned.
      candidates - BrokersBase available for placement.
      brokerToNamespaceToBundleRange - Map from brokers to namespaces to bundle ranges.

    • filterAntiAffinityGroupOwnedBrokers

      public static void filterAntiAffinityGroupOwnedBrokers(PulsarService pulsar, String assignedBundleName, Set<String> candidates, org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<String,org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<String,org.apache.pulsar.common.util.collections.ConcurrentOpenHashSet<String>>> brokerToNamespaceToBundleRange, Map<String,String> brokerToDomainMap)
      It tries to filter out brokers which own namespace with same anti-affinity-group as given namespace. If all the domains own namespace with same anti-affinity group then it will try to keep brokers with domain that has least number of namespaces. It also tries to keep brokers which has least number of namespace with in domain. eg. 
       Before:
 Domain-count  BrokersBase-count
 ____________  ____________
 d1-3          b1-2,b2-1
 d2-3          b3-2,b4-1
 d3-4          b5-2,b6-2

 After filtering: "candidates" brokers
 Domain-count  BrokersBase-count
 ____________  ____________
 d1-3          b2-1
 d2-3          b4-1

 "candidate" broker to own anti-affinity-namespace = b2 or b4
      Parameters:
      pulsar -
      assignedBundleName -
      candidates -
      brokerToNamespaceToBundleRange -

    • getAntiAffinityNamespaceOwnedBrokers

      public static CompletableFuture<Map<String,Integer>> getAntiAffinityNamespaceOwnedBrokers(PulsarService pulsar, String namespaceName, org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<String,org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<String,org.apache.pulsar.common.util.collections.ConcurrentOpenHashSet<String>>> brokerToNamespaceToBundleRange)
      It returns map of broker and count of namespace that are belong to the same anti-affinity group as given.
      Parameters:
      pulsar -
      namespaceName -
      brokerToNamespaceToBundleRange -
      Returns:

    • shouldAntiAffinityNamespaceUnload

      public static boolean shouldAntiAffinityNamespaceUnload(String namespace, String bundle, String currentBroker, PulsarService pulsar, org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<String,org.apache.pulsar.common.util.collections.ConcurrentOpenHashMap<String,org.apache.pulsar.common.util.collections.ConcurrentOpenHashSet<String>>> brokerToNamespaceToBundleRange, Set<String> candidateBrokers) throws Exception
      It checks if given anti-affinity namespace should be unloaded by broker due to load-shedding. If all the brokers are owning same number of anti-affinity namespaces then unloading this namespace again ends up at the same broker from which it was unloaded. So, this util checks that given namespace should be unloaded only if it can be loaded by different broker.
      Parameters:
      namespace -
      bundle -
      currentBroker -
      pulsar -
      brokerToNamespaceToBundleRange -
      candidateBrokers -
      Returns:
      Throws:
      Exception

    • filterBrokersWithLargeTopicCount

      public static void filterBrokersWithLargeTopicCount(Set<String> brokerCandidateCache, LoadData loadData, int loadBalancerBrokerMaxTopics)
      It filters out brokers which owns topic higher than configured threshold at ServiceConfiguration.loadBalancerBrokerMaxTopics.
      if all the brokers own topic higher than threshold then it resets the list with original broker candidates
      Parameters:
      brokerCandidateCache -
      loadData -
      loadBalancerBrokerMaxTopics -