Package org.apache.pinot.common.utils.helix

Class HelixHelper

  • public class HelixHelper
extends Object

    • Method Detail

      • cloneIdealState

        public static org.apache.helix.model.IdealState cloneIdealState​(org.apache.helix.model.IdealState idealState)

      • updateIdealState

        public static org.apache.helix.model.IdealState updateIdealState​(org.apache.helix.HelixManager helixManager,
                                                                 String resourceName,
                                                                 com.google.common.base.Function<org.apache.helix.model.IdealState,​org.apache.helix.model.IdealState> updater,
                                                                 RetryPolicy policy,
                                                                 boolean noChangeOk)
        Updates the ideal state, retrying if necessary in case of concurrent updates to the ideal state.
        Parameters:
        helixManager - The HelixManager used to interact with the Helix cluster
        resourceName - The resource for which to update the ideal state
        updater - A function that returns an updated ideal state given an input ideal state
        Returns:
        updated ideal state if successful, null if not

      • updateIdealState

        public static org.apache.helix.model.IdealState updateIdealState​(org.apache.helix.HelixManager helixManager,
                                                                 String resourceName,
                                                                 com.google.common.base.Function<org.apache.helix.model.IdealState,​org.apache.helix.model.IdealState> updater)

      • updateIdealState

        public static org.apache.helix.model.IdealState updateIdealState​(org.apache.helix.HelixManager helixManager,
                                                                 String resourceName,
                                                                 com.google.common.base.Function<org.apache.helix.model.IdealState,​org.apache.helix.model.IdealState> updater,
                                                                 RetryPolicy policy)

      • updateBrokerResource

        public static void updateBrokerResource​(org.apache.helix.HelixManager helixManager,
                                        String brokerId,
                                        List<String> brokerTags,
                                        @Nullable
                                        List<String> tablesAdded,
                                        @Nullable
                                        List<String> tablesRemoved)
        Updates broker resource ideal state for the given broker with the given broker tags. Optional tablesAdded and tablesRemoved can be provided to track the tables added/removed during the update.

      • getAllInstances

        public static List<String> getAllInstances​(org.apache.helix.HelixAdmin helixAdmin,
                                           String clusterName)
        Returns all instances for the given cluster.
        Parameters:
        helixAdmin - The HelixAdmin object used to interact with the Helix cluster
        clusterName - Name of the cluster for which to get all the instances for.
        Returns:
        Returns a List of strings containing the instance names for the given cluster.

      • getAllInstancesForResource

        public static Set<String> getAllInstancesForResource​(org.apache.helix.model.IdealState idealState)
        Returns all instances for the given resource.
        Parameters:
        idealState - IdealState of the resource for which to return the instances of.
        Returns:
        Returns a Set of strings containing the instance names for the given cluster.

      • setInstanceState

        public static void setInstanceState​(String instanceName,
                                    String clusterName,
                                    org.apache.helix.HelixAdmin admin,
                                    boolean enable)
        Toggle the state of the instance between OFFLINE and ONLINE.
        Parameters:
        instanceName - Name of the instance for which to toggle the state.
        clusterName - Name of the cluster to which the instance belongs.
        admin - HelixAdmin to access the cluster.
        enable - Set enable to true for ONLINE and FALSE for OFFLINE.

      • setStateForInstanceList

        public static void setStateForInstanceList​(List<String> instances,
                                           String clusterName,
                                           org.apache.helix.HelixAdmin admin,
                                           boolean enable)

      • setStateForInstanceSet

        public static void setStateForInstanceSet​(Set<String> instances,
                                          String clusterName,
                                          org.apache.helix.HelixAdmin admin,
                                          boolean enable)

      • getInstanceConfigsMapFor

        public static Map<String,​String> getInstanceConfigsMapFor​(String instanceName,
                                                                String clusterName,
                                                                org.apache.helix.HelixAdmin admin)

      • getInstanceScopefor

        public static org.apache.helix.model.HelixConfigScope getInstanceScopefor​(String clusterName,
                                                                          String instanceName)

      • getResourceScopeFor

        public static org.apache.helix.model.HelixConfigScope getResourceScopeFor​(String clusterName,
                                                                          String resourceName)

      • getResourceConfigsFor

        public static Map<String,​String> getResourceConfigsFor​(String clusterName,
                                                             String resourceName,
                                                             org.apache.helix.HelixAdmin admin)

      • updateResourceConfigsFor

        public static void updateResourceConfigsFor​(Map<String,​String> newConfigs,
                                            String resourceName,
                                            String clusterName,
                                            org.apache.helix.HelixAdmin admin)

      • deleteResourcePropertyFromHelix

        public static void deleteResourcePropertyFromHelix​(org.apache.helix.HelixAdmin admin,
                                                   String clusterName,
                                                   String resourceName,
                                                   String configKey)

      • getTableIdealState

        public static org.apache.helix.model.IdealState getTableIdealState​(org.apache.helix.HelixManager manager,
                                                                   String resourceName)

      • getExternalViewForResource

        public static org.apache.helix.model.ExternalView getExternalViewForResource​(org.apache.helix.HelixAdmin admin,
                                                                             String clusterName,
                                                                             String resourceName)

      • getBrokerResourceConfig

        public static Map<String,​String> getBrokerResourceConfig​(org.apache.helix.HelixAdmin admin,
                                                               String clusterName)

      • updateBrokerConfig

        public static void updateBrokerConfig​(Map<String,​String> brokerResourceConfig,
                                      org.apache.helix.HelixAdmin admin,
                                      String clusterName)

      • getBrokerIdealStates

        public static org.apache.helix.model.IdealState getBrokerIdealStates​(org.apache.helix.HelixAdmin admin,
                                                                     String clusterName)

      • removeResourceFromBrokerIdealState

        public static void removeResourceFromBrokerIdealState​(org.apache.helix.HelixManager helixManager,
                                                      String resourceTag)
        Remove a resource (offline/realtime table) from the Broker's ideal state.
        Parameters:
        helixManager - The HelixManager object for accessing helix cluster.
        resourceTag - Name of the resource that needs to be removed from Broker ideal state.

      • getOnlineInstanceFromExternalView

        public static Set<String> getOnlineInstanceFromExternalView​(org.apache.helix.model.ExternalView resourceExternalView)
        Returns the set of online instances from external view.
        Parameters:
        resourceExternalView - External view for the resource.
        Returns:
        Set<String> of online instances in the external view for the resource.

      • getOfflineInstanceFromExternalView

        public static Set<String> getOfflineInstanceFromExternalView​(org.apache.helix.model.ExternalView resourceExternalView)
        Get a set of offline instance from the external view of the resource.
        Parameters:
        resourceExternalView - External view of the resource
        Returns:
        Set of string instance names of the offline instances in the external view.

      • removeSegmentFromIdealState

        public static void removeSegmentFromIdealState​(org.apache.helix.HelixManager helixManager,
                                               String tableName,
                                               String segmentName)
        Remove the segment from the cluster.
        Parameters:
        helixManager - The HelixManager object to access the helix cluster.
        tableName - Name of the table to which the new segment is to be added.
        segmentName - Name of the new segment to be added

      • removeSegmentsFromIdealState

        public static void removeSegmentsFromIdealState​(org.apache.helix.HelixManager helixManager,
                                                String tableName,
                                                List<String> segments)

      • getInstanceConfigs

        public static List<org.apache.helix.model.InstanceConfig> getInstanceConfigs​(org.apache.helix.HelixManager helixManager)
        Returns the config for all the instances in the cluster.

      • getInstancesWithTag

        public static List<String> getInstancesWithTag​(org.apache.helix.HelixManager helixManager,
                                               String tag)
        Returns the instances in the cluster with the given tag.

      • getInstancesWithTag

        public static List<String> getInstancesWithTag​(List<org.apache.helix.model.InstanceConfig> instanceConfigs,
                                               String tag)
        Returns the instances in the cluster with the given tag. TODO: refactor code to use this method over getInstancesWithTag(HelixManager, String) if applicable to reuse instance configs in order to reduce ZK accesses

      • getInstancesConfigsWithTag

        public static List<org.apache.helix.model.InstanceConfig> getInstancesConfigsWithTag​(List<org.apache.helix.model.InstanceConfig> instanceConfigs,
                                                                                     String tag)

      • getEnabledInstancesWithTag

        public static List<String> getEnabledInstancesWithTag​(org.apache.helix.HelixManager helixManager,
                                                      String tag)
        Returns the enabled instances in the cluster with the given tag.

      • getEnabledInstancesWithTag

        public static List<String> getEnabledInstancesWithTag​(List<org.apache.helix.model.InstanceConfig> instanceConfigs,
                                                      String tag)
        Returns the enabled instances in the cluster with the given tag. TODO: refactor code to use this method over getEnabledInstancesWithTag(HelixManager, String) if applicable to reuse instance configs in order to reduce ZK accesses

      • getServerInstancesForTenant

        public static Set<String> getServerInstancesForTenant​(org.apache.helix.HelixManager helixManager,
                                                      String tenant)
        Returns the server instances in the cluster for the given tenant.

      • getServerInstancesForTenant

        public static Set<String> getServerInstancesForTenant​(List<org.apache.helix.model.InstanceConfig> instanceConfigs,
                                                      String tenant)
        Returns the server instances in the cluster for the given tenant. TODO: refactor code to use this method if applicable to reuse instance configs in order to reduce ZK accesses

      • getServerInstancesForTenantWithType

        public static Set<String> getServerInstancesForTenantWithType​(List<org.apache.helix.model.InstanceConfig> instanceConfigs,
                                                              String tenant,
                                                              TableType tableType)
        Returns the server instances in the cluster for the given tenant name and tenant type. TODO: refactor code to use this method if applicable to reuse instance configs in order to reduce ZK accesses

      • getBrokerInstancesForTenant

        public static Set<String> getBrokerInstancesForTenant​(List<org.apache.helix.model.InstanceConfig> instanceConfigs,
                                                      String tenant)
        Returns the broker instances in the cluster for the given tenant. TODO: refactor code to use this method if applicable to reuse instance configs in order to reduce ZK accesses

      • getBrokerInstanceConfigsForTenant

        public static Set<org.apache.helix.model.InstanceConfig> getBrokerInstanceConfigsForTenant​(List<org.apache.helix.model.InstanceConfig> instanceConfigs,
                                                                                           String tenant)

      • getTablesForBrokerTag

        public static Set<String> getTablesForBrokerTag​(org.apache.helix.HelixManager helixManager,
                                                String brokerTag)

      • getTablesForBrokerTags

        public static Set<String> getTablesForBrokerTags​(org.apache.helix.HelixManager helixManager,
                                                 List<String> brokerTags)

      • getInstanceConfig

        public static org.apache.helix.model.InstanceConfig getInstanceConfig​(org.apache.helix.HelixManager helixManager,
                                                                      String instanceId)
        Returns the instance config for a specific instance.

      • updateInstanceConfig

        public static void updateInstanceConfig​(org.apache.helix.HelixManager helixManager,
                                        org.apache.helix.model.InstanceConfig instanceConfig)
        Updates instance config to the Helix property store.

      • updateHostnamePort

        public static boolean updateHostnamePort​(org.apache.helix.model.InstanceConfig instanceConfig,
                                         String hostname,
                                         int port)
        Updates hostname and port in the instance config, returns true if the value is updated, false otherwise.

      • updateTlsPort

        public static boolean updateTlsPort​(org.apache.helix.model.InstanceConfig instanceConfig,
                                    int tlsPort)
        Updates a tlsPort value into Pinot instance config so it can be retrieved later
        Parameters:
        instanceConfig - the instance config to update
        tlsPort - the tlsPort number
        Returns:
        true if updated

      • addDefaultTags

        public static boolean addDefaultTags​(org.apache.helix.model.InstanceConfig instanceConfig,
                                     Supplier<List<String>> defaultTagsSupplier)
        Adds default tags to the instance config if no tag exists, returns true if the default tags are added, false otherwise.

        The defaultTagsSupplier is a function which is only invoked when the instance does not have any tag. E.g. () -> Collections.singletonList("DefaultTenant_BROKER").

      • removeDisabledPartitions

        public static boolean removeDisabledPartitions​(org.apache.helix.model.InstanceConfig instanceConfig)
        Removes the disabled partitions from the instance config. Sometimes a partition can be accidentally disabled, and not re-enabled for some reason. When an instance is restarted, we should remove these disabled partitions so that they can be processed.