Class KillStalePendingSegments

  • All Implemented Interfaces:
    CoordinatorDuty

    public class KillStalePendingSegments
    extends Object
    implements CoordinatorDuty
    Duty to kill stale pending segments which are not needed anymore. Pending segments are created when appending realtime or batch tasks allocate segments to build incremental indexes. Under normal operation, these pending segments get committed when the task completes and become regular segments. But in case of task failures, some pending segments might be left around and cause clutter in the metadata store.

    While cleaning up, this duty ensures that the following pending segments are retained for at least DURATION_TO_RETAIN:

    • Pending segments created by any active task (across all datasources)
    • Pending segments created by the latest completed task (across all datasources)
    • Constructor Detail

      • KillStalePendingSegments

        public KillStalePendingSegments​(OverlordClient overlordClient)
    • Method Detail

      • run

        public DruidCoordinatorRuntimeParams run​(DruidCoordinatorRuntimeParams params)
        Description copied from interface: CoordinatorDuty
        Implementations of this method run various activities performed by the coordinator. Input params can be used and modified. The returned DruidCoordinatorRuntimeParams is passed to the next duty.
        Specified by:
        run in interface CoordinatorDuty
        Returns:
        same as input or a modified value to be used by next duty. Null return values will prevent subsequent CoordinatorDuty objects (see the order in the lists passed into the constructor of DruidCoordinator.DutiesRunnable) from running until the next cycle.