Package io.trino.server

Class DynamicFilterService

java.lang.Object

io.trino.server.DynamicFilterService

@ThreadSafe
public class DynamicFilterService
extends Object

Nested Class Summary

Nested Classes

Modifier and Type	Class	Description
static class	DynamicFilterService.DynamicFilterDomainStats
static class	DynamicFilterService.DynamicFiltersStats

Constructor Summary

Constructors

Constructor	Description
DynamicFilterService(Metadata metadata, FunctionManager functionManager, TypeOperators typeOperators, DynamicFilterConfig dynamicFilterConfig)

Method Summary

Modifier and Type	Method	Description
void	addTaskDynamicFilters(TaskId taskId, Map<DynamicFilterId,Domain> newDynamicFilters)
DynamicFilter	createDynamicFilter(QueryId queryId, List<DynamicFilters.Descriptor> dynamicFilterDescriptors, Map<Symbol,ColumnHandle> columnHandles)
DynamicFilterService.DynamicFiltersStats	getDynamicFilteringStats(QueryId queryId, Session session)
static Set<DynamicFilterId>	getOutboundDynamicFilters(PlanFragment plan)
boolean	isCollectingTaskNeeded(QueryId queryId, PlanFragment plan)	Dynamic filters are collected in same stage as the join operator in pipelined execution.
boolean	isStageSchedulingNeededToCollectDynamicFilters(QueryId queryId, PlanFragment plan)
void	registerDynamicFilterConsumer(QueryId queryId, int attemptId, Set<DynamicFilterId> dynamicFilterIds, Consumer<Map<DynamicFilterId,Domain>> consumer)
void	registerQuery(SqlQueryExecution sqlQueryExecution, SubPlan fragmentedPlan)
void	registerQuery(QueryId queryId, Session session, Set<DynamicFilterId> dynamicFilters, Set<DynamicFilterId> lazyDynamicFilters, Set<DynamicFilterId> replicatedDynamicFilters)
void	registerQueryRetry(QueryId queryId, int attemptId)
void	removeQuery(QueryId queryId)
void	stageCannotScheduleMoreTasks(StageId stageId, int attemptId, int numberOfTasks)
void	unblockStageDynamicFilters(QueryId queryId, int attemptId, PlanFragment plan)	Join build source tasks might become blocked waiting for join stage to collect build data.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Details

DynamicFilterService

@Inject
public DynamicFilterService(Metadata metadata,
 FunctionManager functionManager,
 TypeOperators typeOperators,
 DynamicFilterConfig dynamicFilterConfig)

Method Details

registerQuery

public void registerQuery(SqlQueryExecution sqlQueryExecution,
 SubPlan fragmentedPlan)

registerQuery

public void registerQuery(QueryId queryId,
 Session session,
 Set<DynamicFilterId> dynamicFilters,
 Set<DynamicFilterId> lazyDynamicFilters,
 Set<DynamicFilterId> replicatedDynamicFilters)

registerQueryRetry

public void registerQueryRetry(QueryId queryId,
 int attemptId)

getDynamicFilteringStats

public DynamicFilterService.DynamicFiltersStats getDynamicFilteringStats(QueryId queryId,
 Session session)

removeQuery

public void removeQuery(QueryId queryId)

isCollectingTaskNeeded

public boolean isCollectingTaskNeeded(QueryId queryId,
 PlanFragment plan)

Dynamic filters are collected in same stage as the join operator in pipelined execution. This can result in deadlock for source stage joins and connectors that wait for dynamic filters before generating splits (probe splits might be blocked on dynamic filters which require at least one probe task in order to be collected). To overcome this issue an initial task is created for source stages running broadcast join operator. This task allows for dynamic filters collection without any probe side splits being scheduled.

isStageSchedulingNeededToCollectDynamicFilters

public boolean isStageSchedulingNeededToCollectDynamicFilters(QueryId queryId,
 PlanFragment plan)

unblockStageDynamicFilters

public void unblockStageDynamicFilters(QueryId queryId,
 int attemptId,
 PlanFragment plan)

Join build source tasks might become blocked waiting for join stage to collect build data. In such case dynamic filters must be unblocked (and probe split generation resumed) for source stage containing joins to allow build source tasks to flush data and complete.

createDynamicFilter

public DynamicFilter createDynamicFilter(QueryId queryId,
 List<DynamicFilters.Descriptor> dynamicFilterDescriptors,
 Map<Symbol,ColumnHandle> columnHandles)

registerDynamicFilterConsumer

public void registerDynamicFilterConsumer(QueryId queryId,
 int attemptId,
 Set<DynamicFilterId> dynamicFilterIds,
 Consumer<Map<DynamicFilterId,Domain>> consumer)

addTaskDynamicFilters

public void addTaskDynamicFilters(TaskId taskId,
 Map<DynamicFilterId,Domain> newDynamicFilters)

stageCannotScheduleMoreTasks

public void stageCannotScheduleMoreTasks(StageId stageId,
 int attemptId,
 int numberOfTasks)

getOutboundDynamicFilters

public static Set<DynamicFilterId> getOutboundDynamicFilters(PlanFragment plan)