org.apache.spark.sql.catalyst.optimizer
JoinReorderDPFilters
Related Doc:
package optimizer
Builds join graph information to be used by the filtering strategies.
Builds join graph information to be used by the filtering strategies. Currently, it builds the sets of star/non-star joins. It can be extended with the sets of connected/unconnected joins, which can be used to filter Cartesian products.
Returns true if expr can be evaluated using only the output of plan.
Returns true if expr can be evaluated using only the output of plan. This method
can be used to determine when it is acceptable to move expression evaluation within a query
plan.
For example consider a join between two relations R(a, b) and S(c, d).
- canEvaluate(EqualTo(a,b), R) returns true
- canEvaluate(EqualTo(a,c), R) returns false
- canEvaluate(Literal(1), R) returns true as literals CAN be evaluated on any plan
Returns true iff expr could be evaluated as a condition within join.
Returns true iff expr could be evaluated as a condition within join.
Applies the star-join filter that eliminates join combinations among star and non-star tables until the star join is built.
Applies the star-join filter that eliminates join combinations among star and non-star tables until the star join is built.
Given the oneSideJoinPlan/otherSideJoinPlan, which represent all the plan permutations generated by the DP join enumeration, and the star/non-star plans, the following plan combinations are allowed: 1. (oneSideJoinPlan U otherSideJoinPlan) is a subset of star-join 2. star-join is a subset of (oneSideJoinPlan U otherSideJoinPlan) 3. (oneSideJoinPlan U otherSideJoinPlan) is a subset of non star-join
It assumes the sets are disjoint.
Example query graph:
t1 d1 - t2 - t3 \ / f1 | d2
star: {d1, f1, d2} non-star: {t2, t1, t3}
level 0: (f1 ), (d2 ), (t3 ), (d1 ), (t1 ), (t2 ) level 1: {t3 t2 }, {f1 d2 }, {f1 d1 } level 2: {d2 f1 d1 } level 3: {t1 d1 f1 d2 }, {t2 d1 f1 d2 } level 4: {d1 t2 f1 t1 d2 }, {d1 t3 t2 f1 d2 } level 5: {d1 t3 t2 f1 t1 d2 }
One side of the join represented as a set of plan ids.
The other side of the join represented as a set of plan ids.
Star and non-star plans represented as sets of plan ids
Implements optional filters to reduce the search space for join enumeration.
1) Star-join filters: Plan star-joins together since they are assumed to have an optimal execution based on their RI relationship. 2) Cartesian products: Defer their planning later in the graph to avoid large intermediate results (expanding joins, in general). 3) Composite inners: Don't generate "bushy tree" plans to avoid materializing intermediate results.
Filters (2) and (3) are not implemented.