All Classes and Interfaces
Class
Description
Instances of this state use a single PageBuilder for all groups.
Transforms
Adds INTERMEDIATE aggregations between an un-grouped FINAL aggregation and its preceding
PARTIAL aggregation.
Group input data and produce a single block for each sequence of identical values.
Analysis for predicates such as
x IN (subquery) or x = SOME (subquery)This scalar function exists primarily to test lambda expression support.
Replaces
TableExecuteNode with TableExecuteNode.getPreferredPartitioningScheme()
with a TableExecuteNode with TableExecuteNode.getPartitioningScheme() set.Rule verifies if preconditions for using preferred write partitioning are met:
- expected number of partitions to be written (based on table stat) is greater
than or equal to preferred_write_partitioning_min_number_of_partitions session property,
- use_preferred_write_partitioning is set to true.
Calculate the histogram approximately for topk elements based on the
Space-Saving algorithm and the Stream-Summary data structure
as described in:
Efficient Computation of Frequent and Top-k Elements in Data Streams
by Metwally, Agrawal, and Abbadi
A buffer that assigns pages to queues based on a first come, first served basis.
This class provides replacements for TestNG's faulty assertion methods.
This class is inspired by com.google.common.io.Closer
Lightweight version of QueryInfo.
Lightweight version of QueryStats.
This
WorkProcessorOperator adapter allows to adapt WorkProcessor operators
that do not require special input handling (e.g streaming operators).
Aggregation function that approximates the frequency of the top-K elements.
Value-based, immutable
byte[] equivalent.Concatenate all but the first argument with separators.
For every row from
CorrelatedJoinNode.input a CorrelatedJoinNode.subquery relation is calculated.Simple implementation of CostCalculator.
A wrapper around CostCalculator that estimates ExchangeNodes cost.
This rule finds correlated UnnestNode in CorrelatedJoinNode's subquery and folds
them into UnnestNode representing LEFT JOIN UNNEST.
This rule finds correlated UnnestNode in CorrelatedJoinNode's subquery and folds
them into UnnestNode representing LEFT JOIN UNNEST.
This rule decorrelates plans with correlated UnnestNode and optional EnforceSingleRowNode,
optional LimitNode, optional TopNNode and optional projections in the subquery.
This rule must run after the distribution type has already been set for delete queries,
since semi joins in delete queries must be replicated.
Determines whether a given Expression is deterministic
Apply saturated floor casts for implicit coercions on TupleDomain.
Only calling getDriverStats is ThreadSafe
Methods setWithDelay and reset should be used in pairs;
usually follow the following idiom:
When dynamic filter assignments are present on a Join node, they should be consumed by a Filter node on it's probe side
This operator acts as a simple "pass-through" pipe, while saving its input pages.
See
LifespanScheduler about thread safetyComputes the effective predicate at the top of the specified PlanNode
Makes equality based inferences to rewrite Expressions and generate equality sets in terms of specified symbol scopes
Converts an Intersect node with at least one empty branch to an empty Values node.
Replaces 0% sample node with empty values node.
Transforms:
Applies to broadcast spatial joins, inner and left, expressed via ST_Contains,
ST_Intersects and ST_Distance functions.
Only retains rows that have identical values for each respective fieldSet.
See
LifespanScheduler about thread safetyVisitor to count number of tables scanned in the current fragment
(fragments separated by ExchangeNodes).
Extract expressions that are free (unbound) references to a lambda argument.
state object that uses a single histogram for all groups.
Implementations provide various top N page processing strategies for multiple groups.
Memory Layout:
This class finds the top N rows by rank value defined by and for each
group specified by .
Memory Layout:
This class finds the top N rows defined by for each group specified by .
implementation that uses groupId in the hash key, so that we may store all groupId x value -> count in one giant map.
Estimates number of collisions when inserting values into hash table.
If heap nodes are numbered top down, and left to right as follows:
For each request, the addPage method will be called zero or more times,
followed by either requestComplete or clientFinished (if buffer complete).
Object registration system that allows looking up objects via stable IDs.
Transforms:
Implement EXCEPT ALL using union, window and filter.
Converts EXCEPT DISTINCT queries into UNION ALL..GROUP BY...WHERE
E.g.:
Implements filtered aggregations by transforming plans of the following shape:
Implement INTERSECT ALL using union, window and filter.
Converts INTERSECT DISTINCT queries into UNION ALL..GROUP BY...WHERE
E.g.:
Transforms:
Transforms:
Hash array size supplier that uses larger load factors for larger arrays
A priority queue with constant time contains(E) and log time remove(E)
Ties are broken by insertion order
Identify the mapping from the lookup symbols used at the top of the index plan to
the actual symbols produced by the IndexSource.
Replace filter predicate conjuncts with underlying project expressions.
Inlines expressions from a child project node into a parent project node
as long as they are simple constants, or they are referenced only once (to
avoid introducing duplicate computation) and the references don't appear
within a TRY block (to avoid changing semantics).
A node is a server in a cluster than can process queries.
Resource groups form a tree, and all access to a group is guarded by the root of the tree.
This scalar function exists primarily to test lambda expression support.
Remove empty pattern from pattern alternation and replace it with quantification of a neighbouring term.
Optimize row pattern:
- remove nested exclusions
- flatten alternations and concatenations
- remove redundant empty pattern
A bridge that connects build, probe, and outer operators of a join.
JoinGraph represents sequence of Joins, where nodes in the graph
are PlanNodes that are being joined and edges are all equality join
conditions between pair of nodes.
This class must be public as it is accessed via join compiler reflection.
Extracts values from JSON
The stack representation for JSON objects must have the keys in natural sorted order.
Represents inherent cost of some plan node, not including cost of its sources.
Rewriter for expressions specific to row pattern recognition.
A type-specific array-based FIFO queue, supporting also deque operations.
This page builder creates pages with dictionary blocks:
normal dictionary blocks for the probe side and the original blocks for the build side.
This class computes an aggregate function result in row pattern recognition context.
Materialize all values in a row
Special handling is added for Double types for approximate comparisons
Stores a plan in a form that's efficient to mutate locally (i.e.
Manages memory pools on this worker node
This class performs merge of previously hash sorted pages streams.
Transforms:
Transforms:
This rule handles both LimitNode with ties and LimitNode without ties.
This rule handles both LimitNode with ties and LimitNode without ties.
This class is intended to be used right after the PageProcessor to ensure
that the size of the pages returned by FilterAndProject and ScanFilterAndProject
is big enough so it does not introduce considerable synchronization overhead.
Transforms:
Transforms:
Converts cardinality-insensitive aggregations (max, min, "distinct") over partition keys
into simple metadata queries
Implements distinct aggregations with different inputs by transforming plans of the following shape:
Represents a location in the network topology.
Implementations of this interface must be thread safe.
Used on workers.
Used on workers.
Used on workers.
Contains information about
Operator execution.For empty aggregations duplicate input rows can be skipped.
Provides a bridge between classic
Operator push input model
and WorkProcessorOperator pull model.Rewrite input references from columns in the input page (to the filter/project node)
into a compact list that can be used for method parameters.
PagesIndex a low-level data structure which contains the address of every value position of every channel.
This factory is used to pass spatial index built by SpatialIndexBuilderOperator
to SpatialJoinOperators.
Equals and hash evaluated at the specified page positions.
This class represents set of three collections representing implementations of operators
similar to partial template specialization from C++ allowing more optimized implementations to be provided for specific types.
Schedules stages choosing to order to provide the best resource utilization.
This class is designed to facilitate the pipelined mode of execution.
Clones plan and assigns new PlanNodeIds to the copied PlanNodes.
Splits a logical plan into fragments that can be shipped and executed on distributed nodes
A carrier of core, global, non-derived services for planner and analyzer.
It is going to be executed to verify logical planner correctness
This class is responsible for iterating over build rows, which have
same values in hash columns as given probe row (but according to
filterFunction can have non matching values on some other column).
Replaces the prefix given by
packagePrefix with the domain base provided via configuration.Task (and split) priority is composed of a level and a within-level
priority.
This class represents a WindowIndex with additional "channels"
which are not part of the underlying PagesIndex.
This rule restricts the outputs of ApplyNode's input and subquery
based on which ApplyNode's output symbols are referenced.
This rule updates ApplyNode's correlation list.
This rule restricts outputs of ApplyNode's subquery to include only the symbols
needed for subqueryAssignments.
This rule restricts the outputs of CorrelatedJoinNode's input and subquery
based on which CorrelatedJoinNode's output symbols are referenced.
This rule updates CorrelatedJoinNode's correlation list.
A count over a subquery can be reduced to a VALUES(1) provided
the subquery is a scalar
This rule restricts the outputs of ExchangeNode based on which
ExchangeNode's output symbols are either referenced by the
parent node or used for partitioning, ordering or as a hash
symbol by the ExchangeNode.
This rule prunes GroupIdNode's aggregationArguments.
Joins support output symbol selection, so make any project-off of child columns explicit in project nodes.
Joins support output symbol selection, so absorb any project-off into the node.
This rule restricts the inputs to PatternRecognitionNode
based on which symbols are used by the inner structures
of the PatternRecognitionNode.
Spatial joins support output symbol selection, so absorb any project-off into the node.
This is a special case of PushProjectionIntoTableScan that performs only column pruning.
Transforms
Absorb pruning projection into the node.
This optimizer pushes aggregations below outer joins when: the aggregation
is on top of the outer join, it groups by all columns in the outer table, and
the outer rows are guaranteed to be distinct.
Transforms expressions of the form
Transforms:
Transforms:
Transforms:
Transforms:
Transforms:
Transforms:
Transforms:
Transforms:
Transforms:
Transforms:
Transforms:
Transforms:
Transforms:
Aggregate functions in pattern recognition context have special semantics.
Transforms:
Transforms:
Transforms:
Transforms:
These rules should not be run after AddExchanges so as not to overwrite the TableLayout
chosen by AddExchanges
This rule pushes filter predicate concerning row number symbol into RowNumberNode
by modifying maxRowCountPerPartition.
This rule pushes filter predicate concerning a ranking symbol into WindowNode
by converting it into TopNRankingNode.
Transforms:
Utility class for pushing projections through inner join so that joins are not separated
by a project node and can participate in cross join elimination or join reordering.
Pushes RemoteExchange node down through the AssignUniqueId to preserve
partitioned_on(unique) and grouped(unique) properties for the output of
the AssignUniqueId.
Transforms:
Transforms:
Output schema and buffer URIs for query.
Manage queries scheduled on this node
TODO: this needs to be merged with RowType at some point (when the type system is unified)
Remove the aggregation node that produces distinct rows from the Filtering source of a Semi join.
Flattens and removes duplicate conjuncts or disjuncts.
If the predicate for a delete is optimized to false, the target table scan
of the delete will be replaced with an empty values node.
Removes branches from an ExceptNode that are guaranteed to produce 0 rows.
If the predicate for a table execute is optimized to false, the target table scan
of table execute will be replaced with an empty values node.
Removes branches from a UnionNode that are guaranteed to produce 0 rows.
Removes 100% sample nodes.
Replace DistinctLimit node
1.
Given:
Removes projection nodes that only perform non-renaming identity projections
Remove Limit node when the subplan is guaranteed to produce fewer rows than the limit and
replace the plan with empty values if the limit count is 0.
Remove Offset node and its subplan when the subplan is guaranteed to produce no more rows than the offset
and replace the plan with empty values.
This rule removes SortNode being a source of LimitNode with ties.
Replace TopN node
1.
Dynamic filters are supported only right after TableScan and only if the subtree is on
1.
This rule transforms plans with join where one of the sources is
a single-row ValuesNode, and the join condition is `on true`.
This rule is complementary to RemoveRedundantJoin.
This rule transforms plans with joins, where:
- one of the sources is scalar and produces no output symbols.
Classes implementing this interface must be thread safe.
Defines the authentication and authorization required to access a REST resource.
Re-writes spatial_partitioning(geometry) aggregations into spatial_partitioning(envelope, partition_count)
on top of ST_Envelope(geometry) projection, e.g.
Strategy that combines both comparison and hash/equality over row IDs
Comparator strategy that evaluates over row IDs
Hash strategy that evaluates over row IDs
Reference to a row.
Page buffering manager that enables access to individual rows via stable row IDs.
Cursor that allows callers to advance through the registered page and dictate whether a specific position should
be preserved with a stable row ID.
A wrapper for Expressions that can be used as a key in maps and sets.
For the MinHash algorithm, see "On the resemblance and containment of documents" by Andrei Z.
This class returns a set of positions to aggregate over for an aggregate function in row pattern matching context.
Determines whether, and how, a callsite matches a generic function signature.
It is not yet proven whether this heuristic is any better or worse.
Simplify conditional expressions in filter predicate.
Implements distinct aggregations with similar inputs by transforming plans of the following shape:
Maintains position links in sorted order by build side expression.
Extracts sort expression to be used for creating
SortedPositionLinks from join filter expression.This class is merely a container used by coordinator to track tasks for a single stage.
Simple state machine which holds a single state.
Makes stats consistent
Current implementation is based on code points from Unicode and does ignore grapheme cluster boundaries.
Extracts and returns the stream of all expression subtrees within an Expression, including Expression itself
Verifies the plan does not contain any "syntactic sugar" from the AST.
Methods for creating and decoding synthetic addresses.
Converts delete followed immediately by table scan to a special table-only delete node
TableExecuteHandle wraps connectors ConnectorTableExecuteHandle which identifies instance of executing
specific table procedure o specific table.
Deprecated.
Holder of
ScheduledExecutorService used for task management.Manages tasks on this worker node
The purpose of this class is to determine whether two pattern matching threads
are equivalent.
Returns the top N rows from the source sorted according to the specified ordering in the keyChannelIndex channel.
Optimized hash table for streaming Top N peer group lookup operations.
Returns the top N rows from the source sorted according to the specified ordering in the keyChannelIndex channel.
This rule decorrelates a correlated subquery of LEFT correlated join with distinct operator (grouped aggregation with no aggregation assignments)
It is similar to TransformCorrelatedDistinctAggregationWithProjection rule, but does not support projection over aggregation in the subquery
This rule decorrelates a correlated subquery of LEFT correlated join with distinct operator (grouped aggregation with no aggregation assignments)
This rule decorrelates a correlated subquery with:
- single global aggregation, or
- global aggregation over distinct operator (grouped aggregation with no aggregation assignments)
It is similar to TransformCorrelatedGlobalAggregationWithProjection rule, but does not support projection over aggregation in the subquery
This rule decorrelates a correlated subquery of LEFT or INNER correlated join with:
- single global aggregation, or
- global aggregation over distinct operator (grouped aggregation with no aggregation assignments)
This rule decorrelates a correlated subquery of INNER correlated join with:
- single grouped aggregation, or
- grouped aggregation over distinct operator (grouped aggregation with no aggregation assignments)
It is similar to TransformCorrelatedGroupedAggregationWithProjection rule, but does not support projection over aggregation in the subquery
This rule decorrelates a correlated subquery of INNER correlated join with:
- single grouped aggregation, or
- grouped aggregation over distinct operator (grouped aggregation with no aggregation assignments)
Replaces correlated ApplyNode with InPredicate expression with SemiJoin
Tries to decorrelate subquery and rewrite it using normal join.
Scalar filter scan query is something like:
This optimizer can rewrite correlated single row subquery to projection in a way described here:
From:
EXISTS is modeled as (if correlated predicates are equality comparisons):
Rewrite filtering semi-join to inner join.
This optimizers looks for InPredicate expressions in ApplyNodes and replaces the nodes with SemiJoin nodes.
Filters out rows that do not match the values from the specified tuple
This class is to facilitate obtaining the type of an expression and its subexpressions
during planning (i.e., when interacting with IR expression).
A set of unique SQL values stored in a
Block.Ensures that all the expressions and FunctionCalls matches their output symbols
Re-maps symbol references that are just aliases of each other (e.g., due to projections like
$0 := $1)Given s of type S, a constant expression t of type T, and when an implicit
cast exists between S->T, converts expression of the form:
Transforms expressions of the form
ROW(x, y)[1] to x
and
CAST(ROW(x, y) AS row(f1 type1, f2 type2))[1] to CAST(x AS type1)Given x::row(t) and y::row(t), converts assignments of the form
Annotates a program element that is (only) used by generated code.
When an aggregation has an empty grouping set then a default value needs to be returned in the output (e.g: 0 for COUNT(*)).
Ensures that all dependencies (i.e., symbols in expressions) for a plan node are provided by its source nodes
Verifies that input of order-sensitive Limit is ordered by the pre-sorted symbols
Verifies that input of streaming aggregations is grouped on the grouping keys
helper class for
GroupedTypedHistogram
May be used for other cases that need a simple hash for values
Aggregation function that approximates the frequency of the top-K elements.
The interface is used when a function call needs to yield/pause during the process.
This
WorkProcessorOperator adapter allows to adapt WorkProcessor operators
that require customization of input handling (e.g aggregation operators that want to skip extra
buffering step or operators that require more sophisticated initial blocking condition).