org.apache.hadoop.hive.ql.optimizer.calcite.rules

Class RelFieldTrimmer

java.lang.Object

org.apache.hadoop.hive.ql.optimizer.calcite.rules.RelFieldTrimmer

All Implemented Interfaces:

org.apache.calcite.util.ReflectiveVisitor

Direct Known Subclasses:

HiveRelFieldTrimmer

public class RelFieldTrimmer
extends Object
implements org.apache.calcite.util.ReflectiveVisitor

This class comes from Calcite almost as-is. The only change concerns the dispatcher and the builder, so the trimmer is thread-safe and can be reused across different queries. Definition follows.

Transformer that walks over a tree of relational expressions, replacing each RelNode with a 'slimmed down' relational expression that projects only the columns required by its consumer.

Uses multi-methods to fire the right rule for each type of relational expression. This allows the transformer to be extended without having to add a new method to RelNode, and without requiring a collection of rule classes scattered to the four winds.

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
protected static class	RelFieldTrimmer.TrimResult Result of an attempt to trim columns from a relational expression.

Field Summary

Fields

Modifier and Type	Field and Description
protected static ThreadLocal<org.apache.calcite.tools.RelBuilder>	REL_BUILDER
protected HiveReflectUtil.ClassMethodDispatcher<RelFieldTrimmer.TrimResult,org.apache.calcite.rel.RelNode>	trimFieldsDispatcher

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	RelFieldTrimmer(boolean useLMFBasedDispatcher) Creates a RelFieldTrimmer.

Method Summary

Modifier and Type	Method and Description
protected org.apache.calcite.util.mapping.Mapping	createMapping(org.apache.calcite.util.ImmutableBitSet fieldsUsed, int fieldCount)
protected RelFieldTrimmer.TrimResult	dispatchTrimFields(org.apache.calcite.rel.RelNode rel, org.apache.calcite.util.ImmutableBitSet fieldsUsed, Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields) Invokes trimFields(org.apache.calcite.rel.RelNode, org.apache.calcite.util.ImmutableBitSet, java.util.Set<org.apache.calcite.rel.type.RelDataTypeField>), or the appropriate method for the type of the rel parameter, using multi-method dispatch.
protected RelFieldTrimmer.TrimResult	dummyProject(int fieldCount, org.apache.calcite.rel.RelNode input) Creates a project with a dummy column, to protect the parts of the system that cannot handle a relational expression with no columns.
protected RelFieldTrimmer.TrimResult	result(org.apache.calcite.rel.RelNode r, org.apache.calcite.util.mapping.Mapping mapping)
org.apache.calcite.rel.RelNode	trim(org.apache.calcite.tools.RelBuilder relBuilder, org.apache.calcite.rel.RelNode root) Trims unused fields from a relational expression.
protected RelFieldTrimmer.TrimResult	trimChild(org.apache.calcite.rel.RelNode rel, org.apache.calcite.rel.RelNode input, org.apache.calcite.util.ImmutableBitSet fieldsUsed, Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields) Trims the fields of an input relational expression.
protected RelFieldTrimmer.TrimResult	trimChildRestore(org.apache.calcite.rel.RelNode rel, org.apache.calcite.rel.RelNode input, org.apache.calcite.util.ImmutableBitSet fieldsUsed, Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields) Trims a child relational expression, then adds back a dummy project to restore the fields that were removed.
RelFieldTrimmer.TrimResult	trimFields(org.apache.calcite.rel.core.Aggregate aggregate, org.apache.calcite.util.ImmutableBitSet fieldsUsed, Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields) Variant of trimFields(RelNode, ImmutableBitSet, Set) for LogicalAggregate.
RelFieldTrimmer.TrimResult	trimFields(org.apache.calcite.rel.core.Filter filter, org.apache.calcite.util.ImmutableBitSet fieldsUsed, Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields) Variant of trimFields(RelNode, ImmutableBitSet, Set) for LogicalFilter.
RelFieldTrimmer.TrimResult	trimFields(org.apache.calcite.rel.core.Join join, org.apache.calcite.util.ImmutableBitSet fieldsUsed, Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields) Variant of trimFields(RelNode, ImmutableBitSet, Set) for LogicalJoin.
RelFieldTrimmer.TrimResult	trimFields(org.apache.calcite.rel.logical.LogicalTableFunctionScan tabFun, org.apache.calcite.util.ImmutableBitSet fieldsUsed, Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields) Variant of trimFields(RelNode, ImmutableBitSet, Set) for LogicalTableFunctionScan.
RelFieldTrimmer.TrimResult	trimFields(org.apache.calcite.rel.logical.LogicalTableModify modifier, org.apache.calcite.util.ImmutableBitSet fieldsUsed, Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields) Variant of trimFields(RelNode, ImmutableBitSet, Set) for LogicalTableModify.
RelFieldTrimmer.TrimResult	trimFields(org.apache.calcite.rel.logical.LogicalValues values, org.apache.calcite.util.ImmutableBitSet fieldsUsed, Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields) Variant of trimFields(RelNode, ImmutableBitSet, Set) for LogicalValues.
RelFieldTrimmer.TrimResult	trimFields(org.apache.calcite.rel.core.Project project, org.apache.calcite.util.ImmutableBitSet fieldsUsed, Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields) Variant of trimFields(RelNode, ImmutableBitSet, Set) for LogicalProject.
RelFieldTrimmer.TrimResult	trimFields(org.apache.calcite.rel.RelNode rel, org.apache.calcite.util.ImmutableBitSet fieldsUsed, Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields) Visit method, per ReflectiveVisitor.
RelFieldTrimmer.TrimResult	trimFields(org.apache.calcite.rel.core.SetOp setOp, org.apache.calcite.util.ImmutableBitSet fieldsUsed, Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields) Variant of trimFields(RelNode, ImmutableBitSet, Set) for SetOp (including UNION and UNION ALL).
RelFieldTrimmer.TrimResult	trimFields(org.apache.calcite.rel.core.Sort sort, org.apache.calcite.util.ImmutableBitSet fieldsUsed, Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields) Variant of trimFields(RelNode, ImmutableBitSet, Set) for Sort.
RelFieldTrimmer.TrimResult	trimFields(org.apache.calcite.rel.core.TableScan tableAccessRel, org.apache.calcite.util.ImmutableBitSet fieldsUsed, Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields) Variant of trimFields(RelNode, ImmutableBitSet, Set) for LogicalTableScan.

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

REL_BUILDER

protected static final ThreadLocal<org.apache.calcite.tools.RelBuilder> REL_BUILDER

trimFieldsDispatcher

protected final HiveReflectUtil.ClassMethodDispatcher<RelFieldTrimmer.TrimResult,org.apache.calcite.rel.RelNode> trimFieldsDispatcher

Constructor Detail

RelFieldTrimmer

protected RelFieldTrimmer(boolean useLMFBasedDispatcher)

Creates a RelFieldTrimmer.

Parameters:

useLMFBasedDispatcher - True if we want to create a dispatcher based on LambdaMetafactory that is thread-safe, or false if we want to create a Calcite dispatcher based on reflection that is not thread-safe. False should only be used for testing/benchmarking purposes

Method Detail

trim

public org.apache.calcite.rel.RelNode trim(org.apache.calcite.tools.RelBuilder relBuilder,
                                           org.apache.calcite.rel.RelNode root)

Trims unused fields from a relational expression.

We presume that all fields of the relational expression are wanted by its consumer, so only trim fields that are not used within the tree.

Parameters:

relBuilder - Rel builder

root - Root node of relational expression

Returns:

Trimmed relational expression

trimChild

protected RelFieldTrimmer.TrimResult trimChild(org.apache.calcite.rel.RelNode rel,
                                               org.apache.calcite.rel.RelNode input,
                                               org.apache.calcite.util.ImmutableBitSet fieldsUsed,
                                               Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields)

Trims the fields of an input relational expression.

Parameters:

rel - Relational expression

input - Input relational expression, whose fields to trim

fieldsUsed - Bitmap of fields needed by the consumer

Returns:

New relational expression and its field mapping

trimChildRestore

protected RelFieldTrimmer.TrimResult trimChildRestore(org.apache.calcite.rel.RelNode rel,
                                                      org.apache.calcite.rel.RelNode input,
                                                      org.apache.calcite.util.ImmutableBitSet fieldsUsed,
                                                      Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields)

Trims a child relational expression, then adds back a dummy project to restore the fields that were removed.

Sounds pointless? It causes unused fields to be removed further down the tree (towards the leaves), but it ensure that the consuming relational expression continues to see the same fields.

Parameters:

rel - Relational expression

input - Input relational expression, whose fields to trim

fieldsUsed - Bitmap of fields needed by the consumer

Returns:

New relational expression and its field mapping

dispatchTrimFields

protected final RelFieldTrimmer.TrimResult dispatchTrimFields(org.apache.calcite.rel.RelNode rel,
                                                              org.apache.calcite.util.ImmutableBitSet fieldsUsed,
                                                              Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields)

Invokes trimFields(org.apache.calcite.rel.RelNode, org.apache.calcite.util.ImmutableBitSet, java.util.Set<org.apache.calcite.rel.type.RelDataTypeField>), or the appropriate method for the type of the rel parameter, using multi-method dispatch.

Parameters:

rel - Relational expression

fieldsUsed - Bitmap of fields needed by the consumer

Returns:

New relational expression and its field mapping

result

protected RelFieldTrimmer.TrimResult result(org.apache.calcite.rel.RelNode r,
                                            org.apache.calcite.util.mapping.Mapping mapping)

trimFields

public RelFieldTrimmer.TrimResult trimFields(org.apache.calcite.rel.RelNode rel,
                                             org.apache.calcite.util.ImmutableBitSet fieldsUsed,
                                             Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields)

Visit method, per ReflectiveVisitor.

This method is invoked reflectively, so there may not be any apparent calls to it. The class (or derived classes) may contain overloads of this method with more specific types for the rel parameter.

Returns a pair: the relational expression created, and the mapping between the original fields and the fields of the newly created relational expression.

Parameters:

rel - Relational expression

fieldsUsed - Fields needed by the consumer

Returns:

relational expression and mapping

trimFields

public RelFieldTrimmer.TrimResult trimFields(org.apache.calcite.rel.core.Project project,
                                             org.apache.calcite.util.ImmutableBitSet fieldsUsed,
                                             Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields)

Variant of trimFields(RelNode, ImmutableBitSet, Set) for LogicalProject.

dummyProject

protected RelFieldTrimmer.TrimResult dummyProject(int fieldCount,
                                                  org.apache.calcite.rel.RelNode input)

Creates a project with a dummy column, to protect the parts of the system that cannot handle a relational expression with no columns.

Parameters:

fieldCount - Number of fields in the original relational expression

input - Trimmed input

Returns:

Dummy project, or null if no dummy is required

trimFields

public RelFieldTrimmer.TrimResult trimFields(org.apache.calcite.rel.core.Filter filter,
                                             org.apache.calcite.util.ImmutableBitSet fieldsUsed,
                                             Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields)

Variant of trimFields(RelNode, ImmutableBitSet, Set) for LogicalFilter.

trimFields

public RelFieldTrimmer.TrimResult trimFields(org.apache.calcite.rel.core.Sort sort,
                                             org.apache.calcite.util.ImmutableBitSet fieldsUsed,
                                             Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields)

Variant of trimFields(RelNode, ImmutableBitSet, Set) for Sort.

trimFields

public RelFieldTrimmer.TrimResult trimFields(org.apache.calcite.rel.core.Join join,
                                             org.apache.calcite.util.ImmutableBitSet fieldsUsed,
                                             Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields)

Variant of trimFields(RelNode, ImmutableBitSet, Set) for LogicalJoin.

trimFields

public RelFieldTrimmer.TrimResult trimFields(org.apache.calcite.rel.core.SetOp setOp,
                                             org.apache.calcite.util.ImmutableBitSet fieldsUsed,
                                             Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields)

Variant of trimFields(RelNode, ImmutableBitSet, Set) for SetOp (including UNION and UNION ALL).

trimFields

public RelFieldTrimmer.TrimResult trimFields(org.apache.calcite.rel.core.Aggregate aggregate,
                                             org.apache.calcite.util.ImmutableBitSet fieldsUsed,
                                             Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields)

Variant of trimFields(RelNode, ImmutableBitSet, Set) for LogicalAggregate.

trimFields

public RelFieldTrimmer.TrimResult trimFields(org.apache.calcite.rel.logical.LogicalTableModify modifier,
                                             org.apache.calcite.util.ImmutableBitSet fieldsUsed,
                                             Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields)

Variant of trimFields(RelNode, ImmutableBitSet, Set) for LogicalTableModify.

trimFields

public RelFieldTrimmer.TrimResult trimFields(org.apache.calcite.rel.logical.LogicalTableFunctionScan tabFun,
                                             org.apache.calcite.util.ImmutableBitSet fieldsUsed,
                                             Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields)

Variant of trimFields(RelNode, ImmutableBitSet, Set) for LogicalTableFunctionScan.

trimFields

public RelFieldTrimmer.TrimResult trimFields(org.apache.calcite.rel.logical.LogicalValues values,
                                             org.apache.calcite.util.ImmutableBitSet fieldsUsed,
                                             Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields)

Variant of trimFields(RelNode, ImmutableBitSet, Set) for LogicalValues.

createMapping

protected org.apache.calcite.util.mapping.Mapping createMapping(org.apache.calcite.util.ImmutableBitSet fieldsUsed,
                                                                int fieldCount)

trimFields

public RelFieldTrimmer.TrimResult trimFields(org.apache.calcite.rel.core.TableScan tableAccessRel,
                                             org.apache.calcite.util.ImmutableBitSet fieldsUsed,
                                             Set<org.apache.calcite.rel.type.RelDataTypeField> extraFields)

Variant of trimFields(RelNode, ImmutableBitSet, Set) for LogicalTableScan.