Type Members

1.  case class AddColumns(table: LogicalPlan, columnsToAdd: Seq[QualifiedColType]) extends LogicalPlan with AlterTableCommand with Product with Serializable

   The logical plan of the ALTER TABLE ...

   The logical plan of the ALTER TABLE ... ADD COLUMNS command.

2.  case class AddPartitions(table: LogicalPlan, parts: Seq[PartitionSpec], ifNotExists: Boolean) extends LogicalPlan with V2PartitionCommand with Product with Serializable

   The logical plan of the ALTER TABLE ADD PARTITION command.

   The logical plan of the ALTER TABLE ADD PARTITION command.

   The syntax of this command is:

   ALTER TABLE table ADD [IF NOT EXISTS]
               PARTITION spec1 [LOCATION 'loc1'][, PARTITION spec2 [LOCATION 'loc2'], ...];

3.  case class Aggregate(groupingExpressions: Seq[Expression], aggregateExpressions: Seq[NamedExpression], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

   This is a Group by operator with the aggregate functions and projections.

   This is a Group by operator with the aggregate functions and projections.

   groupingExpressions

   expressions for grouping keys

   aggregateExpressions

   expressions for a project list, which could contain AggregateExpressions. Note: Currently, aggregateExpressions is the project list of this Group by operator. Before separating projection from grouping and aggregate, we should avoid expression-level optimization on aggregateExpressions, which could reference an expression in groupingExpressions. For example, see the rule org.apache.spark.sql.catalyst.optimizer.SimplifyExtractValueOps

4.  case class AlterColumn(table: LogicalPlan, column: FieldName, dataType: Option[DataType], nullable: Option[Boolean], comment: Option[String], position: Option[FieldPosition]) extends LogicalPlan with AlterTableCommand with Product with Serializable

   The logical plan of the ALTER TABLE ...

   The logical plan of the ALTER TABLE ... ALTER COLUMN command.

5.  trait AlterTableCommand extends LogicalPlan with UnaryCommand

   The base trait for commands that need to alter a v2 table with TableChanges.

6.  case class AlterViewAs(child: LogicalPlan, originalText: String, query: LogicalPlan) extends LogicalPlan with BinaryCommand with Product with Serializable

   The logical plan of the ALTER VIEW ...

   The logical plan of the ALTER VIEW ... AS command.

7.  trait AnalysisHelper extends QueryPlan[LogicalPlan]

   AnalysisHelper defines some infrastructure for the query analyzer.

   AnalysisHelper defines some infrastructure for the query analyzer. In particular, in query analysis we don't want to repeatedly re-analyze sub-plans that have previously been analyzed.

   This trait defines a flag analyzed that can be set to true once analysis is done on the tree. This also provides a set of resolve methods that do not recurse down to sub-plans that have the analyzed flag set to true.

   The analyzer rules should use the various resolve methods, in lieu of the various transform methods defined in org.apache.spark.sql.catalyst.trees.TreeNode and QueryPlan.

   To prevent accidental use of the transform methods, this trait also overrides the transform methods to throw exceptions in test mode, if they are used in the analyzer.

8.  trait AnalysisOnlyCommand extends LogicalPlan with Command

   A logical node that can be used for a command that requires its children to be only analyzed, but not optimized.

9.  case class AnalyzeColumn(child: LogicalPlan, columnNames: Option[Seq[String]], allColumns: Boolean) extends LogicalPlan with UnaryCommand with Product with Serializable

   The logical plan of the ANALYZE TABLE FOR COLUMNS command.

10.  case class AnalyzeTable(child: LogicalPlan, partitionSpec: Map[String, Option[String]], noScan: Boolean) extends LogicalPlan with UnaryCommand with Product with Serializable

    The logical plan of the ANALYZE TABLE command.

11.  case class AnalyzeTables(namespace: LogicalPlan, noScan: Boolean) extends LogicalPlan with UnaryCommand with Product with Serializable

    The logical plan of the ANALYZE TABLES command.

12.  case class AppendColumns(func: (Any) => Any, argumentClass: Class[_], argumentSchema: StructType, deserializer: Expression, serializer: Seq[NamedExpression], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

    A relation produced by applying func to each element of the child, concatenating the resulting columns at the end of the input row.

    A relation produced by applying func to each element of the child, concatenating the resulting columns at the end of the input row.

    deserializer

    used to extract the input to func from an input row.

    serializer

    use to serialize the output of func.

13.  case class AppendColumnsWithObject(func: (Any) => Any, childSerializer: Seq[NamedExpression], newColumnsSerializer: Seq[NamedExpression], child: LogicalPlan) extends LogicalPlan with ObjectConsumer with Product with Serializable

    An optimized version of AppendColumns, that can be executed on deserialized object directly.

14.  case class AppendData(table: NamedRelation, query: LogicalPlan, writeOptions: Map[String, String], isByName: Boolean, write: Option[Write] = None) extends LogicalPlan with V2WriteCommand with Product with Serializable

    Append data to an existing table.

15.  case class ArrowEvalPython(udfs: Seq[PythonUDF], resultAttrs: Seq[Attribute], child: LogicalPlan, evalType: Int) extends LogicalPlan with BaseEvalPython with Product with Serializable

    A logical plan that evaluates a PythonUDF with Apache Arrow.

16.  case class Assignment(key: Expression, value: Expression) extends Expression with Unevaluable with BinaryLike[Expression] with Product with Serializable
17.  case class AttachDistributedSequence(sequenceAttr: Attribute, child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

    A logical plan that adds a new long column with the name name that increases one by one.

    A logical plan that adds a new long column with the name name that increases one by one. This is for 'distributed-sequence' default index in pandas API on Spark.

18.  trait BaseEvalPython extends LogicalPlan with UnaryNode
19.  case class BatchEvalPython(udfs: Seq[PythonUDF], resultAttrs: Seq[Attribute], child: LogicalPlan) extends LogicalPlan with BaseEvalPython with Product with Serializable

    A logical plan that evaluates a PythonUDF

20.  trait BinaryCommand extends LogicalPlan with Command with BinaryLike[LogicalPlan]
21.  trait BinaryNode extends LogicalPlan with BinaryLike[LogicalPlan]

    A logical plan node with a left and right child.

22.  case class CTERelationDef(child: LogicalPlan, id: Long = CTERelationDef.newId) extends LogicalPlan with UnaryNode with Product with Serializable

    A wrapper for CTE definition plan with a unique ID.

    A wrapper for CTE definition plan with a unique ID.

    child

    The CTE definition query plan.

    id

    The unique ID for this CTE definition.

23.  case class CTERelationRef(cteId: Long, _resolved: Boolean, output: Seq[Attribute], statsOpt: Option[Statistics] = None) extends LogicalPlan with LeafNode with MultiInstanceRelation with Product with Serializable

    Represents the relation of a CTE reference.

    Represents the relation of a CTE reference.

    cteId

    The ID of the corresponding CTE definition.

    _resolved

    Whether this reference is resolved.

    output

    The output attributes of this CTE reference, which can be different from the output of its corresponding CTE definition after attribute de-duplication.

    statsOpt

    The optional statistics inferred from the corresponding CTE definition.

24.  case class CacheTable(table: LogicalPlan, multipartIdentifier: Seq[String], isLazy: Boolean, options: Map[String, String], isAnalyzed: Boolean = false) extends LogicalPlan with AnalysisOnlyCommand with Product with Serializable

    The logical plan of the CACHE TABLE command.

25.  case class CacheTableAsSelect(tempViewName: String, plan: LogicalPlan, originalText: String, isLazy: Boolean, options: Map[String, String], isAnalyzed: Boolean = false, referredTempFunctions: Seq[String] = Seq.empty) extends LogicalPlan with AnalysisOnlyCommand with Product with Serializable

    The logical plan of the CACHE TABLE ...

    The logical plan of the CACHE TABLE ... AS SELECT command.

26.  case class CoGroup(func: (Any, Iterator[Any], Iterator[Any]) => TraversableOnce[Any], keyDeserializer: Expression, leftDeserializer: Expression, rightDeserializer: Expression, leftGroup: Seq[Attribute], rightGroup: Seq[Attribute], leftAttr: Seq[Attribute], rightAttr: Seq[Attribute], outputObjAttr: Attribute, left: LogicalPlan, right: LogicalPlan) extends LogicalPlan with BinaryNode with ObjectProducer with Product with Serializable

    A relation produced by applying func to each grouping key and associated values from left and right children.

27.  case class CollectMetrics(name: String, metrics: Seq[NamedExpression], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

    Collect arbitrary (named) metrics from a dataset.

    Collect arbitrary (named) metrics from a dataset. As soon as the query reaches a completion point (batch query completes or streaming query epoch completes) an event is emitted on the driver which can be observed by attaching a listener to the spark session. The metrics are named so we can collect metrics at multiple places in a single dataset.

    This node behaves like a global aggregate. All the metrics collected must be aggregate functions or be literals.

28.  case class ColumnStat(distinctCount: Option[BigInt] = None, min: Option[Any] = None, max: Option[Any] = None, nullCount: Option[BigInt] = None, avgLen: Option[Long] = None, maxLen: Option[Long] = None, histogram: Option[Histogram] = None, version: Int = CatalogColumnStat.VERSION) extends Product with Serializable

    Statistics collected for a column.

    Statistics collected for a column.

    1. The JVM data type stored in min/max is the internal data type for the corresponding Catalyst data type. For example, the internal type of DateType is Int, and that the internal type of TimestampType is Long. 2. There is no guarantee that the statistics collected are accurate. Approximation algorithms (sketches) might have been used, and the data collected can also be stale.

    distinctCount

    number of distinct values

    min

    minimum value

    max

    maximum value

    nullCount

    number of nulls

    avgLen

    average length of the values. For fixed-length types, this should be a constant.

    maxLen

    maximum length of the values. For fixed-length types, this should be a constant.

    histogram

    histogram of the values

    version

    version of statistics saved to or retrieved from the catalog

29.  trait Command extends LogicalPlan

    A logical node that represents a non-query command to be executed by the system.

    A logical node that represents a non-query command to be executed by the system. For example, commands can be used by parsers to represent DDL operations. Commands, unlike queries, are eagerly executed.

30.  case class CommentOnNamespace(child: LogicalPlan, comment: String) extends LogicalPlan with UnaryCommand with Product with Serializable

    The logical plan that defines or changes the comment of an NAMESPACE for v2 catalogs.

    The logical plan that defines or changes the comment of an NAMESPACE for v2 catalogs.

    COMMENT ON (DATABASE|SCHEMA|NAMESPACE) namespaceIdentifier IS ('text' | NULL)

    where the text is the new comment written as a string literal; or NULL to drop the comment.

31.  case class CommentOnTable(table: LogicalPlan, comment: String) extends LogicalPlan with AlterTableCommand with Product with Serializable

    The logical plan that defines or changes the comment of an TABLE for v2 catalogs.

    The logical plan that defines or changes the comment of an TABLE for v2 catalogs.

    COMMENT ON TABLE tableIdentifier IS ('text' | NULL)

    where the text is the new comment written as a string literal; or NULL to drop the comment.

32.  trait ConstraintHelper extends AnyRef
33.  case class CreateFunctionStatement(functionName: Seq[String], className: String, resources: Seq[FunctionResource], isTemp: Boolean, ignoreIfExists: Boolean, replace: Boolean) extends ParsedStatement with LeafParsedStatement with Product with Serializable

    CREATE FUNCTION statement, as parsed from SQL

34.  case class CreateNamespace(catalog: SupportsNamespaces, namespace: Seq[String], ifNotExists: Boolean, properties: Map[String, String]) extends LogicalPlan with LeafCommand with Product with Serializable

    The logical plan of the CREATE NAMESPACE command.

35.  case class CreateNamespaceStatement(namespace: Seq[String], ifNotExists: Boolean, properties: Map[String, String]) extends ParsedStatement with LeafParsedStatement with Product with Serializable

    A CREATE NAMESPACE statement, as parsed from SQL.

36.  case class CreateTableAsSelect(catalog: TableCatalog, tableName: Identifier, partitioning: Seq[Transform], query: LogicalPlan, properties: Map[String, String], writeOptions: Map[String, String], ignoreIfExists: Boolean) extends LogicalPlan with UnaryCommand with V2CreateTablePlan with Product with Serializable

    Create a new table from a select query with a v2 catalog.

37.  case class CreateTableAsSelectStatement(tableName: Seq[String], asSelect: LogicalPlan, partitioning: Seq[Transform], bucketSpec: Option[BucketSpec], properties: Map[String, String], provider: Option[String], options: Map[String, String], location: Option[String], comment: Option[String], writeOptions: Map[String, String], serde: Option[SerdeInfo], external: Boolean, ifNotExists: Boolean) extends ParsedStatement with UnaryParsedStatement with Product with Serializable

    A CREATE TABLE AS SELECT command, as parsed from SQL.

38.  case class CreateTableStatement(tableName: Seq[String], tableSchema: StructType, partitioning: Seq[Transform], bucketSpec: Option[BucketSpec], properties: Map[String, String], provider: Option[String], options: Map[String, String], location: Option[String], comment: Option[String], serde: Option[SerdeInfo], external: Boolean, ifNotExists: Boolean) extends ParsedStatement with LeafParsedStatement with Product with Serializable

    A CREATE TABLE command, as parsed from SQL.

    A CREATE TABLE command, as parsed from SQL.

    This is a metadata-only command and is not used to write data to the created table.

39.  case class CreateV2Table(catalog: TableCatalog, tableName: Identifier, tableSchema: StructType, partitioning: Seq[Transform], properties: Map[String, String], ignoreIfExists: Boolean) extends LogicalPlan with LeafCommand with V2CreateTablePlan with Product with Serializable

    Create a new table with a v2 catalog.

40.  case class CreateViewStatement(viewName: Seq[String], userSpecifiedColumns: Seq[(String, Option[String])], comment: Option[String], properties: Map[String, String], originalText: Option[String], child: LogicalPlan, allowExisting: Boolean, replace: Boolean, viewType: ViewType) extends ParsedStatement with UnaryParsedStatement with Product with Serializable

    A CREATE VIEW statement, as parsed from SQL.

41.  case class Deduplicate(keys: Seq[Attribute], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

    A logical plan for dropDuplicates.

42.  case class DeleteAction(condition: Option[Expression]) extends MergeAction with Product with Serializable
43.  case class DeleteFromTable(table: LogicalPlan, condition: Option[Expression]) extends LogicalPlan with UnaryCommand with SupportsSubquery with Product with Serializable

    The logical plan of the DELETE FROM command.

44.  case class DescribeColumn(relation: LogicalPlan, column: Expression, isExtended: Boolean, output: Seq[Attribute] = DescribeColumn.getOutputAttrs) extends LogicalPlan with UnaryCommand with Product with Serializable

    The logical plan of the DESCRIBE relation_name col_name command.

45.  case class DescribeFunction(child: LogicalPlan, isExtended: Boolean) extends LogicalPlan with UnaryCommand with Product with Serializable

    The logical plan of the DESCRIBE FUNCTION command.

46.  case class DescribeNamespace(namespace: LogicalPlan, extended: Boolean, output: Seq[Attribute] = DescribeNamespace.getOutputAttrs) extends LogicalPlan with UnaryCommand with Product with Serializable

    The logical plan of the DESCRIBE NAMESPACE command.

47.  case class DescribeRelation(relation: LogicalPlan, partitionSpec: TablePartitionSpec, isExtended: Boolean, output: Seq[Attribute] = DescribeRelation.getOutputAttrs) extends LogicalPlan with UnaryCommand with Product with Serializable

    The logical plan of the DESCRIBE relation_name command.

48.  case class DeserializeToObject(deserializer: Expression, outputObjAttr: Attribute, child: LogicalPlan) extends LogicalPlan with UnaryNode with ObjectProducer with Product with Serializable

    Takes the input row from child and turns it into object using the given deserializer expression.

49.  case class Distinct(child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

    Returns a new logical plan that dedups input rows.

50.  case class DomainJoin(domainAttrs: Seq[Attribute], child: LogicalPlan, joinType: JoinType = Inner, condition: Option[Expression] = None) extends LogicalPlan with UnaryNode with Product with Serializable

    A placeholder for domain join that can be added when decorrelating subqueries.

    A placeholder for domain join that can be added when decorrelating subqueries. It should be rewritten during the optimization phase.

51.  case class DropColumns(table: LogicalPlan, columnsToDrop: Seq[FieldName]) extends LogicalPlan with AlterTableCommand with Product with Serializable

    The logical plan of the ALTER TABLE ...

    The logical plan of the ALTER TABLE ... DROP COLUMNS command.

52.  case class DropFunction(child: LogicalPlan, ifExists: Boolean, isTemp: Boolean) extends LogicalPlan with UnaryCommand with Product with Serializable

    The logical plan of the DROP FUNCTION command.

53.  case class DropNamespace(namespace: LogicalPlan, ifExists: Boolean, cascade: Boolean) extends LogicalPlan with UnaryCommand with Product with Serializable

    The logical plan of the DROP NAMESPACE command.

54.  case class DropPartitions(table: LogicalPlan, parts: Seq[PartitionSpec], ifExists: Boolean, purge: Boolean) extends LogicalPlan with V2PartitionCommand with Product with Serializable

    The logical plan of the ALTER TABLE DROP PARTITION command.

    The logical plan of the ALTER TABLE DROP PARTITION command. This may remove the data and metadata for this partition.

    If the PURGE option is set, the table catalog must remove partition data by skipping the trash even when the catalog has configured one. The option is applicable only for managed tables.

    The syntax of this command is:

    ALTER TABLE table DROP [IF EXISTS] PARTITION spec1[, PARTITION spec2, ...] [PURGE];

55.  case class DropTable(child: LogicalPlan, ifExists: Boolean, purge: Boolean) extends LogicalPlan with UnaryCommand with Product with Serializable

    The logical plan of the DROP TABLE command.

    The logical plan of the DROP TABLE command.

    If the PURGE option is set, the table catalog must remove table data by skipping the trash even when the catalog has configured one. The option is applicable only for managed tables.

    The syntax of this command is:

    DROP TABLE [IF EXISTS] table [PURGE];

56.  case class DropView(child: LogicalPlan, ifExists: Boolean) extends LogicalPlan with UnaryCommand with Product with Serializable

    The logical plan of the DROP VIEW command.

57.  case class EventTimeWatermark(eventTime: Attribute, delay: CalendarInterval, child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

    Used to mark a user specified column as holding the event time for a row.

58.  case class Except(left: LogicalPlan, right: LogicalPlan, isAll: Boolean) extends SetOperation with Product with Serializable
59.  case class Expand(projections: Seq[Seq[Expression]], output: Seq[Attribute], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

    Apply a number of projections to every input row, hence we will get multiple output rows for an input row.

    Apply a number of projections to every input row, hence we will get multiple output rows for an input row.

    projections

    to apply

    output

    of all projections.

    child

    operator.

60.  case class Filter(condition: Expression, child: LogicalPlan) extends OrderPreservingUnaryNode with PredicateHelper with Product with Serializable
61.  case class FlatMapCoGroupsInPandas(leftGroupingLen: Int, rightGroupingLen: Int, functionExpr: Expression, output: Seq[Attribute], left: LogicalPlan, right: LogicalPlan) extends LogicalPlan with BinaryNode with Product with Serializable

    Flatmap cogroups using a udf: pandas.Dataframe, pandas.Dataframe -> pandas.Dataframe This is used by DataFrame.groupby().cogroup().apply().

62.  case class FlatMapGroupsInPandas(groupingAttributes: Seq[Attribute], functionExpr: Expression, output: Seq[Attribute], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

    FlatMap groups using a udf: pandas.Dataframe -> pandas.DataFrame.

    FlatMap groups using a udf: pandas.Dataframe -> pandas.DataFrame. This is used by DataFrame.groupby().apply().

63.  case class FlatMapGroupsInR(func: Array[Byte], packageNames: Array[Byte], broadcastVars: Array[Broadcast[AnyRef]], inputSchema: StructType, outputSchema: StructType, keyDeserializer: Expression, valueDeserializer: Expression, groupingAttributes: Seq[Attribute], dataAttributes: Seq[Attribute], outputObjAttr: Attribute, child: LogicalPlan) extends LogicalPlan with UnaryNode with ObjectProducer with Product with Serializable
64.  case class FlatMapGroupsInRWithArrow(func: Array[Byte], packageNames: Array[Byte], broadcastVars: Array[Broadcast[AnyRef]], inputSchema: StructType, output: Seq[Attribute], keyDeserializer: Expression, groupingAttributes: Seq[Attribute], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

    Similar with FlatMapGroupsInR but serializes and deserializes input/output in Arrow format.

    Similar with FlatMapGroupsInR but serializes and deserializes input/output in Arrow format. This is also somewhat similar with FlatMapGroupsInPandas.

65.  case class FlatMapGroupsWithState(func: (Any, Iterator[Any], LogicalGroupState[Any]) => Iterator[Any], keyDeserializer: Expression, valueDeserializer: Expression, groupingAttributes: Seq[Attribute], dataAttributes: Seq[Attribute], outputObjAttr: Attribute, stateEncoder: ExpressionEncoder[Any], outputMode: OutputMode, isMapGroupsWithState: Boolean = false, timeout: GroupStateTimeout, hasInitialState: Boolean = false, initialStateGroupAttrs: Seq[Attribute], initialStateDataAttrs: Seq[Attribute], initialStateDeserializer: Expression, initialState: LogicalPlan, child: LogicalPlan) extends LogicalPlan with BinaryNode with ObjectProducer with Product with Serializable

    Applies func to each unique group in child, based on the evaluation of groupingAttributes, while using state data.

    Applies func to each unique group in child, based on the evaluation of groupingAttributes, while using state data. Func is invoked with an object representation of the grouping key an iterator containing the object representation of all the rows with that key.

    func

    function called on each group

    keyDeserializer

    used to extract the key object for each group.

    valueDeserializer

    used to extract the items in the iterator from an input row.

    groupingAttributes

    used to group the data

    dataAttributes

    used to read the data

    outputObjAttr

    used to define the output object

    stateEncoder

    used to serialize/deserialize state before calling func

    outputMode

    the output mode of func

    isMapGroupsWithState

    whether it is created by the mapGroupsWithState method

    timeout

    used to timeout groups that have not received data in a while

    hasInitialState

    Indicates whether initial state needs to be applied or not.

    initialStateGroupAttrs

    grouping attributes for the initial state

    initialStateDataAttrs

    used to read the initial state

    initialStateDeserializer

    used to extract the initial state objects.

    initialState

    user defined initial state that is applied in the first batch.

    child

    logical plan of the underlying data

66.  case class FormatClasses(input: String, output: String) extends Product with Serializable
67.  case class Generate(generator: Generator, unrequiredChildIndex: Seq[Int], outer: Boolean, qualifier: Option[String], generatorOutput: Seq[Attribute], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

    Applies a Generator to a stream of input rows, combining the output of each into a new stream of rows.

    Applies a Generator to a stream of input rows, combining the output of each into a new stream of rows. This operation is similar to a flatMap in functional programming with one important additional feature, which allows the input rows to be joined with their output.

    generator

    the generator expression

    unrequiredChildIndex

    this parameter starts as Nil and gets filled by the Optimizer. It's used as an optimization for omitting data generation that will be discarded next by a projection. A common use case is when we explode(array(..)) and are interested only in the exploded data and not in the original array. before this optimization the array got duplicated for each of its elements, causing O(n^^2) memory consumption. (see [SPARK-21657])

    outer

    when true, each input row will be output at least once, even if the output of the given generator is empty.

    qualifier

    Qualifier for the attributes of generator(UDTF)

    generatorOutput

    The output schema of the Generator.

    child

    Children logical plan node

68.  case class GlobalLimit(limitExpr: Expression, child: LogicalPlan) extends OrderPreservingUnaryNode with Product with Serializable

    A global (coordinated) limit.

    A global (coordinated) limit. This operator can emit at most limitExpr number in total.

    See Limit for more information.

69.  trait HintErrorHandler extends AnyRef

    The callback for implementing customized strategies of handling hint errors.

70.  case class HintInfo(strategy: Option[JoinStrategyHint] = None) extends Product with Serializable

    The hint attributes to be applied on a specific node.

    The hint attributes to be applied on a specific node.

    strategy

    The preferred join strategy.

71.  case class Histogram(height: Double, bins: Array[HistogramBin]) extends Product with Serializable

    This class is an implementation of equi-height histogram.

    This class is an implementation of equi-height histogram. Equi-height histogram represents the distribution of a column's values by a sequence of bins. Each bin has a value range and contains approximately the same number of rows.

    height

    number of rows in each bin

    bins

    equi-height histogram bins

72.  case class HistogramBin(lo: Double, hi: Double, ndv: Long) extends Product with Serializable

    A bin in an equi-height histogram.

    A bin in an equi-height histogram. We use double type for lower/higher bound for simplicity.

    lo

    lower bound of the value range in this bin

    hi

    higher bound of the value range in this bin

    ndv

    approximate number of distinct values in this bin

73.  trait IgnoreCachedData extends LogicalPlan

    A LogicalPlan operator that does not use the cached results stored in CacheManager

74.  case class InsertAction(condition: Option[Expression], assignments: Seq[Assignment]) extends MergeAction with Product with Serializable
75.  case class InsertIntoDir(isLocal: Boolean, storage: CatalogStorageFormat, provider: Option[String], child: LogicalPlan, overwrite: Boolean = true) extends LogicalPlan with UnaryNode with Product with Serializable

    Insert query result into a directory.

    Insert query result into a directory.

    isLocal

    Indicates whether the specified directory is local directory

    storage

    Info about output file, row and what serialization format

    provider

    Specifies what data source to use; only used for data source file.

    child

    The query to be executed

    overwrite

    If true, the existing directory will be overwritten Note that this plan is unresolved and has to be replaced by the concrete implementations during analysis.

76.  case class InsertIntoStatement(table: LogicalPlan, partitionSpec: Map[String, Option[String]], userSpecifiedCols: Seq[String], query: LogicalPlan, overwrite: Boolean, ifPartitionNotExists: Boolean) extends ParsedStatement with UnaryParsedStatement with Product with Serializable

    An INSERT INTO statement, as parsed from SQL.

    An INSERT INTO statement, as parsed from SQL.

    table

    the logical plan representing the table.

    partitionSpec

    a map from the partition key to the partition value (optional). If the value is missing, dynamic partition insert will be performed. As an example, INSERT INTO tbl PARTITION (a=1, b=2) AS would have Map('a' -> Some('1'), 'b' -> Some('2')), and INSERT INTO tbl PARTITION (a=1, b) AS ... would have Map('a' -> Some('1'), 'b' -> None).

    userSpecifiedCols

    the user specified list of columns that belong to the table.

    query

    the logical plan representing data to write to.

    overwrite

    overwrite existing table or partitions.

    ifPartitionNotExists

    If true, only write if the partition does not exist. Only valid for static partitions.

77.  case class InsertStarAction(condition: Option[Expression]) extends MergeAction with Product with Serializable
78.  case class Intersect(left: LogicalPlan, right: LogicalPlan, isAll: Boolean) extends SetOperation with Product with Serializable
79.  case class Join(left: LogicalPlan, right: LogicalPlan, joinType: JoinType, condition: Option[Expression], hint: JoinHint) extends LogicalPlan with BinaryNode with PredicateHelper with Product with Serializable
80.  case class JoinHint(leftHint: Option[HintInfo], rightHint: Option[HintInfo]) extends Product with Serializable

    Hint that is associated with a Join node, with HintInfo on its left child and on its right child respectively.

81.  sealed abstract class JoinStrategyHint extends AnyRef
82.  case class LateralJoin(left: LogicalPlan, right: LateralSubquery, joinType: JoinType, condition: Option[Expression]) extends LogicalPlan with UnaryNode with Product with Serializable

    A logical plan for lateral join.

83.  trait LeafCommand extends LogicalPlan with Command with LeafLike[LogicalPlan]
84.  trait LeafNode extends LogicalPlan with LeafLike[LogicalPlan]

    A logical plan node with no children.

85.  trait LeafParsedStatement extends ParsedStatement with LeafLike[LogicalPlan]
86.  case class LoadData(child: LogicalPlan, path: String, isLocal: Boolean, isOverwrite: Boolean, partition: Option[TablePartitionSpec]) extends LogicalPlan with UnaryCommand with Product with Serializable

    The logical plan of the LOAD DATA INTO TABLE command.

87.  case class LocalLimit(limitExpr: Expression, child: LogicalPlan) extends OrderPreservingUnaryNode with Product with Serializable

    A partition-local (non-coordinated) limit.

    A partition-local (non-coordinated) limit. This operator can emit at most limitExpr number of tuples on each physical partition.

    See Limit for more information.

88.  case class LocalRelation(output: Seq[Attribute], data: Seq[InternalRow] = Nil, isStreaming: Boolean = false) extends LogicalPlan with LeafNode with MultiInstanceRelation with Product with Serializable

    Logical plan node for scanning data from a local collection.

    Logical plan node for scanning data from a local collection.

    data

    The local collection holding the data. It doesn't need to be sent to executors and then doesn't need to be serializable.

89.  trait LogicalGroupState[S] extends AnyRef

    Internal class representing State

90.  abstract class LogicalPlan extends QueryPlan[LogicalPlan] with AnalysisHelper with LogicalPlanStats with QueryPlanConstraints with Logging
91.  trait LogicalPlanVisitor[T] extends AnyRef

    A visitor pattern for traversing a LogicalPlan tree and computing some properties.

92.  case class MapElements(func: AnyRef, argumentClass: Class[_], argumentSchema: StructType, outputObjAttr: Attribute, child: LogicalPlan) extends LogicalPlan with ObjectConsumer with ObjectProducer with Product with Serializable

    A relation produced by applying func to each element of the child.

93.  case class MapGroups(func: (Any, Iterator[Any]) => TraversableOnce[Any], keyDeserializer: Expression, valueDeserializer: Expression, groupingAttributes: Seq[Attribute], dataAttributes: Seq[Attribute], outputObjAttr: Attribute, child: LogicalPlan) extends LogicalPlan with UnaryNode with ObjectProducer with Product with Serializable

    Applies func to each unique group in child, based on the evaluation of groupingAttributes.

    Applies func to each unique group in child, based on the evaluation of groupingAttributes. Func is invoked with an object representation of the grouping key an iterator containing the object representation of all the rows with that key.

    keyDeserializer

    used to extract the key object for each group.

    valueDeserializer

    used to extract the items in the iterator from an input row.

94.  case class MapInPandas(functionExpr: Expression, output: Seq[Attribute], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

    Map partitions using a udf: iter(pandas.Dataframe) -> iter(pandas.DataFrame).

    Map partitions using a udf: iter(pandas.Dataframe) -> iter(pandas.DataFrame). This is used by DataFrame.mapInPandas()

95.  case class MapPartitions(func: (Iterator[Any]) => Iterator[Any], outputObjAttr: Attribute, child: LogicalPlan) extends LogicalPlan with ObjectConsumer with ObjectProducer with Product with Serializable

    A relation produced by applying func to each partition of the child.

96.  case class MapPartitionsInR(func: Array[Byte], packageNames: Array[Byte], broadcastVars: Array[Broadcast[AnyRef]], inputSchema: StructType, outputSchema: StructType, outputObjAttr: Attribute, child: LogicalPlan) extends LogicalPlan with ObjectConsumer with ObjectProducer with Product with Serializable

    A relation produced by applying a serialized R function func to each partition of the child.

97.  case class MapPartitionsInRWithArrow(func: Array[Byte], packageNames: Array[Byte], broadcastVars: Array[Broadcast[AnyRef]], inputSchema: StructType, output: Seq[Attribute], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

    Similar with MapPartitionsInR but serializes and deserializes input/output in Arrow format.

    Similar with MapPartitionsInR but serializes and deserializes input/output in Arrow format.

    This is somewhat similar with org.apache.spark.sql.execution.python.ArrowEvalPython

98.  sealed abstract class MergeAction extends Expression with Unevaluable
99.  case class MergeIntoTable(targetTable: LogicalPlan, sourceTable: LogicalPlan, mergeCondition: Expression, matchedActions: Seq[MergeAction], notMatchedActions: Seq[MergeAction]) extends LogicalPlan with BinaryCommand with SupportsSubquery with Product with Serializable

    The logical plan of the MERGE INTO command.

100.  case class NoopCommand(commandName: String, multipartIdentifier: Seq[String]) extends LogicalPlan with LeafCommand with Product with Serializable

     The logical plan for no-op command handling non-existing table.

101.  trait ObjectConsumer extends LogicalPlan with UnaryNode

     A trait for logical operators that consumes domain objects as input.

     A trait for logical operators that consumes domain objects as input. The output of its child must be a single-field row containing the input object.

102.  trait ObjectProducer extends LogicalPlan

     A trait for logical operators that produces domain objects as output.

     A trait for logical operators that produces domain objects as output. The output of this operator is a single-field safe row containing the produced object.

103.  case class OneRowRelation() extends LogicalPlan with LeafNode with Product with Serializable

     A relation with one row.

     A relation with one row. This is used in "SELECT ..." without a from clause.

104.  abstract class OrderPreservingUnaryNode extends LogicalPlan with UnaryNode
105.  case class OverwriteByExpression(table: NamedRelation, deleteExpr: Expression, query: LogicalPlan, writeOptions: Map[String, String], isByName: Boolean, write: Option[Write] = None) extends LogicalPlan with V2WriteCommand with Product with Serializable

     Overwrite data matching a filter in an existing table.

106.  case class OverwritePartitionsDynamic(table: NamedRelation, query: LogicalPlan, writeOptions: Map[String, String], isByName: Boolean, write: Option[Write] = None) extends LogicalPlan with V2WriteCommand with Product with Serializable

     Dynamically overwrite partitions in an existing table.

107.  abstract class ParsedStatement extends LogicalPlan

     A logical plan node that contains exactly what was parsed from SQL.

     A logical plan node that contains exactly what was parsed from SQL.

     This is used to hold information parsed from SQL when there are multiple implementations of a query or command. For example, CREATE TABLE may be implemented by different nodes for v1 and v2. Instead of parsing directly to a v1 CreateTable that keeps metadata in CatalogTable, and then converting that v1 metadata to the v2 equivalent, the sql CreateTableStatement plan is produced by the parser and converted once into both implementations.

     Parsed logical plans are not resolved because they must be converted to concrete logical plans.

     Parsed logical plans are located in Catalyst so that as much SQL parsing logic as possible is be kept in a org.apache.spark.sql.catalyst.parser.AbstractSqlParser.

108.  case class Pivot(groupByExprsOpt: Option[Seq[NamedExpression]], pivotColumn: Expression, pivotValues: Seq[Expression], aggregates: Seq[Expression], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

     A constructor for creating a pivot, which will later be converted to a Project or an Aggregate during the query analysis.

     A constructor for creating a pivot, which will later be converted to a Project or an Aggregate during the query analysis.

     groupByExprsOpt

     A sequence of group by expressions. This field should be None if coming from SQL, in which group by expressions are not explicitly specified.

     pivotColumn

     The pivot column.

     pivotValues

     A sequence of values for the pivot column.

     aggregates

     The aggregation expressions, each with or without an alias.

     child

     Child operator

109.  case class Project(projectList: Seq[NamedExpression], child: LogicalPlan) extends OrderPreservingUnaryNode with Product with Serializable
110.  case class QualifiedColType(path: Option[FieldName], colName: String, dataType: DataType, nullable: Boolean, comment: Option[String], position: Option[FieldPosition]) extends Product with Serializable

     Column data as parsed by ALTER TABLE ...

     Column data as parsed by ALTER TABLE ... (ADD|REPLACE) COLUMNS.

111.  trait QueryPlanConstraints extends ConstraintHelper
112.  case class Range(start: Long, end: Long, step: Long, numSlices: Option[Int], output: Seq[Attribute] = Range.getOutputAttrs, isStreaming: Boolean = false) extends LogicalPlan with LeafNode with MultiInstanceRelation with Product with Serializable

     Annotations

     @ExpressionDescription()

113.  case class RebalancePartitions(partitionExpressions: Seq[Expression], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

     This operator is used to rebalance the output partitions of the given child, so that every partition is of a reasonable size (not too small and not too big).

     This operator is used to rebalance the output partitions of the given child, so that every partition is of a reasonable size (not too small and not too big). It also try its best to partition the child output by partitionExpressions. If there are skews, Spark will split the skewed partitions, to make these partitions not too big. This operator is useful when you need to write the result of child to a table, to avoid too small/big files.

     Note that, this operator only makes sense when AQE is enabled.

114.  case class RecoverPartitions(child: LogicalPlan) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the ALTER TABLE ...

     The logical plan of the ALTER TABLE ... RECOVER PARTITIONS command.

115.  case class RefreshFunction(child: LogicalPlan) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the REFRESH FUNCTION command.

116.  case class RefreshTable(child: LogicalPlan) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the REFRESH TABLE command.

117.  case class RenameColumn(table: LogicalPlan, column: FieldName, newName: String) extends LogicalPlan with AlterTableCommand with Product with Serializable

     The logical plan of the ALTER TABLE ...

     The logical plan of the ALTER TABLE ... RENAME COLUMN command.

118.  case class RenamePartitions(table: LogicalPlan, from: PartitionSpec, to: PartitionSpec) extends LogicalPlan with V2PartitionCommand with Product with Serializable

     The logical plan of the ALTER TABLE ...

     The logical plan of the ALTER TABLE ... RENAME TO PARTITION command.

119.  case class RenameTable(child: LogicalPlan, newName: Seq[String], isView: Boolean) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the ALTER [TABLE|VIEW] ...

     The logical plan of the ALTER [TABLE|VIEW] ... RENAME TO command.

120.  case class RepairTable(child: LogicalPlan, enableAddPartitions: Boolean, enableDropPartitions: Boolean) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the MSCK REPAIR TABLE command.

121.  case class Repartition(numPartitions: Int, shuffle: Boolean, child: LogicalPlan) extends RepartitionOperation with Product with Serializable

     Returns a new RDD that has exactly numPartitions partitions.

     Returns a new RDD that has exactly numPartitions partitions. Differs from RepartitionByExpression as this method is called directly by DataFrame's, because the user asked for coalesce or repartition. RepartitionByExpression is used when the consumer of the output requires some specific ordering or distribution of the data.

122.  case class RepartitionByExpression(partitionExpressions: Seq[Expression], child: LogicalPlan, optNumPartitions: Option[Int]) extends RepartitionOperation with Product with Serializable

     This method repartitions data using Expressions into optNumPartitions, and receives information about the number of partitions during execution.

     This method repartitions data using Expressions into optNumPartitions, and receives information about the number of partitions during execution. Used when a specific ordering or distribution is expected by the consumer of the query result. Use Repartition for RDD-like coalesce and repartition. If no optNumPartitions is given, by default it partitions data into numShufflePartitions defined in SQLConf, and could be coalesced by AQE.

123.  abstract class RepartitionOperation extends LogicalPlan with UnaryNode

     A base interface for RepartitionByExpression and Repartition

124.  case class ReplaceColumns(table: LogicalPlan, columnsToAdd: Seq[QualifiedColType]) extends LogicalPlan with AlterTableCommand with Product with Serializable

     The logical plan of the ALTER TABLE ...

     The logical plan of the ALTER TABLE ... REPLACE COLUMNS command.

125.  case class ReplaceTable(catalog: TableCatalog, tableName: Identifier, tableSchema: StructType, partitioning: Seq[Transform], properties: Map[String, String], orCreate: Boolean) extends LogicalPlan with LeafCommand with V2CreateTablePlan with Product with Serializable

     Replace a table with a v2 catalog.

     Replace a table with a v2 catalog.

     If the table does not exist, and orCreate is true, then it will be created. If the table does not exist, and orCreate is false, then an exception will be thrown.

     The persisted table will have no contents as a result of this operation.

126.  case class ReplaceTableAsSelect(catalog: TableCatalog, tableName: Identifier, partitioning: Seq[Transform], query: LogicalPlan, properties: Map[String, String], writeOptions: Map[String, String], orCreate: Boolean) extends LogicalPlan with UnaryCommand with V2CreateTablePlan with Product with Serializable

     Replaces a table from a select query with a v2 catalog.

     Replaces a table from a select query with a v2 catalog.

     If the table does not exist, and orCreate is true, then it will be created. If the table does not exist, and orCreate is false, then an exception will be thrown.

127.  case class ReplaceTableAsSelectStatement(tableName: Seq[String], asSelect: LogicalPlan, partitioning: Seq[Transform], bucketSpec: Option[BucketSpec], properties: Map[String, String], provider: Option[String], options: Map[String, String], location: Option[String], comment: Option[String], writeOptions: Map[String, String], serde: Option[SerdeInfo], orCreate: Boolean) extends ParsedStatement with UnaryParsedStatement with Product with Serializable

     A REPLACE TABLE AS SELECT command, as parsed from SQL.

128.  case class ReplaceTableStatement(tableName: Seq[String], tableSchema: StructType, partitioning: Seq[Transform], bucketSpec: Option[BucketSpec], properties: Map[String, String], provider: Option[String], options: Map[String, String], location: Option[String], comment: Option[String], serde: Option[SerdeInfo], orCreate: Boolean) extends ParsedStatement with LeafParsedStatement with Product with Serializable

     A REPLACE TABLE command, as parsed from SQL.

     A REPLACE TABLE command, as parsed from SQL.

     If the table exists prior to running this command, executing this statement will replace the table's metadata and clear the underlying rows from the table.

129.  case class ResolvedHint(child: LogicalPlan, hints: HintInfo = HintInfo()) extends LogicalPlan with UnaryNode with Product with Serializable

     A resolved hint node.

     A resolved hint node. The analyzer should convert all UnresolvedHint into ResolvedHint. This node will be eliminated before optimization starts.

130.  case class ReturnAnswer(child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

     When planning take() or collect() operations, this special node is inserted at the top of the logical plan before invoking the query planner.

     When planning take() or collect() operations, this special node is inserted at the top of the logical plan before invoking the query planner.

     Rules can pattern-match on this node in order to apply transformations that only take effect at the top of the logical query plan.

131.  case class Sample(lowerBound: Double, upperBound: Double, withReplacement: Boolean, seed: Long, child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

     Sample the dataset.

     Sample the dataset.

     lowerBound

     Lower-bound of the sampling probability (usually 0.0)

     upperBound

     Upper-bound of the sampling probability. The expected fraction sampled will be ub - lb.

     withReplacement

     Whether to sample with replacement.

     seed

     the random seed

     child

     the LogicalPlan

132.  case class ScriptInputOutputSchema(inputRowFormat: Seq[(String, String)], outputRowFormat: Seq[(String, String)], inputSerdeClass: Option[String], outputSerdeClass: Option[String], inputSerdeProps: Seq[(String, String)], outputSerdeProps: Seq[(String, String)], recordReaderClass: Option[String], recordWriterClass: Option[String], schemaLess: Boolean) extends Product with Serializable

     Input and output properties when passing data to a script.

     Input and output properties when passing data to a script. For example, in Hive this would specify which SerDes to use.

133.  case class ScriptTransformation(script: String, output: Seq[Attribute], child: LogicalPlan, ioschema: ScriptInputOutputSchema) extends LogicalPlan with UnaryNode with Product with Serializable

     Transforms the input by forking and running the specified script.

     Transforms the input by forking and running the specified script.

     script

     the command that should be executed.

     output

     the attributes that are produced by the script.

     ioschema

     the input and output schema applied in the execution of the script.

134.  case class SerdeInfo(storedAs: Option[String] = None, formatClasses: Option[FormatClasses] = None, serde: Option[String] = None, serdeProperties: Map[String, String] = Map.empty) extends Product with Serializable

     Type to keep track of Hive serde info

135.  case class SerializeFromObject(serializer: Seq[NamedExpression], child: LogicalPlan) extends LogicalPlan with ObjectConsumer with Product with Serializable

     Takes the input object from child and turns it into unsafe row using the given serializer expression.

136.  case class SetCatalogAndNamespace(catalogManager: CatalogManager, catalogName: Option[String], namespace: Option[Seq[String]]) extends LogicalPlan with LeafCommand with Product with Serializable

     The logical plan of the USE/USE NAMESPACE command.

137.  case class SetNamespaceLocation(namespace: LogicalPlan, location: String) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the ALTER (DATABASE|SCHEMA|NAMESPACE) ...

     The logical plan of the ALTER (DATABASE|SCHEMA|NAMESPACE) ... SET LOCATION command.

138.  case class SetNamespaceProperties(namespace: LogicalPlan, properties: Map[String, String]) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the ALTER (DATABASE|SCHEMA|NAMESPACE) ...

     The logical plan of the ALTER (DATABASE|SCHEMA|NAMESPACE) ... SET (DBPROPERTIES|PROPERTIES) command.

139.  abstract class SetOperation extends LogicalPlan with BinaryNode
140.  case class SetTableLocation(table: LogicalPlan, partitionSpec: Option[TablePartitionSpec], location: String) extends LogicalPlan with AlterTableCommand with Product with Serializable

     The logical plan of the ALTER TABLE ...

     The logical plan of the ALTER TABLE ... SET LOCATION command.

141.  case class SetTableProperties(table: LogicalPlan, properties: Map[String, String]) extends LogicalPlan with AlterTableCommand with Product with Serializable

     The logical plan of the ALTER TABLE ...

     The logical plan of the ALTER TABLE ... SET TBLPROPERTIES command.

142.  case class SetTableSerDeProperties(child: LogicalPlan, serdeClassName: Option[String], serdeProperties: Option[Map[String, String]], partitionSpec: Option[TablePartitionSpec]) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the ALTER TABLE ...

     The logical plan of the ALTER TABLE ... SET [SERDE|SERDEPROPERTIES] command.

143.  case class SetViewProperties(child: LogicalPlan, properties: Map[String, String]) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the ALTER VIEW ...

     The logical plan of the ALTER VIEW ... SET TBLPROPERTIES command.

144.  case class ShowColumns(child: LogicalPlan, namespace: Option[Seq[String]], output: Seq[Attribute] = ShowColumns.getOutputAttrs) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the SHOW COLUMN command.

145.  case class ShowCreateTable(child: LogicalPlan, asSerde: Boolean = false, output: Seq[Attribute] = ShowCreateTable.getoutputAttrs) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the SHOW CREATE TABLE command.

146.  case class ShowCurrentNamespace(catalogManager: CatalogManager) extends LogicalPlan with LeafCommand with Product with Serializable

     The logical plan of the SHOW CURRENT NAMESPACE command.

147.  case class ShowCurrentNamespaceStatement() extends ParsedStatement with LeafParsedStatement with Product with Serializable

     A SHOW CURRENT NAMESPACE statement, as parsed from SQL

148.  case class ShowFunctions(child: Option[LogicalPlan], userScope: Boolean, systemScope: Boolean, pattern: Option[String], output: Seq[Attribute] = ShowFunctions.getOutputAttrs) extends LogicalPlan with Command with Product with Serializable

     The logical plan of the SHOW FUNCTIONS command.

149.  case class ShowNamespaces(namespace: LogicalPlan, pattern: Option[String], output: Seq[Attribute] = ShowNamespaces.getOutputAttrs) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the SHOW NAMESPACES command.

150.  case class ShowPartitions(table: LogicalPlan, pattern: Option[PartitionSpec], output: Seq[Attribute] = ShowPartitions.getOutputAttrs) extends LogicalPlan with V2PartitionCommand with Product with Serializable

     The logical plan of the SHOW PARTITIONS command.

151.  case class ShowTableExtended(namespace: LogicalPlan, pattern: String, partitionSpec: Option[PartitionSpec], output: Seq[Attribute] = ShowTableExtended.getOutputAttrs) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the SHOW TABLE EXTENDED command.

152.  case class ShowTableProperties(table: LogicalPlan, propertyKey: Option[String], output: Seq[Attribute] = ShowTableProperties.getOutputAttrs) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the SHOW TBLPROPERTIES command.

153.  case class ShowTables(namespace: LogicalPlan, pattern: Option[String], output: Seq[Attribute] = ShowTables.getOutputAttrs) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the SHOW TABLES command.

154.  case class ShowViews(namespace: LogicalPlan, pattern: Option[String], output: Seq[Attribute] = ShowViews.getOutputAttrs) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the SHOW VIEWS command.

     The logical plan of the SHOW VIEWS command.

     Notes: v2 catalogs do not support views API yet, the command will fallback to v1 ShowViewsCommand during ResolveSessionCatalog.

155.  case class Sort(order: Seq[SortOrder], global: Boolean, child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable
     order

     The ordering expressions

     global

     True means global sorting apply for entire data set, False means sorting only apply within the partition.

     child

     Child logical plan

156.  case class Statistics(sizeInBytes: BigInt, rowCount: Option[BigInt] = None, attributeStats: AttributeMap[ColumnStat] = AttributeMap(Nil), isRuntime: Boolean = false) extends Product with Serializable

     Estimates of various statistics.

     Estimates of various statistics. The default estimation logic simply lazily multiplies the corresponding statistic produced by the children. To override this behavior, override statistics and assign it an overridden version of Statistics.

     NOTE: concrete and/or overridden versions of statistics fields should pay attention to the performance of the implementations. The reason is that estimations might get triggered in performance-critical processes, such as query plan planning.

     Note that we are using a BigInt here since it is easy to overflow a 64-bit integer in cardinality estimation (e.g. cartesian joins).

     sizeInBytes

     Physical size in bytes. For leaf operators this defaults to 1, otherwise it defaults to the product of children's sizeInBytes.

     rowCount

     Estimated number of rows.

     attributeStats

     Statistics for Attributes.

     isRuntime

     Whether the statistics is inferred from query stage runtime statistics during adaptive query execution.

157.  case class Subquery(child: LogicalPlan, correlated: Boolean) extends OrderPreservingUnaryNode with Product with Serializable

     This node is inserted at the top of a subquery when it is optimized.

     This node is inserted at the top of a subquery when it is optimized. This makes sure we can recognize a subquery as such, and it allows us to write subquery aware transformations.

     correlated

     flag that indicates the subquery is correlated, and will be rewritten into a join during analysis.

158.  case class SubqueryAlias(identifier: AliasIdentifier, child: LogicalPlan) extends OrderPreservingUnaryNode with Product with Serializable

     Aliased subquery.

     Aliased subquery.

     identifier

     the alias identifier for this subquery.

     child

     the logical plan of this subquery.

159.  trait SupportsSubquery extends LogicalPlan

     A trait to represent the commands that support subqueries.

     A trait to represent the commands that support subqueries. This is used to allow such commands in the subquery-related checks.

160.  case class Tail(limitExpr: Expression, child: LogicalPlan) extends OrderPreservingUnaryNode with Product with Serializable

     This is similar with Limit except:

     This is similar with Limit except:

     - It does not have plans for global/local separately because currently there is only single implementation which initially mimics both global/local tails. See org.apache.spark.sql.execution.CollectTailExec and org.apache.spark.sql.execution.CollectLimitExec

     - Currently, this plan can only be a root node.

161.  case class TruncatePartition(table: LogicalPlan, partitionSpec: PartitionSpec) extends LogicalPlan with V2PartitionCommand with Product with Serializable

     The logical plan of the TRUNCATE TABLE ...

     The logical plan of the TRUNCATE TABLE ... PARTITION command.

162.  case class TruncateTable(table: LogicalPlan) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the TRUNCATE TABLE command.

163.  case class TypedFilter(func: AnyRef, argumentClass: Class[_], argumentSchema: StructType, deserializer: Expression, child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

     A relation produced by applying func to each element of the child and filter them by the resulting boolean value.

     A relation produced by applying func to each element of the child and filter them by the resulting boolean value.

     This is logically equal to a normal Filter operator whose condition expression is decoding the input row to object and apply the given function with decoded object. However we need the encapsulation of TypedFilter to make the concept more clear and make it easier to write optimizer rules.

164.  trait UnaryCommand extends LogicalPlan with Command with UnaryLike[LogicalPlan]
165.  trait UnaryNode extends LogicalPlan with UnaryLike[LogicalPlan]

     A logical plan node with single child.

166.  trait UnaryParsedStatement extends ParsedStatement with UnaryLike[LogicalPlan]
167.  case class UncacheTable(table: LogicalPlan, ifExists: Boolean, isAnalyzed: Boolean = false) extends LogicalPlan with AnalysisOnlyCommand with Product with Serializable

     The logical plan of the UNCACHE TABLE command.

168.  case class Union(children: Seq[LogicalPlan], byName: Boolean = false, allowMissingCol: Boolean = false) extends LogicalPlan with Product with Serializable

     Logical plan for unioning multiple plans, without a distinct.

     Logical plan for unioning multiple plans, without a distinct. This is UNION ALL in SQL.

     byName

     Whether resolves columns in the children by column names.

     allowMissingCol

     Allows missing columns in children query plans. If it is true, this function allows different set of column names between two Datasets. This can be set to true only if byName is true.

169.  case class UnresolvedHint(name: String, parameters: Seq[Any], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

     A general hint for the child that is not yet resolved.

     A general hint for the child that is not yet resolved. This node is generated by the parser and should be removed This node will be eliminated post analysis.

     name

     the name of the hint

     parameters

     the parameters of the hint

     child

     the LogicalPlan on which this hint applies

170.  case class UnresolvedWith(child: LogicalPlan, cteRelations: Seq[(String, SubqueryAlias)]) extends LogicalPlan with UnaryNode with Product with Serializable

     A container for holding named common table expressions (CTEs) and a query plan.

     A container for holding named common table expressions (CTEs) and a query plan. This operator will be removed during analysis and the relations will be substituted into child.

     child

     The final query of this CTE.

     cteRelations

     A sequence of pair (alias, the CTE definition) that this CTE defined Each CTE can see the base tables and the previously defined CTEs only.

171.  case class UnsetTableProperties(table: LogicalPlan, propertyKeys: Seq[String], ifExists: Boolean) extends LogicalPlan with AlterTableCommand with Product with Serializable

     The logical plan of the ALTER TABLE ...

     The logical plan of the ALTER TABLE ... UNSET TBLPROPERTIES command.

172.  case class UnsetViewProperties(child: LogicalPlan, propertyKeys: Seq[String], ifExists: Boolean) extends LogicalPlan with UnaryCommand with Product with Serializable

     The logical plan of the ALTER VIEW ...

     The logical plan of the ALTER VIEW ... UNSET TBLPROPERTIES command.

173.  case class UpdateAction(condition: Option[Expression], assignments: Seq[Assignment]) extends MergeAction with Product with Serializable
174.  case class UpdateStarAction(condition: Option[Expression]) extends MergeAction with Product with Serializable
175.  case class UpdateTable(table: LogicalPlan, assignments: Seq[Assignment], condition: Option[Expression]) extends LogicalPlan with UnaryCommand with SupportsSubquery with Product with Serializable

     The logical plan of the UPDATE TABLE command.

176.  case class UseStatement(isNamespaceSet: Boolean, nameParts: Seq[String]) extends ParsedStatement with LeafParsedStatement with Product with Serializable

     A USE statement, as parsed from SQL.

177.  trait V2CreateTablePlan extends LogicalPlan

     A trait used for logical plan nodes that create or replace V2 table definitions.

178.  trait V2PartitionCommand extends LogicalPlan with UnaryCommand
179.  trait V2WriteCommand extends LogicalPlan with UnaryCommand

     Base trait for DataSourceV2 write commands

180.  case class View(desc: CatalogTable, isTempView: Boolean, child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable

     A container for holding the view description(CatalogTable) and info whether the view is temporary or not.

     A container for holding the view description(CatalogTable) and info whether the view is temporary or not. If it's a SQL (temp) view, the child should be a logical plan parsed from the CatalogTable.viewText. Otherwise, the view is a temporary one created from a dataframe and the view description should contain a VIEW_CREATED_FROM_DATAFRAME property; in this case, the child must be already resolved.

     This operator will be removed at the end of analysis stage.

     desc

     A view description(CatalogTable) that provides necessary information to resolve the view.

     isTempView

     A flag to indicate whether the view is temporary or not.

     child

     The logical plan of a view operator. If the view description is available, it should be a logical plan parsed from the CatalogTable.viewText.

181.  case class Window(windowExpressions: Seq[NamedExpression], partitionSpec: Seq[Expression], orderSpec: Seq[SortOrder], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable
182.  case class WithCTE(plan: LogicalPlan, cteDefs: Seq[CTERelationDef]) extends LogicalPlan with Product with Serializable

     The resolved version of UnresolvedWith with CTE referrences linked to CTE definitions through unique IDs instead of relation aliases.

     The resolved version of UnresolvedWith with CTE referrences linked to CTE definitions through unique IDs instead of relation aliases.

     plan

     The query plan.

     cteDefs

     The CTE definitions.

183.  case class WithWindowDefinition(windowDefinitions: Map[String, WindowSpecDefinition], child: LogicalPlan) extends LogicalPlan with UnaryNode with Product with Serializable