logical

Type Members

1. case class Aggregate(groupingExpressions: Seq[Expression], aggregateExpressions: Seq[NamedExpression], child: LogicalPlan) extends UnaryNode with Product with Serializable

2. case class AppendColumns(func: (Any) ⇒ Any, argumentClass: Class[_], argumentSchema: StructType, deserializer: Expression, serializer: Seq[NamedExpression], child: LogicalPlan) extends UnaryNode with Product with Serializable

   A relation produced by applying func to each element of the child, concatenating the resulting columns at the end of the input row.

3. case class AppendColumnsWithObject(func: (Any) ⇒ Any, childSerializer: Seq[NamedExpression], newColumnsSerializer: Seq[NamedExpression], child: LogicalPlan) extends UnaryNode with ObjectConsumer with Product with Serializable

   An optimized version of AppendColumns, that can be executed on deserialized object directly.

4. abstract class BinaryNode extends LogicalPlan

   A logical plan node with a left and right child.

5. case class CoGroup(func: (Any, Iterator[Any], Iterator[Any]) ⇒ TraversableOnce[Any], keyDeserializer: Expression, leftDeserializer: Expression, rightDeserializer: Expression, leftGroup: Seq[Attribute], rightGroup: Seq[Attribute], leftAttr: Seq[Attribute], rightAttr: Seq[Attribute], outputObjAttr: Attribute, left: LogicalPlan, right: LogicalPlan) extends BinaryNode with ObjectProducer with Product with Serializable

   A relation produced by applying func to each grouping key and associated values from left and right children.

6. case class ColumnStat(distinctCount: BigInt, min: Option[Any], max: Option[Any], nullCount: BigInt, avgLen: Long, maxLen: Long) extends Product with Serializable

   Statistics collected for a column.

7. trait Command extends LeafNode

   A logical node that represents a non-query command to be executed by the system.

8. case class Deduplicate(keys: Seq[Attribute], child: LogicalPlan, streaming: Boolean) extends UnaryNode with Product with Serializable

   A logical plan for dropDuplicates.

9. case class DeserializeToObject(deserializer: Expression, outputObjAttr: Attribute, child: LogicalPlan) extends UnaryNode with ObjectProducer with Product with Serializable

   Takes the input row from child and turns it into object using the given deserializer expression.

10. case class Distinct(child: LogicalPlan) extends UnaryNode with Product with Serializable

    Returns a new logical plan that dedups input rows.

11. case class EventTimeWatermark(eventTime: Attribute, delay: CalendarInterval, child: LogicalPlan) extends LogicalPlan with Product with Serializable

    Used to mark a user specified column as holding the event time for a row.

12. case class Except(left: LogicalPlan, right: LogicalPlan) extends SetOperation with Product with Serializable

13. case class Expand(projections: Seq[Seq[Expression]], output: Seq[Attribute], child: LogicalPlan) extends UnaryNode with Product with Serializable

    Apply a number of projections to every input row, hence we will get multiple output rows for an input row.

14. case class Filter(condition: Expression, child: LogicalPlan) extends UnaryNode with PredicateHelper with Product with Serializable

15. case class FlatMapGroupsInR(func: Array[Byte], packageNames: Array[Byte], broadcastVars: Array[Broadcast[AnyRef]], inputSchema: StructType, outputSchema: StructType, keyDeserializer: Expression, valueDeserializer: Expression, groupingAttributes: Seq[Attribute], dataAttributes: Seq[Attribute], outputObjAttr: Attribute, child: LogicalPlan) extends UnaryNode with ObjectProducer with Product with Serializable

16. case class FlatMapGroupsWithState(func: (Any, Iterator[Any], LogicalGroupState[Any]) ⇒ Iterator[Any], keyDeserializer: Expression, valueDeserializer: Expression, groupingAttributes: Seq[Attribute], dataAttributes: Seq[Attribute], outputObjAttr: Attribute, stateEncoder: ExpressionEncoder[Any], outputMode: OutputMode, isMapGroupsWithState: Boolean = false, timeout: GroupStateTimeout, child: LogicalPlan) extends UnaryNode with ObjectProducer with Product with Serializable

    Applies func to each unique group in child, based on the evaluation of groupingAttributes, while using state data.

17. case class Generate(generator: Generator, join: Boolean, outer: Boolean, qualifier: Option[String], generatorOutput: Seq[Attribute], child: LogicalPlan) extends UnaryNode with Product with Serializable

    Applies a Generator to a stream of input rows, combining the output of each into a new stream of rows.

18. case class GlobalLimit(limitExpr: Expression, child: LogicalPlan) extends UnaryNode with Product with Serializable

19. case class GroupingSets(selectedGroupByExprs: Seq[Seq[Expression]], groupByExprs: Seq[Expression], child: LogicalPlan, aggregations: Seq[NamedExpression]) extends UnaryNode with Product with Serializable

    A GROUP BY clause with GROUPING SETS can generate a result set equivalent to generated by a UNION ALL of multiple simple GROUP BY clauses.

20. case class HintInfo(isBroadcastable: Option[Boolean] = scala.None) extends Product with Serializable

21. case class InsertIntoTable(table: LogicalPlan, partition: Map[String, Option[String]], query: LogicalPlan, overwrite: Boolean, ifPartitionNotExists: Boolean) extends LogicalPlan with Product with Serializable

    Insert some data into a table.

22. case class Intersect(left: LogicalPlan, right: LogicalPlan) extends SetOperation with Product with Serializable

23. case class Join(left: LogicalPlan, right: LogicalPlan, joinType: JoinType, condition: Option[Expression]) extends BinaryNode with PredicateHelper with Product with Serializable

24. abstract class LeafNode extends LogicalPlan

    A logical plan node with no children.

25. case class LocalLimit(limitExpr: Expression, child: LogicalPlan) extends UnaryNode with Product with Serializable

26. case class LocalRelation(output: Seq[Attribute], data: Seq[InternalRow] = immutable.this.Nil) extends LeafNode with MultiInstanceRelation with Product with Serializable

27. trait LogicalGroupState[S] extends AnyRef

    Internal class representing State

28. abstract class LogicalPlan extends QueryPlan[LogicalPlan] with Logging

29. case class MapElements(func: AnyRef, argumentClass: Class[_], argumentSchema: StructType, outputObjAttr: Attribute, child: LogicalPlan) extends UnaryNode with ObjectConsumer with ObjectProducer with Product with Serializable

    A relation produced by applying func to each element of the child.

30. case class MapGroups(func: (Any, Iterator[Any]) ⇒ TraversableOnce[Any], keyDeserializer: Expression, valueDeserializer: Expression, groupingAttributes: Seq[Attribute], dataAttributes: Seq[Attribute], outputObjAttr: Attribute, child: LogicalPlan) extends UnaryNode with ObjectProducer with Product with Serializable

    Applies func to each unique group in child, based on the evaluation of groupingAttributes.

31. case class MapPartitions(func: (Iterator[Any]) ⇒ Iterator[Any], outputObjAttr: Attribute, child: LogicalPlan) extends UnaryNode with ObjectConsumer with ObjectProducer with Product with Serializable

    A relation produced by applying func to each partition of the child.

32. case class MapPartitionsInR(func: Array[Byte], packageNames: Array[Byte], broadcastVars: Array[Broadcast[AnyRef]], inputSchema: StructType, outputSchema: StructType, outputObjAttr: Attribute, child: LogicalPlan) extends UnaryNode with ObjectConsumer with ObjectProducer with Product with Serializable

    A relation produced by applying a serialized R function func to each partition of the child.

33. trait ObjectConsumer extends UnaryNode

    A trait for logical operators that consumes domain objects as input.

34. trait ObjectProducer extends LogicalPlan

    A trait for logical operators that produces domain objects as output.

35. case class Pivot(groupByExprs: Seq[NamedExpression], pivotColumn: Expression, pivotValues: Seq[Literal], aggregates: Seq[Expression], child: LogicalPlan) extends UnaryNode with Product with Serializable

36. case class Project(projectList: Seq[NamedExpression], child: LogicalPlan) extends UnaryNode with Product with Serializable

37. case class Range(start: Long, end: Long, step: Long, numSlices: Option[Int], output: Seq[Attribute]) extends LeafNode with MultiInstanceRelation with Product with Serializable

38. case class Repartition(numPartitions: Int, shuffle: Boolean, child: LogicalPlan) extends RepartitionOperation with Product with Serializable

    Returns a new RDD that has exactly numPartitions partitions.

39. case class RepartitionByExpression(partitionExpressions: Seq[Expression], child: LogicalPlan, numPartitions: Int) extends RepartitionOperation with Product with Serializable

    This method repartitions data using Expressions into numPartitions, and receives information about the number of partitions during execution.

40. abstract class RepartitionOperation extends UnaryNode

    A base interface for RepartitionByExpression and Repartition

41. case class ResolvedHint(child: LogicalPlan, hints: HintInfo = ...) extends UnaryNode with Product with Serializable

    A resolved hint node.

42. case class ReturnAnswer(child: LogicalPlan) extends UnaryNode with Product with Serializable

    When planning take() or collect() operations, this special node that is inserted at the top of the logical plan before invoking the query planner.

43. case class Sample(lowerBound: Double, upperBound: Double, withReplacement: Boolean, seed: Long, child: LogicalPlan)(isTableSample: Boolean = ...) extends UnaryNode with Product with Serializable

    Sample the dataset.

44. case class ScriptInputOutputSchema(inputRowFormat: Seq[(String, String)], outputRowFormat: Seq[(String, String)], inputSerdeClass: Option[String], outputSerdeClass: Option[String], inputSerdeProps: Seq[(String, String)], outputSerdeProps: Seq[(String, String)], recordReaderClass: Option[String], recordWriterClass: Option[String], schemaLess: Boolean) extends Product with Serializable

    Input and output properties when passing data to a script.

45. case class ScriptTransformation(input: Seq[Expression], script: String, output: Seq[Attribute], child: LogicalPlan, ioschema: ScriptInputOutputSchema) extends UnaryNode with Product with Serializable

    Transforms the input by forking and running the specified script.

46. case class SerializeFromObject(serializer: Seq[NamedExpression], child: LogicalPlan) extends UnaryNode with ObjectConsumer with Product with Serializable

    Takes the input object from child and turns it into unsafe row using the given serializer expression.

47. abstract class SetOperation extends BinaryNode

48. case class Sort(order: Seq[SortOrder], global: Boolean, child: LogicalPlan) extends UnaryNode with Product with Serializable

49. case class Statistics(sizeInBytes: BigInt, rowCount: Option[BigInt] = scala.None, attributeStats: AttributeMap[ColumnStat] = ..., hints: HintInfo = ...) extends Product with Serializable

    Estimates of various statistics.

50. case class Subquery(child: LogicalPlan) extends UnaryNode with Product with Serializable

    This node is inserted at the top of a subquery when it is optimized.

51. case class SubqueryAlias(alias: String, child: LogicalPlan) extends UnaryNode with Product with Serializable

52. case class TypedFilter(func: AnyRef, argumentClass: Class[_], argumentSchema: StructType, deserializer: Expression, child: LogicalPlan) extends UnaryNode with Product with Serializable

    A relation produced by applying func to each element of the child and filter them by the resulting boolean value.

53. abstract class UnaryNode extends LogicalPlan

    A logical plan node with single child.

54. case class Union(children: Seq[LogicalPlan]) extends LogicalPlan with Product with Serializable

55. case class UnresolvedHint(name: String, parameters: Seq[Any], child: LogicalPlan) extends UnaryNode with Product with Serializable

    A general hint for the child that is not yet resolved.

56. case class View(desc: CatalogTable, output: Seq[Attribute], child: LogicalPlan) extends LogicalPlan with MultiInstanceRelation with Product with Serializable

    A container for holding the view description(CatalogTable), and the output of the view.

57. case class Window(windowExpressions: Seq[NamedExpression], partitionSpec: Seq[Expression], orderSpec: Seq[SortOrder], child: LogicalPlan) extends UnaryNode with Product with Serializable

58. case class With(child: LogicalPlan, cteRelations: Seq[(String, SubqueryAlias)]) extends UnaryNode with Product with Serializable

    A container for holding named common table expressions (CTEs) and a query plan.

59. case class WithWindowDefinition(windowDefinitions: Map[String, WindowSpecDefinition], child: LogicalPlan) extends UnaryNode with Product with Serializable