class DataFrame extends AnyRef
- Alphabetic
- By Inheritance
- DataFrame
- AnyRef
- Any
- Hide All
- Show All
- Public
- Protected
Type Members
- class GroupedView extends AnyRef
- class TypedView[T] extends Iterable[T]
Value Members
- final def !=(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- final def ##: Int
- Definition Classes
- AnyRef → Any
- final def ==(arg0: Any): Boolean
- Definition Classes
- AnyRef → Any
- def addColumn(expr: Expr): DataFrame
Add a column to the DataFrame, evaluating to 'expr' at each individual row index.
Add a column to the DataFrame, evaluating to 'expr' at each individual row index. Use the 'as' method on Expr to give the column a name.
- expr
the Expr to evaluate as the new column
- returns
new DataFrame
- def apply(columnNames: String*): DataFrame
Select a subset of columns from this DataFrame.
Select a subset of columns from this DataFrame.
- columnNames
names of columns to select
- returns
new DataFrame
- def apply(range: Range): DataFrame
Retrieve a subset of rows from this DataFrame based on range of indices.
Retrieve a subset of rows from this DataFrame based on range of indices.
- range
range of row indices to retrieve
- returns
new DataFrame
- def apply(index: Int): Seq[Any]
Retrieve a single row by index.
Retrieve a single row by index.
- index
row index
- returns
row as a sequence of values
- def as[T](implicit arg0: scala.reflect.api.JavaUniverse.TypeTag[T]): TypedView[T]
Typecast this DataFrame to a TypedView of the type parameter 'T'.
Typecast this DataFrame to a TypedView of the type parameter 'T'. All columns in this DataFrame will have to be accounted for in the given type. A DataFrame with multiple columns will have its rows represented as tuples of the individual types of these columns.
- T
the type of a row in this DataFrame
- returns
TypedView on the contents of this DataFrame
- final def asInstanceOf[T0]: T0
- Definition Classes
- Any
- def clone(): AnyRef
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.CloneNotSupportedException]) @native() @HotSpotIntrinsicCandidate()
- final def eq(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- def equals(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef → Any
- def explode(expr: Expr): DataFrame
Explode this DataFrame on the given expression, flattening its contents and repeating all other cells on the row for every element in the sequence.
Explode this DataFrame on the given expression, flattening its contents and repeating all other cells on the row for every element in the sequence. The given Expr must evaluate to a list type. Use the 'as' method on Expr to name the flattened column.
- expr
the Expr to explode on
- returns
new DataFrame
- def filter(expr: Expr): DataFrame
Retrieve a subset of rows from this DataFrame based on the boolean evaluation of the given expression.
Retrieve a subset of rows from this DataFrame based on the boolean evaluation of the given expression.
- expr
the Expr to evaluate, if 'true' the given row will appear in the output
- returns
new DataFrame
- final def getClass(): Class[_ <: AnyRef]
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
- def groupBy(keyExprs: Expr*): GroupedView
Partition this DataFrame into groups, defined by the given set of expressions.
Partition this DataFrame into groups, defined by the given set of expressions. The evaluation of each of the 'keyExprs' will appear as a column in the output.
- keyExprs
the list of com.audienceproject.crossbow.expr.Expr that will evaluate to the keys of the groups
- returns
GroupedView on this DataFrame
- def hashCode(): Int
- Definition Classes
- AnyRef → Any
- Annotations
- @native() @HotSpotIntrinsicCandidate()
- def isEmpty: Boolean
- final def isInstanceOf[T0]: Boolean
- Definition Classes
- Any
- def iterator: Iterator[Seq[Any]]
- def join(other: DataFrame, joinExpr: Expr, joinType: JoinType = JoinType.Inner): DataFrame
Join this DataFrame on another DataFrame, with the key evaluated by 'joinExpr'.
Join this DataFrame on another DataFrame, with the key evaluated by 'joinExpr'. The resulting DataFrame will contain all the columns of this DataFrame and the other, where the column names of the other will be prepended with "#".
- other
DataFrame to join with this one
- joinExpr
Expr to evaluate as join key
- joinType
JoinType as one of Inner, FullOuter, LeftOuter or RightOuter
- returns
new DataFrame
- Note
'joinExpr' must evaluate to a type with a natural ordering
- final def ne(arg0: AnyRef): Boolean
- Definition Classes
- AnyRef
- final def notify(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
- final def notifyAll(): Unit
- Definition Classes
- AnyRef
- Annotations
- @native() @HotSpotIntrinsicCandidate()
- val numColumns: Int
- def printSchema(): Unit
- def removeColumns(columnNames: String*): DataFrame
Remove one or more columns from the DataFrame.
Remove one or more columns from the DataFrame.
- columnNames
the names of the columns to remove
- returns
new DataFrame
- def renameColumns(toNewName: (String) => String): DataFrame
Rename the columns of this DataFrame by applying the given function.
Rename the columns of this DataFrame by applying the given function.
- toNewName
function to map over the names of the columns
- returns
new DataFrame
- def renameColumns(newNames: String*): DataFrame
Rename the columns of this DataFrame.
Rename the columns of this DataFrame.
- newNames
list of new names for each column of this DataFrame
- returns
new DataFrame
- val rowCount: Int
- val schema: Schema
- def select(exprs: Expr*): DataFrame
Map over this DataFrame, selecting a set of expressions which will become the columns of a new DataFrame.
Map over this DataFrame, selecting a set of expressions which will become the columns of a new DataFrame. Use the 'as' method on Expr to give names to the new columns. An expression which is only a column accessor will inherit the accessed column's name (unless it is renamed).
- exprs
the list of Expr to evaluate as a new DataFrame
- returns
new DataFrame
- def sortBy(expr: Expr, givenOrderings: Order*): DataFrame
- def sortBy(expr: Expr, givenOrderings: Seq[Order] = Seq.empty, stable: Boolean = false): DataFrame
Sort this DataFrame by the evaluation of 'expr'.
Sort this DataFrame by the evaluation of 'expr'. If a natural ordering exists on this value, it will be used. User-defined orderings on other types or for overwriting the natural orderings with an explicit ordering can be supplied through the 'givenOrderings' argument.
- expr
the Expr to evaluate as a sort key
- givenOrderings
explicit Order to use on the sort key, or list of Order if the key is a tuple
- stable
whether the sorting should be stable or not - quicksort is used if not, else mergesort
- returns
new DataFrame
- final def synchronized[T0](arg0: => T0): T0
- Definition Classes
- AnyRef
- def toString(): String
- Definition Classes
- AnyRef → Any
- def union(other: DataFrame): DataFrame
Union this DataFrame with another DataFrame.
Union this DataFrame with another DataFrame. Columns will be matched by name, and if matched they must have the same type. Columns that are not present in one or the other DataFrame will contain null-values in the output for the rows of the DataFrame in which the column was not present.
- other
DataFrame to union with this one
- returns
new DataFrame
- final def wait(arg0: Long, arg1: Int): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
- final def wait(arg0: Long): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException]) @native()
- final def wait(): Unit
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.InterruptedException])
Deprecated Value Members
- def finalize(): Unit
- Attributes
- protected[lang]
- Definition Classes
- AnyRef
- Annotations
- @throws(classOf[java.lang.Throwable]) @Deprecated
- Deprecated