package expressions
Type Members
-
class
Cast extends ExpressionWithToString
Represents a cast expression in the public logical expression API.
Represents a cast expression in the public logical expression API.
- Annotations
- @Evolving()
- Since
3.3.0
-
trait
Expression extends AnyRef
Base class of the public logical expression API.
Base class of the public logical expression API.
- Annotations
- @Evolving()
- Since
3.0.0
-
class
Expressions extends AnyRef
Helper methods to create logical transforms to pass into Spark.
Helper methods to create logical transforms to pass into Spark.
- Annotations
- @Evolving()
- Since
3.0.0
-
class
Extract extends ExpressionWithToString
Represent an extract function, which extracts and returns the value of a specified datetime field from a datetime or interval value expression.
Represent an extract function, which extracts and returns the value of a specified datetime field from a datetime or interval value expression.
The currently supported fields names following the ISO standard:
-
SECONDSince 3.4.0 -
MINUTESince 3.4.0 -
HOURSince 3.4.0 -
MONTHSince 3.4.0 -
QUARTERSince 3.4.0 -
YEARSince 3.4.0 -
DAY_OF_WEEKSince 3.4.0 -
DAYSince 3.4.0 -
DAY_OF_YEARSince 3.4.0 -
WEEKSince 3.4.0 -
YEAR_OF_WEEKSince 3.4.0
- Annotations
- @Evolving()
- Since
3.4.0
-
-
class
GeneralScalarExpression extends ExpressionWithToString
The general representation of SQL scalar expressions, which contains the upper-cased expression name and all the children expressions.
The general representation of SQL scalar expressions, which contains the upper-cased expression name and all the children expressions. Please also see
Predicatefor the supported predicate expressions.The currently supported SQL scalar expressions:
- Name:
+ - SQL semantic:
expr1 + expr2 - Since version: 3.3.0
- Name:
- - SQL semantic:
expr1 - expr2or- expr - Since version: 3.3.0
- Name:
* - SQL semantic:
expr1 * expr2 - Since version: 3.3.0
- Name:
/ - SQL semantic:
expr1 / expr2 - Since version: 3.3.0
- Name:
% - SQL semantic:
expr1 % expr2 - Since version: 3.3.0
- Name:
& - SQL semantic:
expr1 & expr2 - Since version: 3.3.0
- Name:
| - SQL semantic:
expr1 | expr2 - Since version: 3.3.0
- Name:
^ - SQL semantic:
expr1 ^ expr2 - Since version: 3.3.0
- Name:
~ - SQL semantic:
~ expr - Since version: 3.3.0
- Name:
CASE_WHEN - SQL semantic:
CASE WHEN expr1 THEN expr2 [WHEN expr3 THEN expr4]* [ELSE expr5] END - Since version: 3.3.0
- Name:
ABS - SQL semantic:
ABS(expr) - Since version: 3.3.0
- Name:
COALESCE - SQL semantic:
COALESCE(expr1, expr2) - Since version: 3.3.0
- Name:
GREATEST - SQL semantic:
GREATEST(expr, ...) - Since version: 3.4.0
- Name:
LEAST - SQL semantic:
LEAST(expr, ...) - Since version: 3.4.0
- Name:
RAND - SQL semantic:
RAND([seed]) - Since version: 3.4.0
- Name:
LOG - SQL semantic:
LOG(base, expr) - Since version: 3.4.0
- Name:
LOG10 - SQL semantic:
LOG10(expr) - Since version: 3.4.0
- Name:
LOG2 - SQL semantic:
LOG2(expr) - Since version: 3.4.0
- Name:
LN - SQL semantic:
LN(expr) - Since version: 3.3.0
- Name:
EXP - SQL semantic:
EXP(expr) - Since version: 3.3.0
- Name:
POWER - SQL semantic:
POWER(expr, number) - Since version: 3.3.0
- Name:
SQRT - SQL semantic:
SQRT(expr) - Since version: 3.3.0
- Name:
FLOOR - SQL semantic:
FLOOR(expr) - Since version: 3.3.0
- Name:
CEIL - SQL semantic:
CEIL(expr) - Since version: 3.3.0
- Name:
ROUND - SQL semantic:
ROUND(expr, [scale]) - Since version: 3.4.0
- Name:
SIN - SQL semantic:
SIN(expr) - Since version: 3.4.0
- Name:
SINH - SQL semantic:
SINH(expr) - Since version: 3.4.0
- Name:
COS - SQL semantic:
COS(expr) - Since version: 3.4.0
- Name:
COSH - SQL semantic:
COSH(expr) - Since version: 3.4.0
- Name:
TAN - SQL semantic:
TAN(expr) - Since version: 3.4.0
- Name:
TANH - SQL semantic:
TANH(expr) - Since version: 3.4.0
- Name:
COT - SQL semantic:
COT(expr) - Since version: 3.4.0
- Name:
ASIN - SQL semantic:
ASIN(expr) - Since version: 3.4.0
- Name:
ASINH - SQL semantic:
ASINH(expr) - Since version: 3.4.0
- Name:
ACOS - SQL semantic:
ACOS(expr) - Since version: 3.4.0
- Name:
ACOSH - SQL semantic:
ACOSH(expr) - Since version: 3.4.0
- Name:
ATAN - SQL semantic:
ATAN(expr) - Since version: 3.4.0
- Name:
ATANH - SQL semantic:
ATANH(expr) - Since version: 3.4.0
- Name:
ATAN2 - SQL semantic:
ATAN2(exprY, exprX) - Since version: 3.4.0
- Name:
CBRT - SQL semantic:
CBRT(expr) - Since version: 3.4.0
- Name:
DEGREES - SQL semantic:
DEGREES(expr) - Since version: 3.4.0
- Name:
RADIANS - SQL semantic:
RADIANS(expr) - Since version: 3.4.0
- Name:
SIGN - SQL semantic:
SIGN(expr) - Since version: 3.4.0
- Name:
WIDTH_BUCKET - SQL semantic:
WIDTH_BUCKET(expr) - Since version: 3.3.0
- Name:
SUBSTRING - SQL semantic:
SUBSTRING(str, pos[, len]) - Since version: 3.4.0
- Name:
UPPER - SQL semantic:
UPPER(expr) - Since version: 3.4.0
- Name:
LOWER - SQL semantic:
LOWER(expr) - Since version: 3.4.0
- Name:
TRANSLATE - SQL semantic:
TRANSLATE(input, from, to) - Since version: 3.4.0
- Name:
TRIM - SQL semantic:
TRIM(src, trim) - Since version: 3.4.0
- Name:
LTRIM - SQL semantic:
LTRIM(src, trim) - Since version: 3.4.0
- Name:
RTRIM - SQL semantic:
RTRIM(src, trim) - Since version: 3.4.0
- Name:
BIT_LENGTH - SQL semantic:
BIT_LENGTH(src) - Since version: 3.4.0
- Name:
CHAR_LENGTH - SQL semantic:
CHAR_LENGTH(src) - Since version: 3.4.0
- Name:
CONCAT - SQL semantic:
CONCAT(col1, col2, ..., colN) - Since version: 3.4.0
- Name:
OVERLAY - SQL semantic:
OVERLAY(string, replace, position[, length]) - Since version: 3.4.0
- Name:
DATE_ADD - SQL semantic:
DATE_ADD(start_date, num_days) - Since version: 3.4.0
- Name:
DATE_DIFF - SQL semantic:
DATE_DIFF(end_date, start_date) - Since version: 3.4.0
- Name:
TRUNC - SQL semantic:
TRUNC(date, format) - Since version: 3.4.0
- Name:
AES_ENCRYPT - SQL semantic:
AES_ENCRYPT(expr, key[, mode[, padding]]) - Since version: 3.4.0
- Name:
AES_DECRYPT - SQL semantic:
AES_DECRYPT(expr, key[, mode[, padding]]) - Since version: 3.4.0
- Name:
SHA1 - SQL semantic:
SHA1(expr) - Since version: 3.4.0
- Name:
SHA2 - SQL semantic:
SHA2(expr, bitLength) - Since version: 3.4.0
- Name:
MD5 - SQL semantic:
MD5(expr) - Since version: 3.4.0
- Name:
CRC32 - SQL semantic:
CRC32(expr) - Since version: 3.4.0
Note: SQL semantic conforms ANSI standard, so some expressions are not supported when ANSI off, including: add, subtract, multiply, divide, remainder, pmod.
- Annotations
- @Evolving()
- Since
3.3.0
- Name:
-
trait
Literal[T] extends Expression
Represents a constant literal value in the public expression API.
Represents a constant literal value in the public expression API.
The JVM type of the value held by a literal must be the type used by Spark's InternalRow API for the literal's
SQL data type.- Annotations
- @Evolving()
- Since
3.0.0
-
trait
NamedReference extends Expression
Represents a field or column reference in the public logical expression API.
Represents a field or column reference in the public logical expression API.
- Annotations
- @Evolving()
- Since
3.0.0
-
sealed abstract final
class
NullOrdering extends Enum[NullOrdering]
A null order used in sorting expressions.
A null order used in sorting expressions.
- Annotations
- @Experimental()
- Since
3.2.0
-
sealed abstract final
class
SortDirection extends Enum[SortDirection]
A sort direction used in sorting expressions.
A sort direction used in sorting expressions.
Each direction has a default null ordering that is implied if no null ordering is specified explicitly.
- Annotations
- @Experimental()
- Since
3.2.0
-
trait
SortOrder extends Expression
Represents a sort order in the public expression API.
Represents a sort order in the public expression API.
- Annotations
- @Experimental()
- Since
3.2.0
-
trait
Transform extends Expression
Represents a transform function in the public logical expression API.
Represents a transform function in the public logical expression API.
For example, the transform date(ts) is used to derive a date value from a timestamp column. The transform name is "date" and its argument is a reference to the "ts" column.
- Annotations
- @Evolving()
- Since
3.0.0
-
class
UserDefinedScalarFunc extends ExpressionWithToString
The general representation of user defined scalar function, which contains the upper-cased function name, canonical function name and all the children expressions.
The general representation of user defined scalar function, which contains the upper-cased function name, canonical function name and all the children expressions.
- Annotations
- @Evolving()
- Since
3.4.0