expressions

package expressions

A set of classes that can be used to represent trees of relational expressions. A key goal of the expression library is to hide the details of naming and scoping from developers who want to manipulate trees of relational operators. As such, the library defines a special type of expression, a NamedExpression in addition to the standard collection of expressions.

Standard Expressions

A library of standard expressions (e.g., Add, EqualTo), aggregates (e.g., SUM, COUNT), and other computations (e.g. UDFs). Each expression type is capable of determining its output schema as a function of its children's output schema.

Named Expressions

Some expression are named and thus can be referenced by later operators in the dataflow graph. The two types of named expressions are AttributeReferences and Aliases. AttributeReferences refer to attributes of the input tuple for a given operator and form the leaves of some expression trees. Aliases assign a name to intermediate computations. For example, in the SQL statement SELECT a+b AS c FROM ..., the expressions a and b would be represented by AttributeReferences and c would be represented by an Alias.

During analysis, all named expressions are assigned a globally unique expression id, which can be used for equality comparisons. While the original names are kept around for debugging purposes, they should never be used to check if two attributes refer to the same value, as plan transformations can result in the introduction of naming ambiguity. For example, consider a plan that contains subqueries, both of which are reading from the same table. If an optimization removes the subqueries, scoping information would be destroyed, eliminating the ability to reason about which subquery produced a given attribute.

Evaluation

The result of expressions can be evaluated using the Expression.apply(Row) method.

Linear Supertypes

AnyRef, Any

Ordering

Alphabetic
By Inheritance

Inherited

expressions
AnyRef
Any

Hide All
Show All

Visibility

Public
All

Type Members

case class Abs(child: Expression, failOnError: Boolean = SQLConf.get.ansiEnabled) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
A function that get the absolute value of the numeric or interval value.
A function that get the absolute value of the numeric or interval value.

Annotations
@ExpressionDescription()
case class Acos(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Acosh(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Add(left: Expression, right: Expression, evalMode: EvalMode.Value = EvalMode.fromSQLConf(SQLConf.get)) extends BinaryArithmetic with CommutativeExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class AddMonths(startDate: Expression, numMonths: Expression) extends AddMonthsBase with Product with Serializable
Returns the date that is num_months after start_date.
Returns the date that is num_months after start_date.

Annotations
@ExpressionDescription()
abstract class AddMonthsBase extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant
case class AesDecrypt(input: Expression, key: Expression, mode: Expression, padding: Expression, aad: Expression) extends Expression with RuntimeReplaceable with ImplicitCastInputTypes with Product with Serializable
A function that decrypts input using AES.
A function that decrypts input using AES. Key lengths of 128, 192 or 256 bits can be used. For versions prior to JDK 8u161, 192 and 256 bits keys can be used if Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files are installed. If either argument is NULL or the key length is not one of the permitted values, the return value is NULL.

Annotations
@ExpressionDescription()
case class AesEncrypt(input: Expression, key: Expression, mode: Expression, padding: Expression, iv: Expression, aad: Expression) extends Expression with RuntimeReplaceable with ImplicitCastInputTypes with Product with Serializable
A function that encrypts input using AES.
A function that encrypts input using AES. Key lengths of 128, 192 or 256 bits can be used. For versions prior to JDK 8u161, 192 and 256 bits keys can be used if Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files are installed. If either argument is NULL or the key length is not one of the permitted values, the return value is NULL.

Annotations
@ExpressionDescription()
abstract class AggregateWindowFunction extends DeclarativeAggregate with WindowFunction
case class Alias(child: Expression, name: String)(exprId: ExprId = NamedExpression.newExprId, qualifier: Seq[String] = Seq.empty, explicitMetadata: Option[Metadata] = None, nonInheritableMetadataKeys: Seq[String] = Seq.empty) extends UnaryExpression with NamedExpression with Product with Serializable
Used to assign a new name to a computation.
Used to assign a new name to a computation. For example the SQL expression "1 + 1 AS a" could be represented as follows: Alias(Add(Literal(1), Literal(1)), "a")()
Note that exprId and qualifiers are in a separate parameter list because we only pattern match on child and name.
Note that when creating a new Alias, all the AttributeReference that refer to the original alias should be updated to the new one.
child
The computation being performed
name
The name to be associated with the result of computing child.
exprId
A globally unique id used to check if an AttributeReference refers to this alias. Auto-assigned if left blank.
qualifier
An optional Seq of string that can be used to refer to this attribute in a fully qualified way. Consider the examples tableName.name, subQueryAlias.name. tableName and subQueryAlias are possible qualifiers.
explicitMetadata
Explicit metadata associated with this alias that overwrites child's.
nonInheritableMetadataKeys
Keys of metadata entries that are supposed to be removed when inheriting the metadata from the child.
trait AliasHelper extends AnyRef
Helper methods for collecting and replacing aliases.
case class And(left: Expression, right: Expression) extends BinaryOperator with Predicate with CommutativeExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class ApplyFunctionExpression(function: ScalarFunction[_], children: Seq[Expression]) extends Expression with UserDefinedExpression with CodegenFallback with ImplicitCastInputTypes with Product with Serializable
case class ArrayAggregate(argument: Expression, zero: Expression, merge: Expression, finish: Expression) extends Expression with HigherOrderFunction with CodegenFallback with QuaternaryLike[Expression] with Product with Serializable
Applies a binary operator to a start value and all elements in the array.
Applies a binary operator to a start value and all elements in the array.

Annotations
@ExpressionDescription()
case class ArrayAppend(left: Expression, right: Expression) extends Expression with ArrayPendBase with Product with Serializable
Given an array, and another element append the element at the end of the array.
Given an array, and another element append the element at the end of the array. This function does not return null when the elements are null. It appends null at the end of the array. But returns null if the array passed is null.

Annotations
@ExpressionDescription()
trait ArrayBasedSimpleHigherOrderFunction extends Expression with SimpleHigherOrderFunction
trait ArrayBinaryLike extends BinaryExpression with BinaryArrayExpressionWithImplicitCast with ArraySetLike with NullIntolerant
Will become common base class for ArrayUnion, ArrayIntersect, and ArrayExcept.
case class ArrayCompact(child: Expression) extends Expression with RuntimeReplaceable with UnaryLike[Expression] with ImplicitCastInputTypes with Product with Serializable

Annotations
@ExpressionDescription()
case class ArrayContains(left: Expression, right: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Predicate with QueryErrorsBase with Product with Serializable
Checks if the array (left) has the element (right)
Checks if the array (left) has the element (right)

Annotations
@ExpressionDescription()
case class ArrayDistinct(child: Expression) extends UnaryExpression with ArraySetLike with ExpectsInputTypes with NullIntolerant with Product with Serializable
Removes duplicate values from the array.
Removes duplicate values from the array.

Annotations
@ExpressionDescription()
case class ArrayExcept(left: Expression, right: Expression) extends BinaryExpression with ArrayBinaryLike with ComplexTypeMergingExpression with Product with Serializable
Returns an array of the elements in the intersect of x and y, without duplicates
Returns an array of the elements in the intersect of x and y, without duplicates

Annotations
@ExpressionDescription()
case class ArrayExists(argument: Expression, function: Expression, followThreeValuedLogic: Boolean) extends Expression with ArrayBasedSimpleHigherOrderFunction with CodegenFallback with Predicate with Product with Serializable
Tests whether a predicate holds for one or more elements in the array.
Tests whether a predicate holds for one or more elements in the array.

Annotations
@ExpressionDescription()
case class ArrayFilter(argument: Expression, function: Expression) extends Expression with ArrayBasedSimpleHigherOrderFunction with CodegenFallback with Product with Serializable
Filters the input array using the given lambda function.
Filters the input array using the given lambda function.

Annotations
@ExpressionDescription()
case class ArrayForAll(argument: Expression, function: Expression) extends Expression with ArrayBasedSimpleHigherOrderFunction with CodegenFallback with Predicate with Product with Serializable
Tests whether a predicate holds for all elements in the array.
Tests whether a predicate holds for all elements in the array.

Annotations
@ExpressionDescription()
case class ArrayInsert(srcArrayExpr: Expression, posExpr: Expression, itemExpr: Expression, legacyNegativeIndex: Boolean) extends TernaryExpression with ImplicitCastInputTypes with ComplexTypeMergingExpression with QueryErrorsBase with SupportQueryContext with Product with Serializable

Annotations
@ExpressionDescription()
case class ArrayIntersect(left: Expression, right: Expression) extends BinaryExpression with ArrayBinaryLike with ComplexTypeMergingExpression with Product with Serializable
Returns an array of the elements in the intersect of x and y, without duplicates
Returns an array of the elements in the intersect of x and y, without duplicates

Annotations
@ExpressionDescription()
case class ArrayJoin(array: Expression, delimiter: Expression, nullReplacement: Option[Expression]) extends Expression with ExpectsInputTypes with Product with Serializable
Creates a String containing all the elements of the input array separated by the delimiter.
Creates a String containing all the elements of the input array separated by the delimiter.

Annotations
@ExpressionDescription()
case class ArrayMax(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Returns the maximum value in the array.
Returns the maximum value in the array.

Annotations
@ExpressionDescription()
case class ArrayMin(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Returns the minimum value in the array.
Returns the minimum value in the array.

Annotations
@ExpressionDescription()
trait ArrayPendBase extends Expression with RuntimeReplaceable with ImplicitCastInputTypes with BinaryLike[Expression] with QueryErrorsBase
case class ArrayPosition(left: Expression, right: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with QueryErrorsBase with Product with Serializable
Returns the position of the first occurrence of element in the given array as long.
Returns the position of the first occurrence of element in the given array as long. Returns 0 if the given value could not be found in the array. Returns null if either of the arguments are null
NOTE: that this is not zero based, but 1-based index. The first element in the array has index 1.

Annotations
@ExpressionDescription()
case class ArrayPrepend(left: Expression, right: Expression) extends Expression with ArrayPendBase with Product with Serializable

Annotations
@ExpressionDescription()
case class ArrayRemove(left: Expression, right: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with QueryErrorsBase with Product with Serializable
Remove all elements that equal to element from the given array
Remove all elements that equal to element from the given array

Annotations
@ExpressionDescription()
case class ArrayRepeat(left: Expression, right: Expression) extends BinaryExpression with ExpectsInputTypes with Product with Serializable
Returns the array containing the given input value (left) count (right) times.
Returns the array containing the given input value (left) count (right) times.

Annotations
@ExpressionDescription()
trait ArraySetLike extends AnyRef
Will become common base class for ArrayDistinct, ArrayUnion, ArrayIntersect, and ArrayExcept.
case class ArraySize(child: Expression) extends Expression with RuntimeReplaceable with ImplicitCastInputTypes with UnaryLike[Expression] with Product with Serializable
Given an array, returns total number of elements in it.
Given an array, returns total number of elements in it.

Annotations
@ExpressionDescription()
case class ArraySort(argument: Expression, function: Expression, allowNullComparisonResult: Boolean) extends Expression with ArrayBasedSimpleHigherOrderFunction with CodegenFallback with Product with Serializable
Sorts elements in an array using a comparator function.
Sorts elements in an array using a comparator function.

Annotations
@ExpressionDescription()
trait ArraySortLike extends Expression with ExpectsInputTypes
Common base class for SortArray and ArraySort.
case class ArrayTransform(argument: Expression, function: Expression) extends Expression with ArrayBasedSimpleHigherOrderFunction with CodegenFallback with Product with Serializable
Transform elements in an array using the transform function.
Transform elements in an array using the transform function. This is similar to a map in functional programming.

Annotations
@ExpressionDescription()
case class ArrayUnion(left: Expression, right: Expression) extends BinaryExpression with ArrayBinaryLike with ComplexTypeMergingExpression with Product with Serializable
Returns an array of the elements in the union of x and y, without duplicates
Returns an array of the elements in the union of x and y, without duplicates

Annotations
@ExpressionDescription()
case class ArraysOverlap(left: Expression, right: Expression) extends BinaryExpression with BinaryArrayExpressionWithImplicitCast with NullIntolerant with Predicate with Product with Serializable
Checks if the two arrays contain at least one common element.
Checks if the two arrays contain at least one common element.

Annotations
@ExpressionDescription()
case class ArraysZip(children: Seq[Expression], names: Seq[Expression]) extends Expression with ExpectsInputTypes with Product with Serializable

Annotations
@ExpressionDescription()
case class Ascii(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Returns the numeric value of the first character of str.
Returns the numeric value of the first character of str.

Annotations
@ExpressionDescription()
case class Asin(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Asinh(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class AssertTrue(left: Expression, right: Expression, replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable
A function that throws an exception if 'condition' is not true.
A function that throws an exception if 'condition' is not true.

Annotations
@ExpressionDescription()
case class AtLeastNNonNulls(n: Int, children: Seq[Expression]) extends Expression with Predicate with Product with Serializable
A predicate that is evaluated to be true if there are at least n non-null and non-NaN values.
case class Atan(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Atan2(left: Expression, right: Expression) extends BinaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Atanh(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
abstract class Attribute extends LeafExpression with NamedExpression with NullIntolerant
class AttributeEquals extends AnyRef

Attributes
protected
class AttributeMap[A] extends Map[Attribute, A] with Serializable
case class AttributeReference(name: String, dataType: DataType, nullable: Boolean = true, metadata: Metadata = Metadata.empty)(exprId: ExprId = NamedExpression.newExprId, qualifier: Seq[String] = Seq.empty[String]) extends Attribute with Unevaluable with Product with Serializable
A reference to an attribute produced by another operator in the tree.
A reference to an attribute produced by another operator in the tree.
name
The name of this attribute, should only be used during analysis or for debugging.
dataType
The DataType of this attribute.
nullable
True if null is a valid value for this attribute.
metadata
The metadata of this attribute.
exprId
A globally unique id used to check if different AttributeReferences refer to the same attribute.
qualifier
An optional string that can be used to referred to this attribute in a fully qualified way. Consider the examples tableName.name, subQueryAlias.name. tableName and subQueryAlias are possible qualifiers.
implicit class AttributeSeq extends Serializable
Helper functions for working with Seq[Attribute].
class AttributeSet extends Iterable[Attribute] with Serializable
A Set designed to hold AttributeReference objects, that performs equality checking using expression id instead of standard java equality.
A Set designed to hold AttributeReference objects, that performs equality checking using expression id instead of standard java equality. Using expression id means that these sets will correctly test for membership, even when the AttributeReferences in question differ cosmetically (e.g., the names have different capitalizations).
Note that we do not override equality for Attribute references as it is really weird when AttributeReference("a"...) == AttributeReference("b", ...). This tactic leads to broken tests, and also makes doing transformations hard (we always try keep older trees instead of new ones when the transformation was a no-op).
case class BRound(child: Expression, scale: Expression, ansiEnabled: Boolean = SQLConf.get.ansiEnabled) extends RoundBase with Product with Serializable
Round an expression to d decimal places using HALF_EVEN rounding mode, also known as Gaussian rounding or bankers' rounding.
Round an expression to d decimal places using HALF_EVEN rounding mode, also known as Gaussian rounding or bankers' rounding. round(2.5) = 2.0, round(3.5) = 4.0.

Annotations
@ExpressionDescription()
case class Base64(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Converts the argument from binary to a base 64 string.
Converts the argument from binary to a base 64 string.

Annotations
@ExpressionDescription()
trait BaseGenericInternalRow extends InternalRow
An extended version of InternalRow that implements all special getters, toString and equals/hashCode by genericGet.
trait BaseGroupingSets extends Expression with CodegenFallback
A placeholder expression for cube/rollup, which will be replaced by analyzer
class BaseOrdering extends Ordering[InternalRow]
A base class for generated/interpreted row ordering.
abstract class BasePredicate extends ExpressionsEvaluator
A base class for generated/interpreted predicate
case class Bin(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable with Product

Annotations
@ExpressionDescription()
abstract class BinaryArithmetic extends BinaryOperator with NullIntolerant with SupportQueryContext
trait BinaryArrayExpressionWithImplicitCast extends BinaryExpression with ImplicitCastInputTypes with QueryErrorsBase
Base trait for BinaryExpressions with two arrays of the same element type and implicit casting.
abstract class BinaryComparison extends BinaryOperator with Predicate
abstract class BinaryExpression extends Expression with BinaryLike[Expression]
An expression with two inputs and one output.
An expression with two inputs and one output. The output is by default evaluated to null if any input is evaluated to null.
abstract class BinaryMathExpression extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable
A binary expression specifically for math functions that take two Doubles as input and returns a Double.
abstract class BinaryOperator extends BinaryExpression with ExpectsInputTypes with QueryErrorsBase
A BinaryExpression that is an operator, with two properties:
A BinaryExpression that is an operator, with two properties:
1. The string representation is "x symbol y", rather than "funcName(x, y)". 2. Two inputs are expected to be of the same type. If the two inputs have different types, the analyzer will find the tightest common type and do the proper type casting.
case class BinaryPad(funcName: String, str: Expression, len: Expression, pad: Expression) extends Expression with RuntimeReplaceable with ImplicitCastInputTypes with Product with Serializable
case class BinaryPredicate(prettyName: String, left: Expression, right: Expression) extends Expression with RuntimeReplaceable with ImplicitCastInputTypes with BinaryLike[Expression] with Product with Serializable
case class BitLength(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
A function that returns the bit length of the given string or binary expression.
A function that returns the bit length of the given string or binary expression.

Annotations
@ExpressionDescription()
case class BitmapBitPosition(child: Expression) extends UnaryExpression with RuntimeReplaceable with ImplicitCastInputTypes with Product with Serializable

Annotations
@ExpressionDescription()
case class BitmapBucketNumber(child: Expression) extends UnaryExpression with RuntimeReplaceable with ImplicitCastInputTypes with Product with Serializable

Annotations
@ExpressionDescription()
case class BitmapConstructAgg(child: Expression, mutableAggBufferOffset: Int = 0, inputAggBufferOffset: Int = 0) extends ImperativeAggregate with ImplicitCastInputTypes with UnaryLike[Expression] with Product with Serializable

Annotations
@ExpressionDescription()
case class BitmapCount(child: Expression) extends UnaryExpression with RuntimeReplaceable with Product with Serializable

Annotations
@ExpressionDescription()
class BitmapExpressionUtils extends AnyRef
A utility class for constructing bitmap expressions.
case class BitmapOrAgg(child: Expression, mutableAggBufferOffset: Int = 0, inputAggBufferOffset: Int = 0) extends ImperativeAggregate with UnaryLike[Expression] with Product with Serializable

Annotations
@ExpressionDescription()
case class BitwiseAnd(left: Expression, right: Expression) extends BinaryArithmetic with CommutativeExpression with Product with Serializable
A function that calculates bitwise and(&) of two numbers.
A function that calculates bitwise and(&) of two numbers.
Code generation inherited from BinaryArithmetic.

Annotations
@ExpressionDescription()
case class BitwiseCount(child: Expression) extends UnaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class BitwiseGet(left: Expression, right: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class BitwiseNot(child: Expression) extends UnaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
A function that calculates bitwise not(~) of a number.
A function that calculates bitwise not(~) of a number.

Annotations
@ExpressionDescription()
case class BitwiseOr(left: Expression, right: Expression) extends BinaryArithmetic with CommutativeExpression with Product with Serializable
A function that calculates bitwise or(|) of two numbers.
A function that calculates bitwise or(|) of two numbers.
Code generation inherited from BinaryArithmetic.

Annotations
@ExpressionDescription()
case class BitwiseXor(left: Expression, right: Expression) extends BinaryArithmetic with CommutativeExpression with Product with Serializable
A function that calculates bitwise xor(^) of two numbers.
A function that calculates bitwise xor(^) of two numbers.
Code generation inherited from BinaryArithmetic.

Annotations
@ExpressionDescription()
case class BloomFilterMightContain(bloomFilterExpression: Expression, valueExpression: Expression) extends BinaryExpression with Predicate with Product with Serializable
An internal scalar function that returns the membership check result (either true or false) for values of valueExpression in the Bloom filter represented by bloomFilterExpression.
An internal scalar function that returns the membership check result (either true or false) for values of valueExpression in the Bloom filter represented by bloomFilterExpression. Not that since the function is "might contain", always returning true regardless is not wrong. Note that this expression requires that bloomFilterExpression is either a constant value or an uncorrelated scalar subquery. This is sufficient for the Bloom filter join rewrite.
bloomFilterExpression
the Binary data of Bloom filter.
valueExpression
the Long value to be tested for the membership of bloomFilterExpression.
case class BoundReference(ordinal: Int, dataType: DataType, nullable: Boolean) extends LeafExpression with Product with Serializable
A bound reference points to a specific slot in the input tuple, allowing the actual value to be retrieved more efficiently.
A bound reference points to a specific slot in the input tuple, allowing the actual value to be retrieved more efficiently. However, since operations like column pruning can change the layout of intermediate tuples, BindReferences should be run after all such transformations.
case class Bucket(numBuckets: Literal, child: Expression) extends PartitionTransformExpression with Product with Serializable
Expression for the v2 partition transform bucket.
case class CallMethodViaReflection(children: Seq[Expression], failOnError: Boolean = true) extends Expression with Nondeterministic with CodegenFallback with QueryErrorsBase with Product with Serializable
An expression that invokes a method on a class via reflection.
An expression that invokes a method on a class via reflection.
For now, only types defined in Reflect.typeMapping are supported (basically primitives and string) as input types, and the output is turned automatically to a string.
Note that unlike Hive's reflect function, this expression calls only static methods (i.e. does not support calling non-static methods).
We should also look into how to consolidate this expression with org.apache.spark.sql.catalyst.expressions.objects.StaticInvoke in the future.
children
the first element should be a literal string for the class name, and the second element should be a literal string for the method name, and the remaining are input arguments to the Java method.

Annotations
@ExpressionDescription()
case class CaseWhen(branches: Seq[(Expression, Expression)], elseValue: Option[Expression] = None) extends Expression with ComplexTypeMergingExpression with ConditionalExpression with Product with Serializable
Case statements of the form "CASE WHEN a THEN b [WHEN c THEN d]* [ELSE e] END".
Case statements of the form "CASE WHEN a THEN b [WHEN c THEN d]* [ELSE e] END". When a = true, returns b; when c = true, returns d; else returns e.
branches
seq of (branch condition, branch value)
elseValue
optional value for the else branch

Annotations
@ExpressionDescription()
case class Cast(child: Expression, dataType: DataType, timeZoneId: Option[String] = None, evalMode: EvalMode.Value = EvalMode.fromSQLConf(SQLConf.get)) extends UnaryExpression with TimeZoneAwareExpression with ToStringBase with NullIntolerant with SupportQueryContext with QueryErrorsBase with Product with Serializable
Cast the child expression to the target data type.
Cast the child expression to the target data type.
When cast from/to timezone related types, we need timeZoneId, which will be resolved with session local timezone by an analyzer ResolveTimeZone.

Annotations
@ExpressionDescription()
case class CastTimestampNTZToLong(child: Expression) extends TimestampToLongBase with Product with Serializable
case class Cbrt(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Ceil(child: Expression) extends UnaryMathExpression with Product with Serializable
trait CeilFloorExpressionBuilderBase extends ExpressionBuilder
case class CheckOverflow(child: Expression, dataType: DecimalType, nullOnOverflow: Boolean) extends UnaryExpression with SupportQueryContext with Product with Serializable
Rounds the decimal to given scale and check whether the decimal can fit in provided precision or not.
Rounds the decimal to given scale and check whether the decimal can fit in provided precision or not. If not, if nullOnOverflow is true, it returns null; otherwise an ArithmeticException is thrown.
case class CheckOverflowInSum(child: Expression, dataType: DecimalType, nullOnOverflow: Boolean, context: SQLQueryContext) extends UnaryExpression with SupportQueryContext with Product with Serializable
case class CheckOverflowInTableInsert(child: Expression, columnName: String) extends UnaryExpression with Product with Serializable
Casting a numeric value as another numeric type in store assignment.
Casting a numeric value as another numeric type in store assignment. It can capture the arithmetic errors and show proper error messages to users.
case class Chr(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Returns the ASCII character having the binary equivalent to n.
Returns the ASCII character having the binary equivalent to n. If n is larger than 256 the result is equivalent to chr(n % 256)

Annotations
@ExpressionDescription()
case class Coalesce(children: Seq[Expression]) extends Expression with ComplexTypeMergingExpression with ConditionalExpression with Product with Serializable
An expression that is evaluated to the first non-null input.
An expression that is evaluated to the first non-null input.
```
coalesce(1, 2) => 1
coalesce(null, 1, 2) => 1
coalesce(null, null, 2) => 2
coalesce(null, null, null) => null
```
Annotations
@ExpressionDescription()
abstract class CodeGeneratorWithInterpretedFallback[IN, OUT] extends Logging
A codegen object generator which creates objects with codegen path first.
A codegen object generator which creates objects with codegen path first. Once any compile error happens, it can fallback to interpreted implementation. In tests, we can use a SQL config SQLConf.CODEGEN_FACTORY_MODE to control fallback behavior.
trait CollectionGenerator extends Expression with Generator
A collection producing Generator.
A collection producing Generator. This trait provides a different path for code generation, by allowing code generation to return either an ArrayData or a MapData object.
trait CommutativeExpression extends Expression
trait ComplexTypeMergingExpression extends Expression
A trait used for resolving nullable flags, including nullable, containsNull of ArrayType and valueContainsNull of MapType, containsNull, valueContainsNull flags of the output date type.
A trait used for resolving nullable flags, including nullable, containsNull of ArrayType and valueContainsNull of MapType, containsNull, valueContainsNull flags of the output date type. This is usually utilized by the expressions (e.g. CaseWhen) that combine data from multiple child expressions of non-primitive types.
case class Concat(children: Seq[Expression]) extends Expression with ComplexTypeMergingExpression with QueryErrorsBase with Product with Serializable
Concatenates multiple input columns together into a single column.
Concatenates multiple input columns together into a single column. The function works with strings, binary and compatible array columns.

Annotations
@ExpressionDescription()
case class ConcatWs(children: Seq[Expression]) extends Expression with ImplicitCastInputTypes with Product with Serializable
An expression that concatenates multiple input strings or array of strings into a single string, using a given separator (the first child).
An expression that concatenates multiple input strings or array of strings into a single string, using a given separator (the first child).
Returns null if the separator is null. Otherwise, concat_ws skips all null values.

Annotations
@ExpressionDescription()
trait ConditionalExpression extends Expression
An expression that contains conditional expression branches, so not all branches will be hit.
An expression that contains conditional expression branches, so not all branches will be hit. All optimization should be careful with the evaluation order.
case class Contains(left: Expression, right: Expression) extends StringPredicate with Product with Serializable
case class Conv(numExpr: Expression, fromBaseExpr: Expression, toBaseExpr: Expression, ansiEnabled: Boolean = SQLConf.get.ansiEnabled) extends TernaryExpression with ImplicitCastInputTypes with NullIntolerant with SupportQueryContext with Product with Serializable
Convert a num from one base to another
Convert a num from one base to another
numExpr
the number to be converted
fromBaseExpr
from which base
toBaseExpr
to which base

Annotations
@ExpressionDescription()
case class ConvertTimezone(sourceTz: Expression, targetTz: Expression, sourceTs: Expression) extends TernaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class Cos(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Cosh(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Cot(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Crc32(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
A function that computes a cyclic redundancy check value and returns it as a bigint For input of type BinaryType
A function that computes a cyclic redundancy check value and returns it as a bigint For input of type BinaryType

Annotations
@ExpressionDescription()
case class CreateArray(children: Seq[Expression], useStringTypeWhenEmpty: Boolean) extends Expression with NoThrow with Product with Serializable
Returns an Array containing the evaluation of all children expressions.
Returns an Array containing the evaluation of all children expressions.

Annotations
@ExpressionDescription()
case class CreateMap(children: Seq[Expression], useStringTypeWhenEmpty: Boolean) extends Expression with NoThrow with Product with Serializable
Returns a catalyst Map containing the evaluation of all children expressions as keys and values.
Returns a catalyst Map containing the evaluation of all children expressions as keys and values. The children are a flatted sequence of kv pairs, e.g. (key1, value1, key2, value2, ...)

Annotations
@ExpressionDescription()
case class CreateNamedStruct(children: Seq[Expression]) extends Expression with NoThrow with Product with Serializable
Creates a struct with the given field names and values
Creates a struct with the given field names and values
children
Seq(name1, val1, name2, val2, ...)

Annotations
@ExpressionDescription()
case class Csc(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class CsvToStructs(schema: StructType, options: Map[String, String], child: Expression, timeZoneId: Option[String] = None, requiredSchema: Option[StructType] = None) extends UnaryExpression with TimeZoneAwareExpression with CodegenFallback with ExpectsInputTypes with NullIntolerant with Product with Serializable
Converts a CSV input string to a StructType with the specified schema.
Converts a CSV input string to a StructType with the specified schema.

Annotations
@ExpressionDescription()
case class Cube(groupingSetIndexes: Seq[Seq[Int]], children: Seq[Expression]) extends Expression with BaseGroupingSets with Product with Serializable
case class CumeDist() extends RowNumberLike with SizeBasedWindowFunction with LeafLike[Expression] with Product with Serializable
The CumeDist function computes the position of a value relative to all values in the partition.
The CumeDist function computes the position of a value relative to all values in the partition. The result is the number of rows preceding or equal to the current row in the ordering of the partition divided by the total number of rows in the window partition. Any tie values in the ordering will evaluate to the same position.
This documentation has been based upon similar documentation for the Hive and Presto projects.

Annotations
@ExpressionDescription()
case class CurrentBatchTimestamp(timestampMs: Long, dataType: DataType, timeZoneId: Option[String] = None) extends LeafExpression with TimeZoneAwareExpression with Nondeterministic with CodegenFallback with Product with Serializable
Expression representing the current batch time, which is used by StreamExecution to 1.
Expression representing the current batch time, which is used by StreamExecution to 1. prevent optimizer from pushing this expression below a stateful operator 2. allow IncrementalExecution to substitute this expression with a Literal(timestamp)
There is no code generation since this expression should be replaced with a literal.
case class CurrentCatalog() extends LeafExpression with Unevaluable with Product with Serializable
Returns the current catalog.
Returns the current catalog.

Annotations
@ExpressionDescription()
case class CurrentDatabase() extends LeafExpression with Unevaluable with Product with Serializable
Returns the current database of the SessionCatalog.
Returns the current database of the SessionCatalog.

Annotations
@ExpressionDescription()
case class CurrentDate(timeZoneId: Option[String] = None) extends LeafExpression with TimeZoneAwareExpression with CodegenFallback with Product with Serializable
Returns the current date at the start of query evaluation.
Returns the current date at the start of query evaluation. There is no code generation since this expression should get constant folded by the optimizer.

Annotations
@ExpressionDescription()
case class CurrentTimeZone() extends LeafExpression with Unevaluable with Product with Serializable

Annotations
@ExpressionDescription()
case class CurrentTimestamp() extends CurrentTimestampLike with Product with Serializable
Returns the current timestamp at the start of query evaluation.
Returns the current timestamp at the start of query evaluation. There is no code generation since this expression should get constant folded by the optimizer.

Annotations
@ExpressionDescription()
abstract class CurrentTimestampLike extends LeafExpression with CodegenFallback
case class CurrentUser() extends LeafExpression with Unevaluable with Product with Serializable

Annotations
@ExpressionDescription()
case class DateAdd(startDate: Expression, days: Expression) extends BinaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
Adds a number of days to startdate.
Adds a number of days to startdate.

Annotations
@ExpressionDescription()
case class DateAddInterval(start: Expression, interval: Expression, timeZoneId: Option[String] = None, ansiEnabled: Boolean = SQLConf.get.ansiEnabled) extends BinaryExpression with ExpectsInputTypes with TimeZoneAwareExpression with NullIntolerant with Product with Serializable
Adds date and an interval.
Adds date and an interval.
When ansi mode is on, the microseconds part of interval needs to be 0, otherwise a runtime IllegalArgumentException will be raised. When ansi mode is off, if the microseconds part of interval is 0, we perform date + interval for better performance. if the microseconds part is not 0, then the date will be converted to a timestamp to add with the whole interval parts.
case class DateAddYMInterval(date: Expression, interval: Expression) extends AddMonthsBase with Product with Serializable
case class DateDiff(endDate: Expression, startDate: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Returns the number of days from startDate to endDate.
Returns the number of days from startDate to endDate.

Annotations
@ExpressionDescription()
case class DateFormatClass(left: Expression, right: Expression, timeZoneId: Option[String] = None) extends BinaryExpression with TimestampFormatterHelper with ImplicitCastInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class DateFromUnixDate(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class DateSub(startDate: Expression, days: Expression) extends BinaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
Subtracts a number of days to startdate.
Subtracts a number of days to startdate.

Annotations
@ExpressionDescription()
case class DatetimeSub(start: Expression, interval: Expression, replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable
Subtract an interval from timestamp or date, which is only used to give a pretty sql string for datetime - interval operations
case class DayOfMonth(child: Expression) extends UnaryExpression with GetDateField with Product with Serializable

Annotations
@ExpressionDescription()
case class DayOfWeek(child: Expression) extends UnaryExpression with GetDateField with Product with Serializable

Annotations
@ExpressionDescription()
case class DayOfYear(child: Expression) extends UnaryExpression with GetDateField with Product with Serializable

Annotations
@ExpressionDescription()
case class Days(child: Expression) extends PartitionTransformExpression with Product with Serializable
Expression for the v2 partition transform days.
case class DecimalAddNoOverflowCheck(left: Expression, right: Expression, dataType: DataType) extends BinaryOperator with Product with Serializable
An add expression for decimal values which is only used internally by Sum/Avg/Window.
An add expression for decimal values which is only used internally by Sum/Avg/Window.
Nota that, this expression does not check overflow which is different with Add. When aggregating values, Spark writes the aggregation buffer values to UnsafeRow via UnsafeRowWriter, which already checks decimal overflow, so we don't need to do it again in the add expression used by Sum/Avg.
case class DecimalDivideWithOverflowCheck(left: Expression, right: Expression, dataType: DecimalType, context: SQLQueryContext, nullOnOverflow: Boolean) extends BinaryExpression with ExpectsInputTypes with SupportQueryContext with Product with Serializable
A divide expression for decimal values which is only used internally by Avg.
A divide expression for decimal values which is only used internally by Avg.
It will fail when nullOnOverflow is false follows:
- left (sum in avg) is null due to over the max precision 38, the right (count in avg) should never be null
- the result of divide is overflow
case class Decode(params: Seq[Expression], replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable

Annotations
@ExpressionDescription()
case class DenseRank(children: Seq[Expression]) extends RankLike with Product with Serializable
The DenseRank function computes the rank of a value in a group of values.
The DenseRank function computes the rank of a value in a group of values. The result is one plus the previously assigned rank value. Unlike Rank, DenseRank will not produce gaps in the ranking sequence.
This documentation has been based upon similar documentation for the Hive and Presto projects.

Annotations
@ExpressionDescription()
case class DistributedSequenceID() extends LeafExpression with Unevaluable with NonSQLExpression with Product with Serializable
Returns increasing 64-bit integers consecutive from 0.
Returns increasing 64-bit integers consecutive from 0. The generated ID is guaranteed to be increasing consecutive started from 0.

Note
this expression is dedicated for Pandas API on Spark to use.
trait DivModLike extends BinaryArithmetic
case class Divide(left: Expression, right: Expression, evalMode: EvalMode.Value = EvalMode.fromSQLConf(SQLConf.get)) extends BinaryArithmetic with DivModLike with Product with Serializable

Annotations
@ExpressionDescription()
case class DivideDTInterval(interval: Expression, num: Expression) extends BinaryExpression with ImplicitCastInputTypes with IntervalDivide with NullIntolerant with Serializable with Product
case class DivideInterval(interval: Expression, num: Expression, failOnError: Boolean = SQLConf.get.ansiEnabled) extends IntervalNumOperation with Product with Serializable
case class DivideYMInterval(interval: Expression, num: Expression) extends BinaryExpression with ImplicitCastInputTypes with IntervalDivide with NullIntolerant with Serializable with Product
case class DropField(name: String) extends Expression with StructFieldsOperation with LeafLike[Expression] with Product with Serializable
Drop a field by name.
trait DynamicPruning extends Expression with Predicate
case class DynamicPruningExpression(child: Expression) extends UnaryExpression with DynamicPruning with Product with Serializable
Marker for a planned DynamicPruning expression.
Marker for a planned DynamicPruning expression. The expression is created during planning, and it defers to its child for evaluation.
child
underlying predicate.
case class DynamicPruningSubquery(pruningKey: Expression, buildQuery: LogicalPlan, buildKeys: Seq[Expression], broadcastKeyIndex: Int, onlyInBroadcast: Boolean, exprId: ExprId = NamedExpression.newExprId, hint: Option[HintInfo] = None) extends SubqueryExpression with DynamicPruning with Unevaluable with UnaryLike[Expression] with Product with Serializable
The DynamicPruningSubquery expression is only used in join operations to prune one side of the join with a filter from the other side of the join.
The DynamicPruningSubquery expression is only used in join operations to prune one side of the join with a filter from the other side of the join. It is inserted in cases where partition pruning can be applied.
pruningKey
the filtering key of the plan to be pruned.
buildQuery
the build side of the join.
buildKeys
the join keys corresponding to the build side of the join
broadcastKeyIndex
the index of the filtering key collected from the broadcast
onlyInBroadcast
when set to false it indicates that the pruning filter is likely to be beneficial and so it should be executed even if it cannot reuse the results of the broadcast through ReuseExchange; otherwise, it will use the filter only if it can reuse the results of the broadcast through ReuseExchange
case class EWM(input: Expression, alpha: Double, ignoreNA: Boolean) extends AggregateWindowFunction with UnaryLike[Expression] with Product with Serializable
Exponential Weighted Moment.
Exponential Weighted Moment. This expression is dedicated only for Pandas API on Spark. An exponentially weighted window is similar to an expanding window but with each prior point being exponentially weighted down relative to the current point. See https://pandas.pydata.org/docs/user_guide/window.html#exponentially-weighted-window for details. Currently, only weighted moving average is supported. In general, it is calculated as y_t = \frac{\sum_{i=0}^{t w_i x_{t-i}}{\sum_{i=0}}t w_i}, where x_t is the input, y_t is the result and the w_i are the weights.
case class ElementAt(left: Expression, right: Expression, defaultValueOutOfBound: Option[Literal] = None, failOnError: Boolean = SQLConf.get.ansiEnabled) extends BinaryExpression with GetMapValueUtil with GetArrayItemUtil with NullIntolerant with SupportQueryContext with QueryErrorsBase with Product with Serializable
Returns the value of index right in Array left or the value for key right in Map left.
Returns the value of index right in Array left or the value for key right in Map left.

Annotations
@ExpressionDescription()
case class Elt(children: Seq[Expression], failOnError: Boolean = SQLConf.get.ansiEnabled) extends Expression with SupportQueryContext with Product with Serializable
An expression that returns the n-th input in given inputs.
An expression that returns the n-th input in given inputs. If all inputs are binary, elt returns an output as binary. Otherwise, it returns as string. If any input is null, elt returns null.

Annotations
@ExpressionDescription()
case class Empty2Null(child: Expression) extends UnaryExpression with String2StringExpression with Product with Serializable
A internal function that converts the empty string to null for partition values.
A internal function that converts the empty string to null for partition values. This function should be only used in V1Writes.
case class Encode(value: Expression, charset: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Encodes the first argument into a BINARY using the provided character set (one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16').
Encodes the first argument into a BINARY using the provided character set (one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16'). If either argument is null, the result will also be null.

Annotations
@ExpressionDescription()
case class EndsWith(left: Expression, right: Expression) extends StringPredicate with Product with Serializable
case class EqualNull(left: Expression, right: Expression, replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable

Annotations
@ExpressionDescription()
case class EqualNullSafe(left: Expression, right: Expression) extends BinaryComparison with Product with Serializable

Annotations
@ExpressionDescription()
case class EqualTo(left: Expression, right: Expression) extends BinaryComparison with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
class EquivalentExpressions extends AnyRef
This class is used to compute equality of (sub)expression trees.
This class is used to compute equality of (sub)expression trees. Expressions can be added to this class and they subsequently query for expression equality. Expression trees are considered equal if for the same input(s), the same result is produced.
case class EulerNumber() extends LeafMathExpression with Product with Serializable
Euler's number.
Euler's number. Note that there is no code generation because this is only evaluated by the optimizer during constant folding.

Annotations
@ExpressionDescription()
trait EvalHelper extends AnyRef
Helper methods for evaluating expressions.
case class Exists(plan: LogicalPlan, outerAttrs: Seq[Expression] = Seq.empty, exprId: ExprId = NamedExpression.newExprId, joinCond: Seq[Expression] = Seq.empty, hint: Option[HintInfo] = None) extends SubqueryExpression with Predicate with Unevaluable with Product with Serializable
The Exists expression checks if a row exists in a subquery given some correlated condition or some uncorrelated condition.
The Exists expression checks if a row exists in a subquery given some correlated condition or some uncorrelated condition.
1. correlated condition:
For example (SQL):
```
SELECT  *
FROM    a
WHERE   EXISTS (SELECT  *
                FROM    b
                WHERE   b.id = a.id)
```
2. uncorrelated condition example:
For example (SQL):
```
SELECT  *
FROM    a
WHERE   EXISTS (SELECT  *
                FROM    b
                WHERE   b.id > 10)
```
case class Exp(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
trait ExpectsInputTypes extends Expression
A trait that gets mixin to define the expected input types of an expression.
A trait that gets mixin to define the expected input types of an expression.
This trait is typically used by operator expressions (e.g. Add, Subtract) to define expected input types without any implicit casting.
Most function expressions (e.g. Substring should extend ImplicitCastInputTypes) instead.
case class Explode(child: Expression) extends ExplodeBase with Product with Serializable
Given an input array produces a sequence of rows for each value in the array.
Given an input array produces a sequence of rows for each value in the array.
```
SELECT explode(array(10,20)) ->
10
20
```
abstract class ExplodeBase extends UnaryExpression with CollectionGenerator with Serializable
A base class for Explode and PosExplode.
trait ExplodeGeneratorBuilderBase extends GeneratorBuilder
case class Expm1(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class ExprId(id: Long, jvmId: UUID) extends Product with Serializable
A globally unique id for a given named expression.
A globally unique id for a given named expression. Used to identify which attribute output by a relation is being referenced in a subsequent computation.
The id field is unique within a given JVM, while the uuid is used to uniquely identify JVMs.
abstract class Expression extends TreeNode[Expression]
An expression in Catalyst.
An expression in Catalyst.
If an expression wants to be exposed in the function registry (so users can call it with "name(arguments...)", the concrete implementation must be a case class whose constructor arguments are all Expressions types. See Substring for an example.
There are a few important traits or abstract classes:
- Nondeterministic: an expression that is not deterministic. - Unevaluable: an expression that is not supposed to be evaluated. - CodegenFallback: an expression that does not have code gen implemented and falls back to interpreted mode. - NullIntolerant: an expression that is null intolerant (i.e. any null input will result in null output). - NonSQLExpression: a common base trait for the expressions that do not have SQL expressions like representation. For example, ScalaUDF, ScalaUDAF, and object MapObjects and Invoke. - UserDefinedExpression: a common base trait for user-defined functions, including UDF/UDAF/UDTF. - HigherOrderFunction: a common base trait for higher order functions that take one or more (lambda) functions and applies these to some objects. The function produces a number of variables which can be consumed by some lambda functions. - NamedExpression: An Expression that is named. - TimeZoneAwareExpression: A common base trait for time zone aware expressions. - SubqueryExpression: A base interface for expressions that contain a org.apache.spark.sql.catalyst.plans.logical.LogicalPlan.
- LeafExpression: an expression that has no child. - UnaryExpression: an expression that has one child. - BinaryExpression: an expression that has two children. - TernaryExpression: an expression that has three children. - QuaternaryExpression: an expression that has four children. - BinaryOperator: a special case of BinaryExpression that requires two children to have the same output data type.
A few important traits used for type coercion rules: - ExpectsInputTypes: an expression that has the expected input types. This trait is typically used by operator expressions (e.g. Add, Subtract) to define expected input types without any implicit casting. - ImplicitCastInputTypes: an expression that has the expected input types, which can be implicitly castable using TypeCoercion.ImplicitTypeCasts. - ComplexTypeMergingExpression: to resolve output types of the complex expressions (e.g., CaseWhen).
class ExpressionDescription extends Annotation with Annotation with ClassfileAnnotation

Annotations
@DeveloperApi() @Retention()
case class ExpressionEquals(e: Expression) extends Product with Serializable
Wrapper around an Expression that provides semantic equality.
class ExpressionImplUtils extends AnyRef
A utility class for constructing expressions.
class ExpressionInfo extends AnyRef
Expression information, will be used to describe a expression.
case class ExpressionProxy(child: Expression, id: Int, runtime: SubExprEvaluationRuntime) extends UnaryExpression with Product with Serializable
A proxy for an catalyst Expression.
A proxy for an catalyst Expression. Given a runtime object SubExprEvaluationRuntime, when this is asked to evaluate, it will load from the evaluation cache in the runtime first.
class ExpressionSet extends Set[Expression] with SetLike[Expression, ExpressionSet]
A Set where membership is determined based on determinacy and a canonical representation of an Expression (i.e.
A Set where membership is determined based on determinacy and a canonical representation of an Expression (i.e. one that attempts to ignore cosmetic differences). See Canonicalize for more details.
Internally this set uses the canonical representation, but keeps also track of the original expressions to ease debugging. Since different expressions can share the same canonical representation, this means that operations that extract expressions from this set are only guaranteed to see at least one such expression. For example:
```
val set = ExpressionSet(a + 1, 1 + a)

set.iterator => Iterator(a + 1)
set.contains(a + 1) => true
set.contains(1 + a) => true
set.contains(a + 2) => false
```
For non-deterministic expressions, they are always considered as not contained in the Set. On adding a non-deterministic expression, simply append it to the original expressions. This is consistent with how we define semanticEquals between two expressions.
The constructor of this class is protected so caller can only initialize an Expression from empty, then build it using add and remove methods. So every instance of this class holds the invariant that: 1. Every expr e in baseSet satisfies e.deterministic && e.canonicalized == e 2. Every deterministic expr e in originals satisfies that e.canonicalized is already accessed.
case class ExpressionStats(expr: Expression)(useCount: Int) extends Product with Serializable
A wrapper in place of using Seq[Expression] to record a group of equivalent expressions.
A wrapper in place of using Seq[Expression] to record a group of equivalent expressions.
This saves a lot of memory when there are a lot of expressions in a same equivalence group. Instead of appending to a mutable list/buffer of Expressions, just update the "flattened" useCount in this wrapper in-place.
trait ExpressionWithRandomSeed extends Expression
Represents the behavior of expressions which have a random seed and can renew the seed.
Represents the behavior of expressions which have a random seed and can renew the seed. Usually the random seed needs to be renewed at each execution under streaming queries.
trait ExpressionsEvaluator extends AnyRef
case class Extract(field: Expression, source: Expression, replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable

Annotations
@ExpressionDescription()
case class ExtractANSIIntervalDays(child: Expression) extends ExtractIntervalPart[Long] with Product with Serializable
case class ExtractANSIIntervalHours(child: Expression) extends ExtractIntervalPart[Long] with Product with Serializable
case class ExtractANSIIntervalMinutes(child: Expression) extends ExtractIntervalPart[Long] with Product with Serializable
case class ExtractANSIIntervalMonths(child: Expression) extends ExtractIntervalPart[Int] with Product with Serializable
case class ExtractANSIIntervalSeconds(child: Expression) extends ExtractIntervalPart[Long] with Product with Serializable
case class ExtractANSIIntervalYears(child: Expression) extends ExtractIntervalPart[Int] with Product with Serializable
case class ExtractIntervalDays(child: Expression) extends ExtractIntervalPart[CalendarInterval] with Product with Serializable
case class ExtractIntervalHours(child: Expression) extends ExtractIntervalPart[CalendarInterval] with Product with Serializable
case class ExtractIntervalMinutes(child: Expression) extends ExtractIntervalPart[CalendarInterval] with Product with Serializable
case class ExtractIntervalMonths(child: Expression) extends ExtractIntervalPart[CalendarInterval] with Product with Serializable
abstract class ExtractIntervalPart[T] extends UnaryExpression with NullIntolerant with Serializable
case class ExtractIntervalSeconds(child: Expression) extends ExtractIntervalPart[CalendarInterval] with Product with Serializable
case class ExtractIntervalYears(child: Expression) extends ExtractIntervalPart[CalendarInterval] with Product with Serializable
trait ExtractValue extends Expression with NullIntolerant
case class Factorial(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class FindInSet(left: Expression, right: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
A function that returns the index (1-based) of the given string (left) in the comma- delimited list (right).
A function that returns the index (1-based) of the given string (left) in the comma- delimited list (right). Returns 0, if the string wasn't found or if the given string (left) contains a comma.

Annotations
@ExpressionDescription()
final class FixedLengthRowBasedKeyValueBatch extends RowBasedKeyValueBatch
An implementation of RowBasedKeyValueBatch in which all key-value records have same length.
An implementation of RowBasedKeyValueBatch in which all key-value records have same length.
The format for each record looks like this: [UnsafeRow for key of length klen] [UnsafeRow for Value of length vlen] [8 bytes pointer to next] Thus, record length = klen + vlen + 8
case class Flatten(child: Expression) extends UnaryExpression with NullIntolerant with QueryErrorsBase with Product with Serializable
Transforms an array of arrays into a single array.
Transforms an array of arrays into a single array.

Annotations
@ExpressionDescription()
case class Floor(child: Expression) extends UnaryMathExpression with Product with Serializable
case class FormatNumber(x: Expression, d: Expression) extends BinaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
Formats the number X to a format like '#,###,###.##', rounded to D decimal places, and returns the result as a string.
Formats the number X to a format like '#,###,###.##', rounded to D decimal places, and returns the result as a string. If D is 0, the result has no decimal point or fractional part.

Annotations
@ExpressionDescription()
case class FormatString(children: Expression*) extends Expression with ImplicitCastInputTypes with Product with Serializable
Returns the input formatted according do printf-style format strings
Returns the input formatted according do printf-style format strings

Annotations
@ExpressionDescription()
sealed abstract class FrameLessOffsetWindowFunction extends Expression with OffsetWindowFunction with Unevaluable with ImplicitCastInputTypes with QueryErrorsBase
A frameless offset window function is a window function that cannot specify window frame and returns the value of the input column offset by a number of rows according to the current row within the partition.
A frameless offset window function is a window function that cannot specify window frame and returns the value of the input column offset by a number of rows according to the current row within the partition. For instance: a FrameLessOffsetWindowFunction for value x with offset -2, will get the value of x 2 rows back from the current row in the partition.
sealed trait FrameType extends AnyRef
The trait used to represent the type of a Window Frame.
case class FromUTCTimestamp(left: Expression, right: Expression) extends BinaryExpression with UTCTimestamp with Product with Serializable
This is a common function for databases supporting TIMESTAMP WITHOUT TIMEZONE.
This is a common function for databases supporting TIMESTAMP WITHOUT TIMEZONE. This function takes a timestamp which is timezone-agnostic, and interprets it as a timestamp in UTC, and renders that timestamp as a timestamp in the given time zone.
However, timestamp in Spark represents number of microseconds from the Unix epoch, which is not timezone-agnostic. So in Spark this function just shift the timestamp value from UTC timezone to the given timezone.
This function may return confusing result if the input is a string with timezone, e.g. '2018-03-13T06:18:23+00:00'. The reason is that, Spark firstly cast the string to timestamp according to the timezone in the string, and finally display the result by converting the timestamp to string according to the session local timezone.

Annotations
@ExpressionDescription()
case class FromUnixTime(sec: Expression, format: Expression, timeZoneId: Option[String] = None) extends BinaryExpression with TimestampFormatterHelper with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Converts the number of seconds from unix epoch (1970-01-01 00:00:00 UTC) to a string representing the timestamp of that moment in the current system time zone in the given format.
Converts the number of seconds from unix epoch (1970-01-01 00:00:00 UTC) to a string representing the timestamp of that moment in the current system time zone in the given format. If the format is missing, using format like "1970-01-01 00:00:00". Note that Hive Language Manual says it returns 0 if fail, but in fact it returns null.

Annotations
@ExpressionDescription()
case class FunctionTableSubqueryArgumentExpression(plan: LogicalPlan, outerAttrs: Seq[Expression] = Seq.empty, exprId: ExprId = NamedExpression.newExprId, partitionByExpressions: Seq[Expression] = Seq.empty, withSinglePartition: Boolean = false, orderByExpressions: Seq[SortOrder] = Seq.empty) extends SubqueryExpression with Unevaluable with Product with Serializable
This is the parsed representation of a relation argument for a TableValuedFunction call.
This is the parsed representation of a relation argument for a TableValuedFunction call. The syntax supports passing such relations one of two ways:
1. SELECT ... FROM tvf_call(TABLE t) 2. SELECT ... FROM tvf_call(TABLE (<query>))
In the former case, the relation argument directly refers to the name of a table in the catalog. In the latter case, the relation argument comprises a table subquery that may itself refer to one or more tables in its own FROM clause.
Each TABLE argument may also optionally include a PARTITION BY clause. If present, these indicate how to logically split up the input relation such that the table-valued function evaluates exactly once for each partition, and returns the union of all results. If no partitioning list is present, this splitting of the input relation is undefined. Furthermore, if the PARTITION BY clause includes a following ORDER BY clause, Catalyst will sort the rows in each partition such that the table-valued function receives them one-by-one in the requested order. Otherwise, if no such ordering is specified, the ordering of rows within each partition is undefined.
plan
the logical plan provided as input for the table argument as either a logical relation or as a more complex logical plan in the event of a table subquery.
outerAttrs
outer references of this subquery plan, generally empty since these table arguments do not allow correlated references currently
exprId
expression ID of this subquery expression, generally generated afresh each time
partitionByExpressions
if non-empty, the TABLE argument included the PARTITION BY clause to indicate that the input relation should be repartitioned by the hash of the provided expressions, such that all the rows with each unique combination of values of the partitioning expressions will be consumed by exactly one instance of the table function class.
withSinglePartition
if true, the TABLE argument included the WITH SINGLE PARTITION clause to indicate that the entire input relation should be repartitioned to one worker for consumption by exactly one instance of the table function class.
orderByExpressions
if non-empty, the TABLE argument included the ORDER BY clause to indicate that the rows within each partition of the table function are to arrive in the provided order.
trait Generator extends Expression
An expression that produces zero or more rows given a single input row.
An expression that produces zero or more rows given a single input row.
Generators produce multiple output rows instead of a single value like other expressions, and thus they must have a schema to associate with the rows that are output.
However, unlike row producing relational operators, which are either leaves or determine their output schema functionally from their input, generators can contain other expressions that might result in their modification by rules. This structure means that they might be copied multiple times after first determining their output schema. If a new output schema is created for each copy references up the tree might be rendered invalid. As a result generators must instead define a function makeOutput which is called only once when the schema is first requested. The attributes produced by this function will be automatically copied anytime rules result in changes to the Generator or its children.
case class GeneratorOuter(child: Generator) extends UnaryExpression with Generator with Product with Serializable
Wrapper around another generator to specify outer behavior.
Wrapper around another generator to specify outer behavior. This is used to implement functions such as explode_outer. This expression gets replaced during analysis.
class GenericInternalRow extends InternalRow with BaseGenericInternalRow
An internal row implementation that uses an array of objects as the underlying storage.
An internal row implementation that uses an array of objects as the underlying storage. Note that, while the array is not copied, and thus could technically be mutated after creation, this is not allowed.
case class Get(left: Expression, right: Expression, replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable
Returns the value of index right in Array left.
Returns the value of index right in Array left. If the index points outside of the array boundaries, then this function returns NULL.

Annotations
@ExpressionDescription()
case class GetArrayItem(child: Expression, ordinal: Expression, failOnError: Boolean = SQLConf.get.ansiEnabled) extends BinaryExpression with GetArrayItemUtil with ExpectsInputTypes with ExtractValue with SupportQueryContext with Product with Serializable
Returns the field at ordinal in the Array child.
Returns the field at ordinal in the Array child.
We need to do type checking here as ordinal expression maybe unresolved.
trait GetArrayItemUtil extends AnyRef
Common trait for GetArrayItem and ElementAt.
case class GetArrayStructFields(child: Expression, field: StructField, ordinal: Int, numFields: Int, containsNull: Boolean) extends UnaryExpression with ExtractValue with Product with Serializable
For a child whose data type is an array of structs, extracts the ordinal-th fields of all array elements, and returns them as a new array.
For a child whose data type is an array of structs, extracts the ordinal-th fields of all array elements, and returns them as a new array.
No need to do type checking since it is handled by ExtractValue.
trait GetDateField extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant
case class GetJsonObject(json: Expression, path: Expression) extends BinaryExpression with ExpectsInputTypes with CodegenFallback with Product with Serializable
Extracts json object from a json string based on json path specified, and returns json string of the extracted json object.
Extracts json object from a json string based on json path specified, and returns json string of the extracted json object. It will return null if the input json string is invalid.

Annotations
@ExpressionDescription()
case class GetMapValue(child: Expression, key: Expression) extends BinaryExpression with GetMapValueUtil with ExtractValue with Product with Serializable
Returns the value of key key in Map child.
Returns the value of key key in Map child.
We need to do type checking here as key expression maybe unresolved.
trait GetMapValueUtil extends BinaryExpression with ImplicitCastInputTypes
Common trait for GetMapValue and ElementAt.
case class GetStructField(child: Expression, ordinal: Int, name: Option[String] = None) extends UnaryExpression with ExtractValue with Product with Serializable
Returns the value of fields in the Struct child.
Returns the value of fields in the Struct child.
No need to do type checking since it is handled by ExtractValue.
Note that we can pass in the field name directly to keep case preserving in toString. For example, when get field yEAr from <year: int, month: int>, we should pass in yEAr.
trait GetTimeField extends UnaryExpression with TimeZoneAwareExpression with ImplicitCastInputTypes with NullIntolerant
case class GetTimestamp(left: Expression, right: Expression, dataType: DataType, timeZoneId: Option[String] = None, failOnError: Boolean = SQLConf.get.ansiEnabled) extends ToTimestamp with Product with Serializable
Gets a timestamp from a string or a date.
case class GreaterThan(left: Expression, right: Expression) extends BinaryComparison with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class GreaterThanOrEqual(left: Expression, right: Expression) extends BinaryComparison with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class Greatest(children: Seq[Expression]) extends Expression with ComplexTypeMergingExpression with CommutativeExpression with Product with Serializable
A function that returns the greatest value of all parameters, skipping null values.
A function that returns the greatest value of all parameters, skipping null values. It takes at least 2 parameters, and returns null iff all parameters are null.

Annotations
@ExpressionDescription()
case class Grouping(child: Expression) extends Expression with Unevaluable with UnaryLike[Expression] with Product with Serializable
Indicates whether a specified column expression in a GROUP BY list is aggregated or not.
Indicates whether a specified column expression in a GROUP BY list is aggregated or not. GROUPING returns 1 for aggregated or 0 for not aggregated in the result set.

Annotations
@ExpressionDescription()
case class GroupingID(groupByExprs: Seq[Expression]) extends Expression with Unevaluable with Product with Serializable
GroupingID is a function that computes the level of grouping.
GroupingID is a function that computes the level of grouping.
If groupByExprs is empty, it means all grouping expressions in GroupingSets.

Annotations
@ExpressionDescription()
case class GroupingSets(groupingSetIndexes: Seq[Seq[Int]], flatGroupingSets: Seq[Expression], userGivenGroupByExprs: Seq[Expression]) extends Expression with BaseGroupingSets with Product with Serializable
abstract class HashExpression[E] extends Expression
A function that calculates hash value for a group of expressions.
A function that calculates hash value for a group of expressions. Note that the seed argument is not exposed to users and should only be set inside spark SQL.
The hash value for an expression depends on its type and seed:
- null: seed
- boolean: turn boolean into int, 1 for true, 0 for false, and then use murmur3 to hash this int with seed.
- byte, short, int: use murmur3 to hash the input as int with seed.
- long: use murmur3 to hash the long input with seed.
- float: turn it into int: java.lang.Float.floatToIntBits(input), and hash it.
- double: turn it into long: java.lang.Double.doubleToLongBits(input), and hash it.
- decimal: if it's a small decimal, i.e. precision <= 18, turn it into long and hash it. Else, turn it into bytes and hash it.
- calendar interval: hash microseconds first, and use the result as seed to hash months.
- interval day to second: it store long value of microseconds, use murmur3 to hash the long input with seed.
- interval year to month: it store int value of months, use murmur3 to hash the int input with seed.
- binary: use murmur3 to hash the bytes with seed.
- string: get the bytes of string and hash it.
- array: The result starts with seed, then use result as seed, recursively calculate hash value for each element, and assign the element hash value to result.
- struct: The result starts with seed, then use result as seed, recursively calculate hash value for each field, and assign the field hash value to result.
Finally we aggregate the hash values for each expression by the same way of struct.
case class Hex(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
If the argument is an INT or binary, hex returns the number as a STRING in hexadecimal format.
If the argument is an INT or binary, hex returns the number as a STRING in hexadecimal format. Otherwise if the number is a STRING, it converts each character into its hex representation and returns the resulting STRING. Negative numbers would be treated as two's complement.

Annotations
@ExpressionDescription()
trait HigherOrderFunction extends Expression with ExpectsInputTypes
A higher order function takes one or more (lambda) functions and applies these to some objects.
A higher order function takes one or more (lambda) functions and applies these to some objects. The function produces a number of variables which can be consumed by some lambda function.
case class HiveHash(children: Seq[Expression]) extends HashExpression[Int] with Product with Serializable
Simulates Hive's hashing function from Hive v1.2.1 at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils#hashcode()
Simulates Hive's hashing function from Hive v1.2.1 at org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorUtils#hashcode()
We should use this hash function for both shuffle and bucket of Hive tables, so that we can guarantee shuffle and bucketing have same data distribution

Annotations
@ExpressionDescription()
case class HllSketchEstimate(child: Expression) extends UnaryExpression with CodegenFallback with ExpectsInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class HllUnion(first: Expression, second: Expression, third: Expression) extends TernaryExpression with CodegenFallback with ExpectsInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class Hour(child: Expression, timeZoneId: Option[String] = None) extends UnaryExpression with GetTimeField with Product with Serializable

Annotations
@ExpressionDescription()
case class Hours(child: Expression) extends PartitionTransformExpression with Product with Serializable
Expression for the v2 partition transform hours.
case class Hypot(left: Expression, right: Expression) extends BinaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class ILike(left: Expression, right: Expression, escapeChar: Char) extends Expression with RuntimeReplaceable with ImplicitCastInputTypes with BinaryLike[Expression] with Product with Serializable
Simple RegEx case-insensitive pattern matching function
Simple RegEx case-insensitive pattern matching function

Annotations
@ExpressionDescription()
case class If(predicate: Expression, trueValue: Expression, falseValue: Expression) extends Expression with ComplexTypeMergingExpression with ConditionalExpression with TernaryLike[Expression] with Product with Serializable

Annotations
@ExpressionDescription()
trait ImplicitCastInputTypes extends Expression with ExpectsInputTypes
A mixin for the analyzer to perform implicit type casting using org.apache.spark.sql.catalyst.analysis.TypeCoercion.ImplicitTypeCasts.
case class In(value: Expression, list: Seq[Expression]) extends Expression with Predicate with Product with Serializable
Evaluates to true if list contains value.
Evaluates to true if list contains value.

Annotations
@ExpressionDescription()
case class InSet(child: Expression, hset: Set[Any]) extends UnaryExpression with Predicate with Product with Serializable
Optimized version of In clause, when all filter values of In clause are static.
case class InSubquery(values: Seq[Expression], query: ListQuery) extends Expression with Predicate with Unevaluable with Product with Serializable
Evaluates to true if values are returned in query's result set.
trait InheritAnalysisRules extends UnaryLike[Expression]
An add-on of RuntimeReplaceable.
An add-on of RuntimeReplaceable. It makes replacement the child of the expression, to inherit the analysis rules for it, such as type coercion. The implementation should put replacement in the case class constructor, and define a normal constructor that accepts only the original parameters. For an example, see TryAdd. To make sure the explain plan and expression SQL works correctly, the implementation should also implement the parameters method.
case class InitCap(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Returns string, with the first letter of each word in uppercase, all other letters in lowercase.
Returns string, with the first letter of each word in uppercase, all other letters in lowercase. Words are delimited by whitespace.

Annotations
@ExpressionDescription()
case class Inline(child: Expression) extends UnaryExpression with CollectionGenerator with Product with Serializable
Explodes an array of structs into a table.
Explodes an array of structs into a table.

Annotations
@ExpressionDescription()
case class InputFileBlockLength() extends LeafExpression with Nondeterministic with Product with Serializable

Annotations
@ExpressionDescription()
case class InputFileBlockStart() extends LeafExpression with Nondeterministic with Product with Serializable

Annotations
@ExpressionDescription()
case class InputFileName() extends LeafExpression with Nondeterministic with Product with Serializable

Annotations
@ExpressionDescription()
case class IntegralDivide(left: Expression, right: Expression, evalMode: EvalMode.Value = EvalMode.fromSQLConf(SQLConf.get)) extends BinaryArithmetic with DivModLike with Product with Serializable

Annotations
@ExpressionDescription()
abstract class IntegralToTimestampBase extends UnaryExpression with ExpectsInputTypes with NullIntolerant
abstract class InterpretedHashFunction extends AnyRef
Base class for interpreted hash functions.
class InterpretedMutableProjection extends MutableProjection
A MutableProjection that is calculated by calling eval on each of the specified expressions.
class InterpretedOrdering extends BaseOrdering
An interpreted row ordering comparator.
case class InterpretedPredicate(expression: Expression) extends BasePredicate with Product with Serializable
class InterpretedProjection extends Projection
A Projection that is calculated by calling the eval of each of the specified expressions.
class InterpretedSafeProjection extends Projection
An interpreted version of a safe projection.
class InterpretedUnsafeProjection extends UnsafeProjection
An interpreted unsafe projection.
An interpreted unsafe projection. This class reuses the UnsafeRow it produces, a consumer should copy the row if it is being buffered. This class is not thread safe.
trait IntervalDivide extends AnyRef
abstract class IntervalNumOperation extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable
case class IsNaN(child: Expression) extends UnaryExpression with Predicate with ImplicitCastInputTypes with Product with Serializable
Evaluates to true iff it's NaN.
Evaluates to true iff it's NaN.

Annotations
@ExpressionDescription()
case class IsNotNull(child: Expression) extends UnaryExpression with Predicate with Product with Serializable
An expression that is evaluated to true if the input is not null.
An expression that is evaluated to true if the input is not null.

Annotations
@ExpressionDescription()
case class IsNull(child: Expression) extends UnaryExpression with Predicate with Product with Serializable
An expression that is evaluated to true if the input is null.
An expression that is evaluated to true if the input is null.

Annotations
@ExpressionDescription()
class JoinedRow extends InternalRow
A mutable wrapper that makes two rows appear as a single concatenated row.
A mutable wrapper that makes two rows appear as a single concatenated row. Designed to be instantiated once per thread and reused.
case class JsonObjectKeys(child: Expression) extends UnaryExpression with CodegenFallback with ExpectsInputTypes with Product with Serializable
A function which returns all the keys of the outermost JSON object.
A function which returns all the keys of the outermost JSON object.

Annotations
@ExpressionDescription()
case class JsonToStructs(schema: DataType, options: Map[String, String], child: Expression, timeZoneId: Option[String] = None) extends UnaryExpression with TimeZoneAwareExpression with CodegenFallback with ExpectsInputTypes with NullIntolerant with QueryErrorsBase with Product with Serializable
Converts an json input string to a StructType, ArrayType or MapType with the specified schema.
Converts an json input string to a StructType, ArrayType or MapType with the specified schema.

Annotations
@ExpressionDescription()
case class JsonTuple(children: Seq[Expression]) extends Expression with Generator with CodegenFallback with QueryErrorsBase with Product with Serializable

Annotations
@ExpressionDescription()
case class KnownFloatingPointNormalized(child: Expression) extends UnaryExpression with TaggingExpression with Product with Serializable
case class KnownNotNull(child: Expression) extends UnaryExpression with TaggingExpression with Product with Serializable
case class KnownNullable(child: Expression) extends UnaryExpression with TaggingExpression with Product with Serializable
case class Lag(input: Expression, inputOffset: Expression, default: Expression, ignoreNulls: Boolean) extends FrameLessOffsetWindowFunction with TernaryLike[Expression] with Product with Serializable
The Lag function returns the value of input at the offsetth row before the current row in the window.
The Lag function returns the value of input at the offsetth row before the current row in the window. Offsets start at 0, which is the current row. The offset must be constant integer value. The default offset is 1. When the value of input is null at the offsetth row, null is returned. If there is no such offset row, the default expression is evaluated.

Annotations
@ExpressionDescription()
case class LambdaFunction(function: Expression, arguments: Seq[NamedExpression], hidden: Boolean = false) extends Expression with CodegenFallback with Product with Serializable
A lambda function and its arguments.
A lambda function and its arguments. A lambda function can be hidden when a user wants to process an completely independent expression in a HigherOrderFunction, the lambda function and its variables are then only used for internal bookkeeping within the higher order function.
case class LastDay(startDate: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Returns the last day of the month which the date belongs to.
Returns the last day of the month which the date belongs to.

Annotations
@ExpressionDescription()
case class LateralColumnAliasReference(ne: NamedExpression, nameParts: Seq[String], a: Attribute) extends LeafExpression with NamedExpression with Unevaluable with Product with Serializable
A placeholder used to hold a NamedExpression that has been temporarily resolved as the reference to a lateral column alias.
A placeholder used to hold a NamedExpression that has been temporarily resolved as the reference to a lateral column alias. It will be restored back to UnresolvedAttribute if the lateral column alias can't be resolved, or become a normal resolved column in the rewritten plan after lateral column resolution. There should be no LateralColumnAliasReference beyond analyzer: if the plan passes all analysis check, then all LateralColumnAliasReference should already be removed.
ne
the NamedExpression produced by column resolution. Can be UnresolvedAttribute if the referenced lateral column alias is not resolved yet.
nameParts
the name parts of the original UnresolvedAttribute. Used to restore back to UnresolvedAttribute when needed
a
the attribute of referenced lateral column alias. Used to match alias when unwrapping and resolving lateral column aliases and rewriting the query plan.
case class LateralSubquery(plan: LogicalPlan, outerAttrs: Seq[Expression] = Seq.empty, exprId: ExprId = NamedExpression.newExprId, joinCond: Seq[Expression] = Seq.empty, hint: Option[HintInfo] = None) extends SubqueryExpression with Unevaluable with Product with Serializable
A subquery that can return multiple rows and columns.
A subquery that can return multiple rows and columns. This should be rewritten as a join with the outer query during the optimization phase.
Note: exprId is used to have a unique name in explain string output.
case class Lead(input: Expression, offset: Expression, default: Expression, ignoreNulls: Boolean) extends FrameLessOffsetWindowFunction with TernaryLike[Expression] with Product with Serializable
The Lead function returns the value of input at the offsetth row after the current row in the window.
The Lead function returns the value of input at the offsetth row after the current row in the window. Offsets start at 0, which is the current row. The offset must be constant integer value. The default offset is 1. When the value of input is null at the offsetth row, null is returned. If there is no such offset row, the default expression is evaluated.

Annotations
@ExpressionDescription()
abstract class LeafExpression extends Expression with LeafLike[Expression]
A leaf expression, i.e.
A leaf expression, i.e. one without any child expressions.
abstract class LeafMathExpression extends LeafExpression with CodegenFallback with Serializable
A leaf expression specifically for math constants.
A leaf expression specifically for math constants. Math constants expect no input.
There is no code generation because they should get constant folded by the optimizer.
case class Least(children: Seq[Expression]) extends Expression with ComplexTypeMergingExpression with CommutativeExpression with Product with Serializable
A function that returns the least value of all parameters, skipping null values.
A function that returns the least value of all parameters, skipping null values. It takes at least 2 parameters, and returns null iff all parameters are null.

Annotations
@ExpressionDescription()
case class Left(str: Expression, len: Expression) extends Expression with RuntimeReplaceable with ImplicitCastInputTypes with BinaryLike[Expression] with Product with Serializable
Returns the leftmost n characters from the string.
Returns the leftmost n characters from the string.

Annotations
@ExpressionDescription()
case class Length(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
A function that returns the char length of the given string expression or number of bytes of the given binary expression.
A function that returns the char length of the given string expression or number of bytes of the given binary expression.

Annotations
@ExpressionDescription()
case class LengthOfJsonArray(child: Expression) extends UnaryExpression with CodegenFallback with ExpectsInputTypes with Product with Serializable
A function that returns the number of elements in the outermost JSON array.
A function that returns the number of elements in the outermost JSON array.

Annotations
@ExpressionDescription()
case class LessThan(left: Expression, right: Expression) extends BinaryComparison with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class LessThanOrEqual(left: Expression, right: Expression) extends BinaryComparison with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class Levenshtein(left: Expression, right: Expression, threshold: Option[Expression] = None) extends Expression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
A function that return the Levenshtein distance between the two given strings.
A function that return the Levenshtein distance between the two given strings.

Annotations
@ExpressionDescription()
case class Like(left: Expression, right: Expression, escapeChar: Char) extends StringRegexExpression with Product with Serializable
Simple RegEx pattern matching function
Simple RegEx pattern matching function

Annotations
@ExpressionDescription()
case class LikeAll(child: Expression, patterns: Seq[UTF8String]) extends LikeAllBase with Product with Serializable
sealed abstract class LikeAllBase extends MultiLikeBase
Optimized version of LIKE ALL, when all pattern values are literal.
case class LikeAny(child: Expression, patterns: Seq[UTF8String]) extends LikeAnyBase with Product with Serializable
sealed abstract class LikeAnyBase extends MultiLikeBase
Optimized version of LIKE ANY, when all pattern values are literal.
case class ListQuery(plan: LogicalPlan, outerAttrs: Seq[Expression] = Seq.empty, exprId: ExprId = NamedExpression.newExprId, numCols: Int = -1, joinCond: Seq[Expression] = Seq.empty, hint: Option[HintInfo] = None) extends SubqueryExpression with Unevaluable with Product with Serializable
A ListQuery expression defines the query which we want to search in an IN subquery expression.
A ListQuery expression defines the query which we want to search in an IN subquery expression. It should and can only be used in conjunction with an IN expression.
For example (SQL):
```
SELECT  *
FROM    a
WHERE   a.id IN (SELECT  id
                 FROM    b)
```
case class Literal(value: Any, dataType: DataType) extends LeafExpression with Product with Serializable
In order to do type checking, use Literal.create() instead of constructor
case class LocalTimestamp(timeZoneId: Option[String] = None) extends LeafExpression with TimeZoneAwareExpression with CodegenFallback with Product with Serializable
Returns the current timestamp without time zone at the start of query evaluation.
Returns the current timestamp without time zone at the start of query evaluation. There is no code generation since this expression should get constant folded by the optimizer.

Annotations
@ExpressionDescription()
case class Log(child: Expression) extends UnaryLogExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Log10(child: Expression) extends UnaryLogExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Log1p(child: Expression) extends UnaryLogExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Log2(child: Expression) extends UnaryLogExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Logarithm(left: Expression, right: Expression) extends BinaryMathExpression with Product with Serializable
Computes the logarithm of a number.
Computes the logarithm of a number.
left
the logarithm base, default to e.
right
the number to compute the logarithm of.

Annotations
@ExpressionDescription()
case class Lower(child: Expression) extends UnaryExpression with String2StringExpression with NullIntolerant with Product with Serializable
A function that converts the characters of a string to lowercase.
A function that converts the characters of a string to lowercase.

Annotations
@ExpressionDescription()
case class Luhncheck(input: Expression) extends Expression with RuntimeReplaceable with ImplicitCastInputTypes with Product with Serializable
Function to check if a given number string is a valid Luhn number.
Function to check if a given number string is a valid Luhn number. Returns true, if the number string is a valid Luhn number, false otherwise.

Annotations
@ExpressionDescription()
case class MakeDTInterval(days: Expression, hours: Expression, mins: Expression, secs: Expression) extends QuaternaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class MakeDate(year: Expression, month: Expression, day: Expression, failOnError: Boolean = SQLConf.get.ansiEnabled) extends TernaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class MakeDecimal(child: Expression, precision: Int, scale: Int, nullOnOverflow: Boolean) extends UnaryExpression with NullIntolerant with Product with Serializable
Create a Decimal from an unscaled Long value.
Create a Decimal from an unscaled Long value. Note: this expression is internal and created only by the optimizer, we don't need to do type check for it.
case class MakeInterval(years: Expression, months: Expression, weeks: Expression, days: Expression, hours: Expression, mins: Expression, secs: Expression, failOnError: Boolean = SQLConf.get.ansiEnabled) extends SeptenaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class MakeTimestamp(year: Expression, month: Expression, day: Expression, hour: Expression, min: Expression, sec: Expression, timezone: Option[Expression] = None, timeZoneId: Option[String] = None, failOnError: Boolean = SQLConf.get.ansiEnabled, dataType: DataType = SQLConf.get.timestampType) extends SeptenaryExpression with TimeZoneAwareExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class MakeYMInterval(years: Expression, months: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable with Product

Annotations
@ExpressionDescription()
trait MapBasedSimpleHigherOrderFunction extends Expression with SimpleHigherOrderFunction
case class MapConcat(children: Seq[Expression]) extends Expression with ComplexTypeMergingExpression with QueryErrorsBase with Product with Serializable
Returns the union of all the given maps.
Returns the union of all the given maps.

Annotations
@ExpressionDescription()
case class MapContainsKey(left: Expression, right: Expression) extends Expression with RuntimeReplaceable with BinaryLike[Expression] with ImplicitCastInputTypes with QueryErrorsBase with Product with Serializable
Returns an unordered array containing the keys of the map.
Returns an unordered array containing the keys of the map.

Annotations
@ExpressionDescription()
case class MapEntries(child: Expression) extends UnaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
Returns an unordered array of all entries in the given map.
Returns an unordered array of all entries in the given map.

Annotations
@ExpressionDescription()
case class MapFilter(argument: Expression, function: Expression) extends Expression with MapBasedSimpleHigherOrderFunction with CodegenFallback with Product with Serializable
Filters entries in a map using the provided function.
Filters entries in a map using the provided function.

Annotations
@ExpressionDescription()
case class MapFromArrays(left: Expression, right: Expression) extends BinaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
Returns a catalyst Map containing the two arrays in children expressions as keys and values.
Returns a catalyst Map containing the two arrays in children expressions as keys and values.

Annotations
@ExpressionDescription()
case class MapFromEntries(child: Expression) extends UnaryExpression with NullIntolerant with QueryErrorsBase with Product with Serializable
Returns a map created from the given array of entries.
Returns a map created from the given array of entries.

Annotations
@ExpressionDescription()
case class MapKeys(child: Expression) extends UnaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
Returns an unordered array containing the keys of the map.
Returns an unordered array containing the keys of the map.

Annotations
@ExpressionDescription()
case class MapValues(child: Expression) extends UnaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
Returns an unordered array containing the values of the map.
Returns an unordered array containing the values of the map.

Annotations
@ExpressionDescription()
case class MapZipWith(left: Expression, right: Expression, function: Expression) extends Expression with HigherOrderFunction with CodegenFallback with TernaryLike[Expression] with Product with Serializable
Merges two given maps into a single map by applying function to the pair of values with the same key.
Merges two given maps into a single map by applying function to the pair of values with the same key.

Annotations
@ExpressionDescription()
case class Mask(input: Expression, upperChar: Expression, lowerChar: Expression, digitChar: Expression, otherChar: Expression) extends QuinaryExpression with ExpectsInputTypes with QueryErrorsBase with Product with Serializable
case class MaskArgument(maskChar: Char, ignore: Boolean) extends Product with Serializable
case class Md5(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
A function that calculates an MD5 128-bit checksum and returns it as a hex string For input of type BinaryType
A function that calculates an MD5 128-bit checksum and returns it as a hex string For input of type BinaryType

Annotations
@ExpressionDescription()
case class MicrosToTimestamp(child: Expression) extends IntegralToTimestampBase with Product with Serializable

Annotations
@ExpressionDescription()
case class MillisToTimestamp(child: Expression) extends IntegralToTimestampBase with Product with Serializable

Annotations
@ExpressionDescription()
case class Minute(child: Expression, timeZoneId: Option[String] = None) extends UnaryExpression with GetTimeField with Product with Serializable

Annotations
@ExpressionDescription()
case class MonotonicallyIncreasingID() extends LeafExpression with Nondeterministic with Product with Serializable
Returns monotonically increasing 64-bit integers.
Returns monotonically increasing 64-bit integers.
The generated ID is guaranteed to be monotonically increasing and unique, but not consecutive. The current implementation puts the partition ID in the upper 31 bits, and the lower 33 bits represent the record number within each partition. The assumption is that the data frame has less than 1 billion partitions, and each partition has less than 8 billion records.
Since this expression is stateful, it cannot be a case object.

Annotations
@ExpressionDescription()
case class Month(child: Expression) extends UnaryExpression with GetDateField with Product with Serializable

Annotations
@ExpressionDescription()
case class Months(child: Expression) extends PartitionTransformExpression with Product with Serializable
Expression for the v2 partition transform months.
case class MonthsBetween(date1: Expression, date2: Expression, roundOff: Expression, timeZoneId: Option[String] = None) extends TernaryExpression with TimeZoneAwareExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Returns number of months between times timestamp1 and timestamp2.
Returns number of months between times timestamp1 and timestamp2. If timestamp1 is later than timestamp2, then the result is positive. If timestamp1 and timestamp2 are on the same day of month, or both are the last day of month, time of day will be ignored. Otherwise, the difference is calculated based on 31 days per month, and rounded to 8 digits unless roundOff=false.

Annotations
@ExpressionDescription()
case class MultiCommutativeOp(operands: Seq[Expression], opCls: Class[_], evalMode: Option[EvalMode.Value])(originalRoot: Expression) extends Expression with Unevaluable with Product with Serializable
A helper class used by the Commutative expressions during canonicalization.
A helper class used by the Commutative expressions during canonicalization. During canonicalization, when we have a long tree of commutative operations, we use the MultiCommutative expression to represent that tree instead of creating new commutative objects. This class is added as a memory optimization for processing large commutative operation trees without creating a large number of new intermediate objects. The MultiCommutativeOp memory optimization is applied to the following commutative expressions: Add, Multiply, And, Or, BitwiseAnd, BitwiseOr, BitwiseXor.
operands
A sequence of operands that produces a commutative expression tree.
opCls
The class of the root operator of the expression tree.
evalMode
The optional expression evaluation mode.
originalRoot
Root operator of the commutative expression tree before canonicalization. This object reference is used to deduce the return dataType of Add and Multiply operations when the input datatype is decimal.
sealed abstract class MultiLikeBase extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Predicate
case class Multiply(left: Expression, right: Expression, evalMode: EvalMode.Value = EvalMode.fromSQLConf(SQLConf.get)) extends BinaryArithmetic with CommutativeExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class MultiplyDTInterval(interval: Expression, num: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable with Product
case class MultiplyInterval(interval: Expression, num: Expression, failOnError: Boolean = SQLConf.get.ansiEnabled) extends IntervalNumOperation with Product with Serializable
case class MultiplyYMInterval(interval: Expression, num: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable with Product
case class Murmur3Hash(children: Seq[Expression], seed: Int) extends HashExpression[Int] with Product with Serializable
A MurMur3 Hash expression.
A MurMur3 Hash expression.
We should use this hash function for both shuffle and bucket, so that we can guarantee shuffle and bucketing have same data distribution.

Annotations
@ExpressionDescription()
final class MutableAny extends MutableValue
final class MutableBoolean extends MutableValue
final class MutableByte extends MutableValue
final class MutableDouble extends MutableValue
final class MutableFloat extends MutableValue
final class MutableInt extends MutableValue
final class MutableLong extends MutableValue
abstract class MutableProjection extends Projection
Converts a InternalRow to another Row given a sequence of expression that define each column of the new row.
Converts a InternalRow to another Row given a sequence of expression that define each column of the new row. If the schema of the input row is specified, then the given expression will be bound to that schema.
In contrast to a normal projection, a MutableProjection reuses the same underlying row object each time an input row is added. This significantly reduces the cost of calculating the projection, but means that it is not safe to hold on to a reference to a InternalRow after next() has been called on the Iterator that produced it. Instead, the user must call InternalRow.copy() and hold on to the returned InternalRow before calling next().
final class MutableShort extends MutableValue

abstract class MutableValue extends Serializable

A parent class for mutable container objects that are reused when the values are changed, resulting in less garbage.

A parent class for mutable container objects that are reused when the values are changed, resulting in less garbage. These values are held by a SpecificInternalRow.

The following code was roughly used to generate these objects:

val types = "Int,Float,Boolean,Double,Short,Long,Byte,Any".split(",")
types.map {tpe =>
s"""
final class Mutable$tpe extends MutableValue {
  var value: $tpe = 0
  def boxed = if (isNull) null else value
  def update(v: Any) = value = {
    isNull = false
    v.asInstanceOf[$tpe]
  }
  def copy() = {
    val newCopy = new Mutable$tpe
    newCopy.isNull = isNull
    newCopy.value = value
    newCopy
  }
}"""
}.foreach(println)

types.map { tpe =>
s"""
  override def set$tpe(ordinal: Int, value: $tpe): Unit = {
    val currentValue = values(ordinal).asInstanceOf[Mutable$tpe]
    currentValue.isNull = false
    currentValue.value = value
  }

  override def get$tpe(i: Int): $tpe = {
    values(i).asInstanceOf[Mutable$tpe].value
  }"""
}.foreach(println)

case class NTile(buckets: Expression) extends RowNumberLike with SizeBasedWindowFunction with UnaryLike[Expression] with QueryErrorsBase with Product with Serializable
The NTile function divides the rows for each window partition into n buckets ranging from 1 to at most n.
The NTile function divides the rows for each window partition into n buckets ranging from 1 to at most n. Bucket values will differ by at most 1. If the number of rows in the partition does not divide evenly into the number of buckets, then the remainder values are distributed one per bucket, starting with the first bucket.
The NTile function is particularly useful for the calculation of tertiles, quartiles, deciles and other common summary statistics
The function calculates two variables during initialization: The size of a regular bucket, and the number of buckets that will have one extra row added to it (when the rows do not evenly fit into the number of buckets); both variables are based on the size of the current partition. During the calculation process the function keeps track of the current row number, the current bucket number, and the row number at which the bucket will change (bucketThreshold). When the current row number reaches bucket threshold, the bucket value is increased by one and the threshold is increased by the bucket size (plus one extra if the current bucket is padded).
This documentation has been based upon similar documentation for the Hive and Presto projects.

Annotations
@ExpressionDescription()
case class NaNvl(left: Expression, right: Expression) extends BinaryExpression with ConditionalExpression with ImplicitCastInputTypes with Product with Serializable
An Expression evaluates to left iff it's not NaN, or evaluates to right otherwise.
An Expression evaluates to left iff it's not NaN, or evaluates to right otherwise. This Expression is useful for mapping NaN values to null.

Annotations
@ExpressionDescription()
case class NamedArgumentExpression(key: String, value: Expression) extends UnaryExpression with Unevaluable with Product with Serializable
This represents an argument expression to a function call accompanied with an explicit reference to the corresponding argument name as a string.
This represents an argument expression to a function call accompanied with an explicit reference to the corresponding argument name as a string. In this way, the analyzer can make sure that the provided values match up to the arguments as intended, and the arguments may appear in any order. This unary expression is unevaluable because we intend to replace it with the provided value itself during query analysis (after possibly rearranging the parsed argument list to match up the names to the expected function signature).
SQL Syntax: key => value SQL grammar: key=identifier FAT_ARROW value=expression
Example usage with the "encode" scalar function: SELECT encode("abc", charset => "utf-8"); The second argument generates NamedArgumentExpression("charset", Literal("utf-8")) SELECT encode(charset => "utf-8", value => "abc");
key
The name of the function argument
value
The value of the function argument
trait NamedExpression extends Expression
An Expression that is named.
case class NamedLambdaVariable(name: String, dataType: DataType, nullable: Boolean, exprId: ExprId = NamedExpression.newExprId, value: AtomicReference[Any] = new AtomicReference()) extends LeafExpression with NamedExpression with CodegenFallback with Product with Serializable
A named lambda variable.
case class NextDay(startDate: Expression, dayOfWeek: Expression, failOnError: Boolean = SQLConf.get.ansiEnabled) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Returns the first date which is later than startDate and named as dayOfWeek.
Returns the first date which is later than startDate and named as dayOfWeek. For example, NextDay(2015-07-27, Sunday) would return 2015-08-02, which is the first Sunday later than 2015-07-27.
Allowed "dayOfWeek" is defined in DateTimeUtils.getDayOfWeekFromString.

Annotations
@ExpressionDescription()
trait NoThrow extends AnyRef
Trait to indicate the expression does not throw an exception by itself when they are evaluated.
Trait to indicate the expression does not throw an exception by itself when they are evaluated. For example, UDFs, AssertTrue, etc can throw an exception when they are executed. In such case, it is necessary to call Expression.eval, and the optimization rule should not ignore it.
This trait can be used in an optimization rule such as org.apache.spark.sql.catalyst.optimizer.ConstantFolding to fold the expressions that do not need to execute, for example, size(array(c0, c1, c2)).
trait NonSQLExpression extends Expression
Expressions that don't have SQL representation should extend this trait.
Expressions that don't have SQL representation should extend this trait. Examples are ScalaUDF, ScalaUDAF, and object expressions like MapObjects and Invoke.
trait Nondeterministic extends Expression
An expression that is nondeterministic.
case class Not(child: Expression) extends UnaryExpression with Predicate with ImplicitCastInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class NotLikeAll(child: Expression, patterns: Seq[UTF8String]) extends LikeAllBase with Product with Serializable
case class NotLikeAny(child: Expression, patterns: Seq[UTF8String]) extends LikeAnyBase with Product with Serializable
case class Now() extends CurrentTimestampLike with Product with Serializable

Annotations
@ExpressionDescription()
case class NthValue(input: Expression, offset: Expression, ignoreNulls: Boolean) extends AggregateWindowFunction with OffsetWindowFunction with ImplicitCastInputTypes with BinaryLike[Expression] with QueryErrorsBase with Product with Serializable

Annotations
@ExpressionDescription()
case class NullIf(left: Expression, right: Expression, replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable

Annotations
@ExpressionDescription()
case class NullIndex(input: Expression) extends AggregateWindowFunction with UnaryLike[Expression] with Product with Serializable
Return the indices for consecutive null values, for non-null values, it returns 0.
Return the indices for consecutive null values, for non-null values, it returns 0. This expression is dedicated only for Pandas API on Spark. For example, Input: null, 1, 2, 3, null, null, null, 5, null, null Output: 1, 0, 0, 0, 1, 2, 3, 0, 1, 2
trait NullIntolerant extends Expression
When an expression inherits this, meaning the expression is null intolerant (i.e.
When an expression inherits this, meaning the expression is null intolerant (i.e. any null input will result in null output). We will use this information during constructing IsNotNull constraints.
sealed abstract class NullOrdering extends AnyRef
case class Nvl(left: Expression, right: Expression, replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable

Annotations
@ExpressionDescription()
case class Nvl2(expr1: Expression, expr2: Expression, expr3: Expression, replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable

Annotations
@ExpressionDescription()
case class OctetLength(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
A function that returns the byte length of the given string or binary expression.
A function that returns the byte length of the given string or binary expression.

Annotations
@ExpressionDescription()
trait OffsetWindowFunction extends Expression with WindowFunction
case class Or(left: Expression, right: Expression) extends BinaryOperator with Predicate with CommutativeExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class OuterReference(e: NamedExpression) extends LeafExpression with NamedExpression with Unevaluable with Product with Serializable
A place holder used to hold a reference that has been resolved to a field outside of the current plan.
A place holder used to hold a reference that has been resolved to a field outside of the current plan. This is used for correlated subqueries.
case class Overlay(input: Expression, replace: Expression, pos: Expression, len: Expression) extends QuaternaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
trait PadExpressionBuilderBase extends ExpressionBuilder
case class ParseToDate(left: Expression, format: Option[Expression], timeZoneId: Option[String] = None, ansiEnabled: Boolean = SQLConf.get.ansiEnabled) extends Expression with RuntimeReplaceable with ImplicitCastInputTypes with TimeZoneAwareExpression with Product with Serializable
Parses a column to a date based on the given format.
Parses a column to a date based on the given format.

Annotations
@ExpressionDescription()
case class ParseToTimestamp(left: Expression, format: Option[Expression], dataType: DataType, timeZoneId: Option[String] = None, failOnError: Boolean = SQLConf.get.ansiEnabled) extends Expression with RuntimeReplaceable with ImplicitCastInputTypes with TimeZoneAwareExpression with Product with Serializable
Parses a column to a timestamp based on the supplied format.
Parses a column to a timestamp based on the supplied format.

Annotations
@ExpressionDescription()
case class ParseUrl(children: Seq[Expression], failOnError: Boolean = SQLConf.get.ansiEnabled) extends Expression with ExpectsInputTypes with CodegenFallback with Product with Serializable
Extracts a part from a URL
Extracts a part from a URL

Annotations
@ExpressionDescription()
abstract class PartitionTransformExpression extends Expression with Unevaluable with UnaryLike[Expression]
Base class for expressions that are converted to v2 partition transforms.
Base class for expressions that are converted to v2 partition transforms.
Subclasses represent abstract transform functions with concrete implementations that are determined by data source implementations. Because the concrete implementation is not known, these expressions are Unevaluable.
These expressions are used to pass transformations from the DataFrame API:
```
df.writeTo("catalog.db.table").partitionedBy($"category", days($"timestamp")).create()
```
case class PercentRank(children: Seq[Expression]) extends RankLike with SizeBasedWindowFunction with Product with Serializable
The PercentRank function computes the percentage ranking of a value in a group of values.
The PercentRank function computes the percentage ranking of a value in a group of values. The result the rank of the minus one divided by the total number of rows in the partition minus one: (r - 1) / (n - 1). If a partition only contains one row, the function will return 0.
The PercentRank function is similar to the CumeDist function, but it uses rank values instead of row counts in the its numerator.
This documentation has been based upon similar documentation for the Hive and Presto projects.

Annotations
@ExpressionDescription()
case class Pi() extends LeafMathExpression with Product with Serializable
Pi.
Pi. Note that there is no code generation because this is only evaluated by the optimizer during constant folding.

Annotations
@ExpressionDescription()
abstract class PlanExpression[T <: QueryPlan[_]] extends Expression
An interface for expressions that contain a QueryPlan.
case class Pmod(left: Expression, right: Expression, evalMode: EvalMode.Value = EvalMode.fromSQLConf(SQLConf.get)) extends BinaryArithmetic with Product with Serializable

Annotations
@ExpressionDescription()
case class PosExplode(child: Expression) extends ExplodeBase with Product with Serializable
Given an input array produces a sequence of rows for each position and value in the array.
Given an input array produces a sequence of rows for each position and value in the array.
```
SELECT posexplode(array(10,20)) ->
0  10
1  20
```
Annotations
@ExpressionDescription()
case class Pow(left: Expression, right: Expression) extends BinaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class PreciseTimestampConversion(child: Expression, fromType: DataType, toType: DataType) extends UnaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
Expression used internally to convert the TimestampType to Long and back without losing precision, i.e.
Expression used internally to convert the TimestampType to Long and back without losing precision, i.e. in microseconds. Used in time windowing.
trait Predicate extends Expression
An Expression that returns a boolean value.
trait PredicateHelper extends AliasHelper with Logging
case class PrettyAttribute(name: String, dataType: DataType = NullType) extends Attribute with Unevaluable with Product with Serializable
A place holder used when printing expressions without debugging information such as the expression id or the unresolved indicator.
case class PrettyPythonUDF(name: String, dataType: DataType, children: Seq[Expression]) extends UnevaluableAggregateFunc with NonSQLExpression with Product with Serializable
A place holder used when printing expressions without debugging information such as the result id.
case class PrintToStderr(child: Expression) extends UnaryExpression with Product with Serializable
Print the result of an expression to stderr (used for debugging codegen).
abstract class Projection extends (InternalRow) ⇒ InternalRow with ExpressionsEvaluator
Converts a InternalRow to another Row given a sequence of expression that define each column of the new row.
Converts a InternalRow to another Row given a sequence of expression that define each column of the new row. If the schema of the input row is specified, then the given expression will be bound to that schema.
case class ProjectionOverSchema(schema: StructType, output: AttributeSet) extends Product with Serializable
A Scala extractor that projects an expression over a given schema.
A Scala extractor that projects an expression over a given schema. Data types, field indexes and field counts of complex type extractors and attributes are adjusted to fit the schema. All other expressions are left as-is. This class is motivated by columnar nested schema pruning.
schema
nested column schema
output
output attributes of the data source relation. They are used to filter out attributes in the schema that do not belong to the current relation.
trait PythonFuncExpression extends Expression with NonSQLExpression with UserDefinedExpression
case class PythonUDAF(name: String, func: PythonFunction, dataType: DataType, children: Seq[Expression], udfDeterministic: Boolean, resultId: ExprId = NamedExpression.newExprId) extends UnevaluableAggregateFunc with PythonFuncExpression with Product with Serializable
A serialized version of a Python lambda function for aggregation.
A serialized version of a Python lambda function for aggregation. This is a special expression, which needs a dedicated physical operator to execute it, instead of the normal Aggregate operator.
case class PythonUDF(name: String, func: PythonFunction, dataType: DataType, children: Seq[Expression], evalType: Int, udfDeterministic: Boolean, resultId: ExprId = NamedExpression.newExprId) extends Expression with PythonFuncExpression with Unevaluable with Product with Serializable
A serialized version of a Python lambda function.
A serialized version of a Python lambda function. This is a special expression, which needs a dedicated physical operator to execute it, and thus can't be pushed down to data sources.
case class PythonUDTF(name: String, func: PythonFunction, elementSchema: StructType, children: Seq[Expression], evalType: Int, udfDeterministic: Boolean, resultId: ExprId = NamedExpression.newExprId, pythonUDTFPartitionColumnIndexes: Option[PythonUDTFPartitionColumnIndexes] = None, analyzeResult: Option[PythonUDTFAnalyzeResult] = None) extends UnevaluableGenerator with PythonFuncExpression with Product with Serializable
A serialized version of a Python table-valued function call.
A serialized version of a Python table-valued function call. This is a special expression, which needs a dedicated physical operator to execute it.
name
name of the Python UDTF being called
func
string contents of the Python code in the UDTF, along with other environment state
elementSchema
result schema of the function call
children
input arguments to the UDTF call; for scalar arguments these are the expressions themeselves, and for TABLE arguments, these are instances of FunctionTableSubqueryArgumentExpression
evalType
identifies whether this is a scalar or aggregate or table function, using an instance of the PythonEvalType enumeration
udfDeterministic
true if this function is deterministic wherein it returns the same result rows for every call with the same input arguments
resultId
unique expression ID for this function invocation
pythonUDTFPartitionColumnIndexes
holds the indexes of the TABLE argument to the Python UDTF call, if applicable
analyzeResult
holds the result of the polymorphic Python UDTF 'analze' method, if the UDTF defined one
case class PythonUDTFAnalyzeResult(schema: StructType, withSinglePartition: Boolean, partitionByExpressions: Seq[Expression], orderByExpressions: Seq[SortOrder]) extends Product with Serializable
Represents the result of invoking the polymorphic 'analyze' method on a Python user-defined table function.
Represents the result of invoking the polymorphic 'analyze' method on a Python user-defined table function. This returns the table function's output schema in addition to other optional metadata.
schema
result schema of this particular function call in response to the particular arguments provided, including the types of any provided scalar arguments (and their values, in the case of literals) as well as the names and types of columns of the provided TABLE argument (if any)
withSinglePartition
true if the 'analyze' method explicitly indicated that the UDTF call should consume all rows of the input TABLE argument in a single instance of the UDTF class, in which case Catalyst will invoke a repartitioning to a separate stage with a single worker for this purpose
partitionByExpressions
if non-empty, this contains the list of column names that the 'analyze' method explicitly indicated that the UDTF call should partition the input table by, wherein all rows corresponding to each unique combination of values of the partitioning columns are consumed by exactly one unique instance of the UDTF class
orderByExpressions
if non-empty, this contains the list of ordering items that the 'analyze' method explicitly indicated that the UDTF call should consume the input table rows by
case class PythonUDTFPartitionColumnIndexes(partitionChildIndexes: Seq[Int]) extends Product with Serializable
Holds the indexes of the TABLE argument to a Python UDTF call, if applicable.
Holds the indexes of the TABLE argument to a Python UDTF call, if applicable.
partitionChildIndexes
The indexes of the partitioning columns in each TABLE argument.
case class Quarter(child: Expression) extends UnaryExpression with GetDateField with Product with Serializable

Annotations
@ExpressionDescription()
abstract class QuaternaryExpression extends Expression with QuaternaryLike[Expression]
An expression with four inputs and one output.
An expression with four inputs and one output. The output is by default evaluated to null if any input is evaluated to null.
abstract class QuinaryExpression extends Expression
An expression with five inputs and one output.
An expression with five inputs and one output. The output is by default evaluated to null if any input is evaluated to null.
abstract class RDG extends UnaryExpression with ExpectsInputTypes with Nondeterministic with ExpressionWithRandomSeed
A Random distribution generating expression.
A Random distribution generating expression. TODO: This can be made generic to generate any type of random distribution, or any type of StructType.
Since this expression is stateful, it cannot be a case object.
case class RLike(left: Expression, right: Expression) extends StringRegexExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class RaiseError(child: Expression, dataType: DataType) extends UnaryExpression with ImplicitCastInputTypes with Product with Serializable
Throw with the result of an expression (used for debugging).
Throw with the result of an expression (used for debugging).

Annotations
@ExpressionDescription()
case class Rand(child: Expression, hideSeed: Boolean = false) extends RDG with Product with Serializable
Generate a random column with i.i.d.
Generate a random column with i.i.d. uniformly distributed values in [0, 1).

Annotations
@ExpressionDescription()
case class Randn(child: Expression, hideSeed: Boolean = false) extends RDG with Product with Serializable
Generate a random column with i.i.d.
Generate a random column with i.i.d. values drawn from the standard normal distribution.

Annotations
@ExpressionDescription()
case class Rank(children: Seq[Expression]) extends RankLike with Product with Serializable
The Rank function computes the rank of a value in a group of values.
The Rank function computes the rank of a value in a group of values. The result is one plus the number of rows preceding or equal to the current row in the ordering of the partition. The values will produce gaps in the sequence.
This documentation has been based upon similar documentation for the Hive and Presto projects.

Annotations
@ExpressionDescription()
abstract class RankLike extends AggregateWindowFunction
A RankLike function is a WindowFunction that changes its value based on a change in the value of the order of the window in which is processed.
A RankLike function is a WindowFunction that changes its value based on a change in the value of the order of the window in which is processed. For instance, when the value of input changes in a window ordered by input the rank function also changes. The size of the change of the rank function is (typically) not dependent on the size of the change in input.
This documentation has been based upon similar documentation for the Hive and Presto projects.
case class RegExpCount(left: Expression, right: Expression) extends Expression with RuntimeReplaceable with ImplicitCastInputTypes with Product with Serializable

Annotations
@ExpressionDescription()
case class RegExpExtract(subject: Expression, regexp: Expression, idx: Expression) extends RegExpExtractBase with Product with Serializable
Extract a specific(idx) group identified by a Java regex.
Extract a specific(idx) group identified by a Java regex.
NOTE: this expression is not THREAD-SAFE, as it has some internal mutable status.

Annotations
@ExpressionDescription()
case class RegExpExtractAll(subject: Expression, regexp: Expression, idx: Expression) extends RegExpExtractBase with Product with Serializable
Extract all specific(idx) groups identified by a Java regex.
Extract all specific(idx) groups identified by a Java regex.
NOTE: this expression is not THREAD-SAFE, as it has some internal mutable status.

Annotations
@ExpressionDescription()
abstract class RegExpExtractBase extends TernaryExpression with ImplicitCastInputTypes with NullIntolerant
case class RegExpInStr(subject: Expression, regexp: Expression, idx: Expression) extends RegExpExtractBase with Product with Serializable

Annotations
@ExpressionDescription()
case class RegExpReplace(subject: Expression, regexp: Expression, rep: Expression, pos: Expression) extends QuaternaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Replace all substrings of str that match regexp with rep.
Replace all substrings of str that match regexp with rep.
NOTE: this expression is not THREAD-SAFE, as it has some internal mutable status.

Annotations
@ExpressionDescription()
case class RegExpSubStr(left: Expression, right: Expression) extends Expression with RuntimeReplaceable with ImplicitCastInputTypes with Product with Serializable

Annotations
@ExpressionDescription()
case class Remainder(left: Expression, right: Expression, evalMode: EvalMode.Value = EvalMode.fromSQLConf(SQLConf.get)) extends BinaryArithmetic with DivModLike with Product with Serializable

Annotations
@ExpressionDescription()
case class ReplicateRows(children: Seq[Expression]) extends Expression with Generator with CodegenFallback with Product with Serializable
Replicate the row N times.
Replicate the row N times. N is specified as the first argument to the function. This is an internal function solely used by optimizer to rewrite EXCEPT ALL AND INTERSECT ALL queries.
case class ResultProxy(result: Any) extends Product with Serializable
A simple wrapper for holding Any in the cache of SubExprEvaluationRuntime.
case class Reverse(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Returns a reversed string or an array with reverse order of elements.
Returns a reversed string or an array with reverse order of elements.

Annotations
@ExpressionDescription()
case class Right(str: Expression, len: Expression) extends Expression with RuntimeReplaceable with ImplicitCastInputTypes with BinaryLike[Expression] with Product with Serializable
Returns the rightmost n characters from the string.
Returns the rightmost n characters from the string.

Annotations
@ExpressionDescription()
case class Rint(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Rollup(groupingSetIndexes: Seq[Seq[Int]], children: Seq[Expression]) extends Expression with BaseGroupingSets with Product with Serializable
case class Round(child: Expression, scale: Expression, ansiEnabled: Boolean = SQLConf.get.ansiEnabled) extends RoundBase with Product with Serializable
Round an expression to d decimal places using HALF_UP rounding mode.
Round an expression to d decimal places using HALF_UP rounding mode. round(2.5) == 3.0, round(3.5) == 4.0.

Annotations
@ExpressionDescription()
abstract class RoundBase extends BinaryExpression with Serializable with ImplicitCastInputTypes with SupportQueryContext
Round the child's result to scale decimal place when scale >= 0 or round at integral part when scale < 0.
Round the child's result to scale decimal place when scale >= 0 or round at integral part when scale < 0.
Child of IntegralType would round to itself when scale >= 0. Child of FractionalType whose value is NaN or Infinite would always round to itself.
Round's dataType would always equal to child's dataType except for DecimalType, which would lead scale decrease from the origin DecimalType.
case class RoundCeil(child: Expression, scale: Expression) extends RoundBase with Product with Serializable
case class RoundFloor(child: Expression, scale: Expression) extends RoundBase with Product with Serializable
abstract class RowBasedKeyValueBatch extends MemoryConsumer with Closeable
RowBasedKeyValueBatch stores key value pairs in contiguous memory region.
RowBasedKeyValueBatch stores key value pairs in contiguous memory region.
Each key or value is stored as a single UnsafeRow. Each record contains one key and one value and some auxiliary data, which differs based on implementation: i.e., FixedLengthRowBasedKeyValueBatch and VariableLengthRowBasedKeyValueBatch.
We use FixedLengthRowBasedKeyValueBatch if all fields in the key and the value are fixed-length data types. Otherwise we use VariableLengthRowBasedKeyValueBatch.
RowBasedKeyValueBatch is backed by a single page / MemoryBlock (ranges from 1 to 64MB depending on the system configuration). If the page is full, the aggregate logic should fallback to a second level, larger hash map. We intentionally use the single-page design because it simplifies memory address encoding & decoding for each key-value pair. Because the maximum capacity for RowBasedKeyValueBatch is only 2^16, it is unlikely we need a second page anyway. Filling the page requires an average size for key value pairs to be larger than 1024 bytes.
case class RowNumber() extends RowNumberLike with LeafLike[Expression] with Product with Serializable
The RowNumber function computes a unique, sequential number to each row, starting with one, according to the ordering of rows within the window partition.
The RowNumber function computes a unique, sequential number to each row, starting with one, according to the ordering of rows within the window partition.
This documentation has been based upon similar documentation for the Hive and Presto projects.

Annotations
@ExpressionDescription()
abstract class RowNumberLike extends AggregateWindowFunction
trait RuntimeReplaceable extends Expression
An expression that gets replaced at runtime (currently by the optimizer) into a different expression for evaluation.
An expression that gets replaced at runtime (currently by the optimizer) into a different expression for evaluation. This is mainly used to provide compatibility with other databases. For example, we use this to support "nvl" by replacing it with "coalesce".
trait RuntimeReplaceableAggregate extends Expression with RuntimeReplaceable
An add-on of AggregateFunction.
An add-on of AggregateFunction. This gets rewritten (currently by the optimizer) into a different aggregate expression for evaluation. This is mainly used to provide compatibility with other databases. For example, we use this to support every, any/some aggregates by rewriting them with Min and Max respectively.
case class SQLKeywords() extends LeafExpression with Generator with CodegenFallback with Product with Serializable

Annotations
@ExpressionDescription()
case class ScalaUDF(function: AnyRef, dataType: DataType, children: Seq[Expression], inputEncoders: Seq[Option[ExpressionEncoder[_]]] = Nil, outputEncoder: Option[ExpressionEncoder[_]] = None, udfName: Option[String] = None, nullable: Boolean = true, udfDeterministic: Boolean = true) extends Expression with NonSQLExpression with UserDefinedExpression with Product with Serializable
User-defined function.
User-defined function.
function
The user defined scala function to run. Note that if you use primitive parameters, you are not able to check if it is null or not, and the UDF will return null for you if the primitive input is null. Use boxed type or Option if you wanna do the null-handling yourself.
dataType
Return type of function.
children
The input expressions of this UDF.
inputEncoders
ExpressionEncoder for each input parameters. For a input parameter which serialized as struct will use encoder instead of CatalystTypeConverters to convert internal value to Scala value.
outputEncoder
ExpressionEncoder for the return type of function. It's only defined when this is a typed Scala UDF.
udfName
The user-specified name of this UDF.
nullable
True if the UDF can return null value.
udfDeterministic
True if the UDF is deterministic. Deterministic UDF returns same result each time it is invoked with a particular input.
case class ScalarSubquery(plan: LogicalPlan, outerAttrs: Seq[Expression] = Seq.empty, exprId: ExprId = NamedExpression.newExprId, joinCond: Seq[Expression] = Seq.empty, hint: Option[HintInfo] = None, mayHaveCountBug: Option[Boolean] = None) extends SubqueryExpression with Unevaluable with Product with Serializable
A subquery that will return only one row and one column.
A subquery that will return only one row and one column. This will be converted into a physical scalar subquery during planning.
Note: exprId is used to have a unique name in explain string output.
mayHaveCountBug is whether it's possible for the subquery to evaluate to non-null on empty input (zero tuples). It is false if the subquery has a GROUP BY clause, because in that case the subquery yields no row at all on empty input to the GROUP BY, which evaluates to NULL. It is set in PullupCorrelatedPredicates to true/false, before it is set its value is None. See constructLeftJoins in RewriteCorrelatedScalarSubquery for more details.
case class SchemaOfCsv(child: Expression, options: Map[String, String]) extends UnaryExpression with CodegenFallback with QueryErrorsBase with Product with Serializable
A function infers schema of CSV string.
A function infers schema of CSV string.

Annotations
@ExpressionDescription()
case class SchemaOfJson(child: Expression, options: Map[String, String]) extends UnaryExpression with CodegenFallback with QueryErrorsBase with Product with Serializable
A function infers schema of JSON string.
A function infers schema of JSON string.

Annotations
@ExpressionDescription()
case class Sec(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Second(child: Expression, timeZoneId: Option[String] = None) extends UnaryExpression with GetTimeField with Product with Serializable

Annotations
@ExpressionDescription()
case class SecondWithFraction(child: Expression, timeZoneId: Option[String] = None) extends UnaryExpression with GetTimeField with Product with Serializable
case class SecondsToTimestamp(child: Expression) extends UnaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class Sentences(str: Expression, language: Expression = Literal(""), country: Expression = Literal("")) extends TernaryExpression with ImplicitCastInputTypes with CodegenFallback with Product with Serializable
Splits a string into arrays of sentences, where each sentence is an array of words.
Splits a string into arrays of sentences, where each sentence is an array of words. The 'lang' and 'country' arguments are optional, and if omitted, the default locale is used.

Annotations
@ExpressionDescription()
abstract class SeptenaryExpression extends Expression
An expression with six inputs + 7th optional input and one output.
An expression with six inputs + 7th optional input and one output. The output is by default evaluated to null if any input is evaluated to null.
case class Sequence(start: Expression, stop: Expression, stepOpt: Option[Expression], timeZoneId: Option[String] = None) extends Expression with TimeZoneAwareExpression with QueryErrorsBase with Product with Serializable

Annotations
@ExpressionDescription()
case class SessionWindow(timeColumn: Expression, gapDuration: Expression) extends Expression with ImplicitCastInputTypes with Unevaluable with NonSQLExpression with Product with Serializable
Represent the session window.
Represent the session window.
timeColumn
the start time of session window
gapDuration
the duration of session gap. For static gap duration, meaning the session will close if there is no new element appeared within "the last element in session + gap". Besides a static gap duration value, users can also provide an expression to specify gap duration dynamically based on the input row. With dynamic gap duration, the closing of a session window does not depend on the latest input anymore. A session window's range is the union of all events' ranges which are determined by event start time and evaluated gap duration during the query execution. Note that the rows with negative or zero gap duration will be filtered out from the aggregation.

Annotations
@ExpressionDescription()
case class Sha1(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
A function that calculates a sha1 hash value and returns it as a hex string For input of type BinaryType or StringType
A function that calculates a sha1 hash value and returns it as a hex string For input of type BinaryType or StringType

Annotations
@ExpressionDescription()
case class Sha2(left: Expression, right: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable with Product
A function that calculates the SHA-2 family of functions (SHA-224, SHA-256, SHA-384, and SHA-512) and returns it as a hex string.
A function that calculates the SHA-2 family of functions (SHA-224, SHA-256, SHA-384, and SHA-512) and returns it as a hex string. The first argument is the string or binary to be hashed. The second argument indicates the desired bit length of the result, which must have a value of 224, 256, 384, 512, or 0 (which is equivalent to 256). SHA-224 is supported starting from Java 8. If asking for an unsupported SHA function, the return value is NULL. If either argument is NULL or the hash length is not one of the permitted values, the return value is NULL.

Annotations
@ExpressionDescription()
case class ShiftLeft(left: Expression, right: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Bitwise left shift.
Bitwise left shift.
left
the base number to shift.
right
number of bits to left shift.

Annotations
@ExpressionDescription()
case class ShiftRight(left: Expression, right: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Bitwise (signed) right shift.
Bitwise (signed) right shift.
left
the base number to shift.
right
number of bits to right shift.

Annotations
@ExpressionDescription()
case class ShiftRightUnsigned(left: Expression, right: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Bitwise unsigned right shift, for integer and long data type.
Bitwise unsigned right shift, for integer and long data type.
left
the base number.
right
the number of bits to right shift.

Annotations
@ExpressionDescription()
case class Shuffle(child: Expression, randomSeed: Option[Long] = None) extends UnaryExpression with ExpectsInputTypes with Nondeterministic with ExpressionWithRandomSeed with Product with Serializable
Returns a random permutation of the given array.
Returns a random permutation of the given array.

Annotations
@ExpressionDescription()
case class Signum(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
trait SimpleHigherOrderFunction extends Expression with HigherOrderFunction with BinaryLike[Expression]
Trait for functions having as input one argument and one function.
case class Sin(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Sinh(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Size(child: Expression, legacySizeOfNull: Boolean) extends UnaryExpression with ExpectsInputTypes with Product with Serializable
Given an array or map, returns total number of elements in it.
Given an array or map, returns total number of elements in it.

Annotations
@ExpressionDescription()
trait SizeBasedWindowFunction extends AggregateWindowFunction
A SizeBasedWindowFunction needs the size of the current window for its calculation.
case class Slice(x: Expression, start: Expression, length: Expression) extends TernaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Slices an array according to the requested start index and length
Slices an array according to the requested start index and length

Annotations
@ExpressionDescription()
case class SortArray(base: Expression, ascendingOrder: Expression) extends BinaryExpression with ArraySortLike with NullIntolerant with QueryErrorsBase with Product with Serializable
Sorts the input array in ascending / descending order according to the natural ordering of the array elements and returns it.
Sorts the input array in ascending / descending order according to the natural ordering of the array elements and returns it.

Annotations
@ExpressionDescription()
sealed abstract class SortDirection extends AnyRef
case class SortOrder(child: Expression, direction: SortDirection, nullOrdering: NullOrdering, sameOrderExpressions: Seq[Expression]) extends Expression with Unevaluable with Product with Serializable
An expression that can be used to sort a tuple.
An expression that can be used to sort a tuple. This class extends expression primarily so that transformations over expression will descend into its child. sameOrderExpressions is a set of expressions with the same sort order as the child. It is derived from equivalence relation in an operator, e.g. left/right keys of an inner sort merge join.
case class SortPrefix(child: SortOrder) extends UnaryExpression with Product with Serializable
An expression to generate a 64-bit long prefix used in sorting.
An expression to generate a 64-bit long prefix used in sorting. If the sort must operate over null keys as well, this.nullValue can be used in place of emitted null prefixes in the sort.
case class SoundEx(child: Expression) extends UnaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
A function that return Soundex code of the given string expression.
A function that return Soundex code of the given string expression.

Annotations
@ExpressionDescription()
case class SparkPartitionID() extends LeafExpression with Nondeterministic with Product with Serializable
Expression that returns the current partition id.
Expression that returns the current partition id.

Annotations
@ExpressionDescription()
case class SparkVersion() extends LeafExpression with CodegenFallback with Product with Serializable

Annotations
@ExpressionDescription()
sealed trait SpecialFrameBoundary extends LeafExpression with Unevaluable
The trait used to represent special boundaries used in a window frame.
trait SpecializedGetters extends AnyRef
final class SpecializedGettersReader extends AnyRef
final class SpecificInternalRow extends InternalRow with BaseGenericInternalRow
A row type that holds an array specialized container objects, of type MutableValue, chosen based on the dataTypes of each column.
A row type that holds an array specialized container objects, of type MutableValue, chosen based on the dataTypes of each column. The intent is to decrease garbage when modifying the values of primitive columns.
case class SpecifiedWindowFrame(frameType: FrameType, lower: Expression, upper: Expression) extends Expression with WindowFrame with BinaryLike[Expression] with QueryErrorsBase with Product with Serializable
A specified Window Frame.
A specified Window Frame. The val lower/upper can be either a foldable Expression or a SpecialFrameBoundary.
case class SplitPart(str: Expression, delimiter: Expression, partNum: Expression) extends Expression with RuntimeReplaceable with ImplicitCastInputTypes with Product with Serializable
Splits a given string by a specified delimiter and returns the requested part.
Splits a given string by a specified delimiter and returns the requested part. If any input is null, returns null. If index is out of range of split parts, return empty string. If index is 0, throws an ArrayIndexOutOfBoundsException.

Annotations
@ExpressionDescription()
case class Sqrt(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Stack(children: Seq[Expression]) extends Expression with Generator with Product with Serializable
Separate v1, ..., vk into n rows.
Separate v1, ..., vk into n rows. Each row will have k/n columns. n must be constant.
```
SELECT stack(2, 1, 2, 3) ->
1      2
3      NULL
```
Annotations
@ExpressionDescription()
case class StartsWith(left: Expression, right: Expression) extends StringPredicate with Product with Serializable
trait String2StringExpression extends Expression with ImplicitCastInputTypes
trait String2TrimExpression extends Expression with ImplicitCastInputTypes
trait StringBinaryPredicateExpressionBuilderBase extends ExpressionBuilder
case class StringDecode(bin: Expression, charset: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Decodes the first argument into a String using the provided character set (one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16').
Decodes the first argument into a String using the provided character set (one of 'US-ASCII', 'ISO-8859-1', 'UTF-8', 'UTF-16BE', 'UTF-16LE', 'UTF-16'). If either argument is null, the result will also be null.

Annotations
@ExpressionDescription()
case class StringInstr(str: Expression, substr: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
A function that returns the position of the first occurrence of substr in the given string.
A function that returns the position of the first occurrence of substr in the given string. Returns null if either of the arguments are null and returns 0 if substr could not be found in str.
NOTE: that this is not zero based, but 1-based index. The first character in str has index 1.

Annotations
@ExpressionDescription()
case class StringLPad(str: Expression, len: Expression, pad: Expression) extends TernaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
case class StringLocate(substr: Expression, str: Expression, start: Expression) extends TernaryExpression with ImplicitCastInputTypes with Product with Serializable
A function that returns the position of the first occurrence of substr in given string after position pos.
A function that returns the position of the first occurrence of substr in given string after position pos.

Annotations
@ExpressionDescription()
abstract class StringPredicate extends BinaryExpression with Predicate with ImplicitCastInputTypes with NullIntolerant
A base trait for functions that compare two strings, returning a boolean.
case class StringRPad(str: Expression, len: Expression, pad: Expression = Literal(" ")) extends TernaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
abstract class StringRegexExpression extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Predicate
case class StringRepeat(str: Expression, times: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Returns the string which repeat the given string value n times.
Returns the string which repeat the given string value n times.

Annotations
@ExpressionDescription()
case class StringReplace(srcExpr: Expression, searchExpr: Expression, replaceExpr: Expression) extends TernaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Replace all occurrences with string.
Replace all occurrences with string.

Annotations
@ExpressionDescription()
case class StringSpace(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Returns a string consisting of n spaces.
Returns a string consisting of n spaces.

Annotations
@ExpressionDescription()
case class StringSplit(str: Expression, regex: Expression, limit: Expression) extends TernaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Splits str around matches of the given regex.
Splits str around matches of the given regex.

Annotations
@ExpressionDescription()
case class StringSplitSQL(str: Expression, delimiter: Expression) extends BinaryExpression with NullIntolerant with Product with Serializable
Splits a given string by a specified delimiter and return splits into a GenericArrayData.
Splits a given string by a specified delimiter and return splits into a GenericArrayData. This expression is different from split function as split takes regex expression as the pattern to split strings while this expression take delimiter (a string without carrying special meaning on its characters, thus is not treated as regex) to split strings.
case class StringToMap(text: Expression, pairDelim: Expression, keyValueDelim: Expression) extends TernaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
Creates a map after splitting the input text into key/value pairs using delimiters
Creates a map after splitting the input text into key/value pairs using delimiters

Annotations
@ExpressionDescription()
case class StringTranslate(srcExpr: Expression, matchingExpr: Expression, replaceExpr: Expression) extends TernaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
A function translate any character in the srcExpr by a character in replaceExpr.
A function translate any character in the srcExpr by a character in replaceExpr. The characters in replaceExpr is corresponding to the characters in matchingExpr. The translate will happen when any character in the string matching with the character in the matchingExpr.

Annotations
@ExpressionDescription()
case class StringTrim(srcStr: Expression, trimStr: Option[Expression] = None) extends Expression with String2TrimExpression with Product with Serializable
A function that takes a character string, removes the leading and trailing characters matching with any character in the trim string, returns the new string.
A function that takes a character string, removes the leading and trailing characters matching with any character in the trim string, returns the new string. If BOTH and trimStr keywords are not specified, it defaults to remove space character from both ends. The trim function will have one argument, which contains the source string. If BOTH and trimStr keywords are specified, it trims the characters from both ends, and the trim function will have two arguments, the first argument contains trimStr, the second argument contains the source string. trimStr: A character string to be trimmed from the source string, if it has multiple characters, the function searches for each character in the source string, removes the characters from the source string until it encounters the first non-match character. BOTH: removes any character from both ends of the source string that matches characters in the trim string.

Annotations
@ExpressionDescription()
case class StringTrimBoth(srcStr: Expression, trimStr: Option[Expression], replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable
A function that takes a character string, removes the leading and trailing characters matching with any character in the trim string, returns the new string.
A function that takes a character string, removes the leading and trailing characters matching with any character in the trim string, returns the new string. trimStr: A character string to be trimmed from the source string, if it has multiple characters, the function searches for each character in the source string, removes the characters from the source string until it encounters the first non-match character.

Annotations
@ExpressionDescription()
case class StringTrimLeft(srcStr: Expression, trimStr: Option[Expression] = None) extends Expression with String2TrimExpression with Product with Serializable
A function that trims the characters from left end for a given string.
A function that trims the characters from left end for a given string. If LEADING and trimStr keywords are not specified, it defaults to remove space character from the left end. The ltrim function will have one argument, which contains the source string. If LEADING and trimStr keywords are not specified, it trims the characters from left end. The ltrim function will have two arguments, the first argument contains trimStr, the second argument contains the source string. trimStr: the function removes any character from the left end of the source string which matches with the characters from trimStr, it stops at the first non-match character. LEADING: removes any character from the left end of the source string that matches characters in the trim string.

Annotations
@ExpressionDescription()
case class StringTrimRight(srcStr: Expression, trimStr: Option[Expression] = None) extends Expression with String2TrimExpression with Product with Serializable
A function that trims the characters from right end for a given string.
A function that trims the characters from right end for a given string. If TRAILING and trimStr keywords are not specified, it defaults to remove space character from the right end. The rtrim function will have one argument, which contains the source string. If TRAILING and trimStr keywords are specified, it trims the characters from right end. The rtrim function will have two arguments, the first argument contains trimStr, the second argument contains the source string. trimStr: the function removes any character from the right end of source string which matches with the characters from trimStr, it stops at the first non-match character. TRAILING: removes any character from the right end of the source string that matches characters in the trim string.

Annotations
@ExpressionDescription()
trait StructFieldsOperation extends Expression with Unevaluable
Represents an operation to be applied to the fields of a struct.
case class StructsToCsv(options: Map[String, String], child: Expression, timeZoneId: Option[String] = None) extends UnaryExpression with TimeZoneAwareExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
Converts a StructType to a CSV output string.
Converts a StructType to a CSV output string.

Annotations
@ExpressionDescription()
case class StructsToJson(options: Map[String, String], child: Expression, timeZoneId: Option[String] = None) extends UnaryExpression with TimeZoneAwareExpression with CodegenFallback with ExpectsInputTypes with NullIntolerant with QueryErrorsBase with Product with Serializable
Converts a StructType, ArrayType or MapType to a JSON output string.
Converts a StructType, ArrayType or MapType to a JSON output string.

Annotations
@ExpressionDescription()
class SubExprEvaluationRuntime extends AnyRef
This class helps subexpression elimination for interpreted evaluation such as InterpretedUnsafeProjection.
This class helps subexpression elimination for interpreted evaluation such as InterpretedUnsafeProjection. It maintains an evaluation cache. This class wraps ExpressionProxy around given expressions. The ExpressionProxy intercepts expression evaluation and loads from the cache first.
abstract class SubqueryExpression extends PlanExpression[LogicalPlan]
A base interface for expressions that contain a LogicalPlan.
case class Substring(str: Expression, pos: Expression, len: Expression) extends TernaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
A function that takes a substring of its first argument starting at a given position.
A function that takes a substring of its first argument starting at a given position. Defined for String and Binary types.
NOTE: that this is not zero based, but 1-based index. The first character in str has index 1.

Annotations
@ExpressionDescription()
case class SubstringIndex(strExpr: Expression, delimExpr: Expression, countExpr: Expression) extends TernaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Returns the substring from string str before count occurrences of the delimiter delim.
Returns the substring from string str before count occurrences of the delimiter delim. If count is positive, everything the left of the final delimiter (counting from left) is returned. If count is negative, every to the right of the final delimiter (counting from the right) is returned. substring_index performs a case-sensitive match when searching for delim.

Annotations
@ExpressionDescription()
case class Subtract(left: Expression, right: Expression, evalMode: EvalMode.Value = EvalMode.fromSQLConf(SQLConf.get)) extends BinaryArithmetic with Product with Serializable

Annotations
@ExpressionDescription()
case class SubtractDates(left: Expression, right: Expression, legacyInterval: Boolean) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Returns the interval from the left date (inclusive) to the right date (exclusive).
Returns the interval from the left date (inclusive) to the right date (exclusive).
- When the SQL config spark.sql.legacy.interval.enabled is true, it returns CalendarIntervalType in which the microseconds field is set to 0 and the months and days fields are initialized to the difference between the given dates.
- Otherwise the expression returns DayTimeIntervalType with the difference in days between the given dates.
case class SubtractTimestamps(left: Expression, right: Expression, legacyInterval: Boolean, timeZoneId: Option[String] = None) extends BinaryExpression with TimeZoneAwareExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
Returns the interval from right to left timestamps.
Returns the interval from right to left timestamps.
- When the SQL config spark.sql.legacy.interval.enabled is true, it returns CalendarIntervalType in which the months and dayfield is set to 0 and themicrosecondsfield is initialized to the microsecond difference between the given timestamps.
- Otherwise the expression returns DayTimeIntervalType with the difference in microseconds between given timestamps.
trait SupportQueryContext extends Expression with Serializable
An expression with SQL query context.
An expression with SQL query context. The context string can be serialized from the Driver to executors. It will also be kept after rule transforms.
trait TaggingExpression extends UnaryExpression
case class Tan(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class Tanh(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
abstract class TernaryExpression extends Expression with TernaryLike[Expression]
An expression with three inputs and one output.
An expression with three inputs and one output. The output is by default evaluated to null if any input is evaluated to null.
case class TimeAdd(start: Expression, interval: Expression, timeZoneId: Option[String] = None) extends BinaryExpression with TimeZoneAwareExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
Adds an interval to timestamp.
case class TimeWindow(timeColumn: Expression, windowDuration: Long, slideDuration: Long, startTime: Long) extends UnaryExpression with ImplicitCastInputTypes with Unevaluable with NonSQLExpression with QueryErrorsBase with Product with Serializable

Annotations
@ExpressionDescription()
trait TimeZoneAwareExpression extends Expression
Common base class for time zone aware expressions.
case class TimestampAdd(unit: String, quantity: Expression, timestamp: Expression, timeZoneId: Option[String] = None) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with TimeZoneAwareExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class TimestampAddYMInterval(timestamp: Expression, interval: Expression, timeZoneId: Option[String] = None) extends BinaryExpression with TimeZoneAwareExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable
case class TimestampDiff(unit: String, startTimestamp: Expression, endTimestamp: Expression, timeZoneId: Option[String] = None) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with TimeZoneAwareExpression with Product with Serializable

Annotations
@ExpressionDescription()
trait TimestampFormatterHelper extends Expression with TimeZoneAwareExpression
abstract class TimestampToLongBase extends UnaryExpression with ExpectsInputTypes with NullIntolerant
case class ToBinary(expr: Expression, format: Option[Expression], nullOnInvalidFormat: Boolean = false) extends Expression with RuntimeReplaceable with ImplicitCastInputTypes with Product with Serializable
Converts the input expression to a binary value based on the supplied format.
Converts the input expression to a binary value based on the supplied format.

Annotations
@ExpressionDescription()
case class ToCharacter(left: Expression, right: Expression) extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
case class ToDegrees(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class ToNumber(left: Expression, right: Expression) extends ToNumberBase with Product with Serializable
A function that converts strings to decimal values, returning an exception if the input string fails to match the format string.
A function that converts strings to decimal values, returning an exception if the input string fails to match the format string.

Annotations
@ExpressionDescription()
abstract class ToNumberBase extends BinaryExpression with Serializable with ImplicitCastInputTypes with NullIntolerant
case class ToPrettyString(child: Expression, timeZoneId: Option[String] = None) extends UnaryExpression with TimeZoneAwareExpression with ToStringBase with Product with Serializable
An internal expressions which is used to generate pretty string for all kinds of values.
An internal expressions which is used to generate pretty string for all kinds of values. It has several differences with casting value to string:
- It prints null values (either from column or struct field) as "NULL".
- It prints binary values (either from column or struct field) using the hex format.
case class ToRadians(child: Expression) extends UnaryMathExpression with Product with Serializable

Annotations
@ExpressionDescription()
trait ToStringBase extends AnyRef
abstract class ToTimestamp extends BinaryExpression with TimestampFormatterHelper with ExpectsInputTypes
case class ToUTCTimestamp(left: Expression, right: Expression) extends BinaryExpression with UTCTimestamp with Product with Serializable
This is a common function for databases supporting TIMESTAMP WITHOUT TIMEZONE.
This is a common function for databases supporting TIMESTAMP WITHOUT TIMEZONE. This function takes a timestamp which is timezone-agnostic, and interprets it as a timestamp in the given timezone, and renders that timestamp as a timestamp in UTC.
However, timestamp in Spark represents number of microseconds from the Unix epoch, which is not timezone-agnostic. So in Spark this function just shift the timestamp value from the given timezone to UTC timezone.
This function may return confusing result if the input is a string with timezone, e.g. '2018-03-13T06:18:23+00:00'. The reason is that, Spark firstly cast the string to timestamp according to the timezone in the string, and finally display the result by converting the timestamp to string according to the session local timezone.

Annotations
@ExpressionDescription()
case class ToUnixTimestamp(timeExp: Expression, format: Expression, timeZoneId: Option[String] = None, failOnError: Boolean = SQLConf.get.ansiEnabled) extends UnixTime with Product with Serializable
Converts time string with given pattern.
Converts time string with given pattern. Deterministic version of UnixTimestamp, must have at least one parameter.

Annotations
@ExpressionDescription()
case class TransformExpression(function: BoundFunction, children: Seq[Expression], numBucketsOpt: Option[Int] = None) extends Expression with Unevaluable with Product with Serializable
Represents a partition transform expression, for instance, bucket, days, years, etc.
Represents a partition transform expression, for instance, bucket, days, years, etc.
function
the transform function itself. Spark will use it to decide whether two partition transform expressions are compatible.
numBucketsOpt
the number of buckets if the transform is bucket. Unset otherwise.
case class TransformKeys(argument: Expression, function: Expression) extends Expression with MapBasedSimpleHigherOrderFunction with CodegenFallback with Product with Serializable
Transform Keys for every entry of the map by applying the transform_keys function.
Transform Keys for every entry of the map by applying the transform_keys function. Returns map with transformed key entries

Annotations
@ExpressionDescription()
case class TransformValues(argument: Expression, function: Expression) extends Expression with MapBasedSimpleHigherOrderFunction with CodegenFallback with Product with Serializable
Returns a map that applies the function to each value of the map.
Returns a map that applies the function to each value of the map.

Annotations
@ExpressionDescription()
case class TruncDate(date: Expression, format: Expression) extends BinaryExpression with TruncInstant with Product with Serializable
Returns date truncated to the unit specified by the format.
Returns date truncated to the unit specified by the format.

Annotations
@ExpressionDescription()
trait TruncInstant extends BinaryExpression with ImplicitCastInputTypes
case class TruncTimestamp(format: Expression, timestamp: Expression, timeZoneId: Option[String] = None) extends BinaryExpression with TruncInstant with TimeZoneAwareExpression with Product with Serializable
Returns timestamp truncated to the unit specified by the format.
Returns timestamp truncated to the unit specified by the format.

Annotations
@ExpressionDescription()
case class TryAdd(left: Expression, right: Expression, replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable

Annotations
@ExpressionDescription()
case class TryAesDecrypt(input: Expression, key: Expression, mode: Expression, padding: Expression, aad: Expression, replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable

Annotations
@ExpressionDescription()
case class TryDivide(left: Expression, right: Expression, replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable

Annotations
@ExpressionDescription()
case class TryElementAt(left: Expression, right: Expression, replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable
Returns the value of index right in Array left or the value for key right in Map left.
Returns the value of index right in Array left or the value for key right in Map left. The function is identical to the function element_at, except that it returns NULL result instead of throwing an exception on array's index out of bound or map's key not found when spark.sql.ansi.enabled is true.

Annotations
@ExpressionDescription()
case class TryEval(child: Expression) extends UnaryExpression with NullIntolerant with Product with Serializable
case class TryMultiply(left: Expression, right: Expression, replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable

Annotations
@ExpressionDescription()
case class TryReflect(params: Seq[Expression], replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable

Annotations
@ExpressionDescription()
case class TrySubtract(left: Expression, right: Expression, replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable

Annotations
@ExpressionDescription()
case class TryToBinary(expr: Expression, format: Option[Expression], replacement: Expression) extends Expression with RuntimeReplaceable with InheritAnalysisRules with Product with Serializable

Annotations
@ExpressionDescription()
case class TryToNumber(left: Expression, right: Expression) extends ToNumberBase with Product with Serializable
A function that converts strings to decimal values, returning NULL if the input string fails to match the format string.
A function that converts strings to decimal values, returning NULL if the input string fails to match the format string.

Annotations
@ExpressionDescription()
case class TypeOf(child: Expression) extends UnaryExpression with Product with Serializable

Annotations
@ExpressionDescription()
sealed trait UTCTimestamp extends BinaryExpression with ImplicitCastInputTypes with NullIntolerant
case class UnBase64(child: Expression, failOnError: Boolean = false) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Converts the argument from a base 64 string to BINARY.
Converts the argument from a base 64 string to BINARY.

Annotations
@ExpressionDescription()
abstract class UnaryExpression extends Expression with UnaryLike[Expression]
An expression with one input and one output.
An expression with one input and one output. The output is by default evaluated to null if the input is evaluated to null.
abstract class UnaryLogExpression extends UnaryMathExpression
abstract class UnaryMathExpression extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Serializable
A unary expression specifically for math functions.
A unary expression specifically for math functions. Math Functions expect a specific type of input format, therefore these functions extend ExpectsInputTypes.
case class UnaryMinus(child: Expression, failOnError: Boolean = SQLConf.get.ansiEnabled) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class UnaryPositive(child: Expression) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
trait Unevaluable extends Expression
An expression that cannot be evaluated.
An expression that cannot be evaluated. These expressions don't live past analysis or optimization time (e.g. Star) and should not be evaluated during query planning and execution.
abstract class UnevaluableAggregateFunc extends AggregateFunction
abstract class UnevaluableGenerator extends Expression with Generator
case class Unhex(child: Expression, failOnError: Boolean = false) extends UnaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Performs the inverse operation of HEX.
Performs the inverse operation of HEX. Resulting characters are returned as a byte array.

Annotations
@ExpressionDescription()
case class UnixDate(child: Expression) extends UnaryExpression with ExpectsInputTypes with NullIntolerant with Product with Serializable

Annotations
@ExpressionDescription()
case class UnixMicros(child: Expression) extends TimestampToLongBase with Product with Serializable

Annotations
@ExpressionDescription()
case class UnixMillis(child: Expression) extends TimestampToLongBase with Product with Serializable

Annotations
@ExpressionDescription()
case class UnixSeconds(child: Expression) extends TimestampToLongBase with Product with Serializable

Annotations
@ExpressionDescription()
abstract class UnixTime extends ToTimestamp
case class UnixTimestamp(timeExp: Expression, format: Expression, timeZoneId: Option[String] = None, failOnError: Boolean = SQLConf.get.ansiEnabled) extends UnixTime with Product with Serializable
Converts time string with given pattern to Unix time stamp (in seconds), returns null if fail.
Converts time string with given pattern to Unix time stamp (in seconds), returns null if fail. See Datetime Patterns. Note that hive Language Manual says it returns 0 if fail, but in fact it returns null. If the second parameter is missing, use "yyyy-MM-dd HH:mm:ss". If no parameters provided, the first parameter will be current_timestamp. If the first parameter is a Date or Timestamp instead of String, we will ignore the second parameter.

Annotations
@ExpressionDescription()
case class UnresolvedNamedLambdaVariable(nameParts: Seq[String]) extends LeafExpression with NamedExpression with Unevaluable with Product with Serializable
A placeholder of lambda variables to prevent unexpected resolution of LambdaFunction.
case class UnresolvedPolymorphicPythonUDTF(name: String, func: PythonFunction, children: Seq[Expression], evalType: Int, udfDeterministic: Boolean, resolveElementMetadata: (PythonFunction, Seq[Expression]) ⇒ PythonUDTFAnalyzeResult, resultId: ExprId = NamedExpression.newExprId, pythonUDTFPartitionColumnIndexes: Option[PythonUDTFPartitionColumnIndexes] = None) extends UnevaluableGenerator with PythonFuncExpression with Product with Serializable
A placeholder of a polymorphic Python table-valued function.
case class UnresolvedWindowExpression(child: Expression, windowSpec: WindowSpecReference) extends UnaryExpression with Unevaluable with Product with Serializable
final class UnsafeArrayData extends ArrayData with Externalizable with KryoSerializable
An Unsafe implementation of Array which is backed by raw memory instead of Java objects.
An Unsafe implementation of Array which is backed by raw memory instead of Java objects.
Each array has four parts: [numElements][null bits][values or offset&length][variable length portion]
The numElements is 8 bytes storing the number of elements of this array.
In the null bits region, we store 1 bit per element, represents whether an element is null Its total size is ceil(numElements / 8) bytes, and it is aligned to 8-byte boundaries.
In the values or offset&length region, we store the content of elements. For fields that hold fixed-length primitive types, such as long, double, or int, we store the value directly in the field. The whole fixed-length portion (even for byte) is aligned to 8-byte boundaries. For fields with non-primitive or variable-length values, we store a relative offset (w.r.t. the base address of the array) that points to the beginning of the variable-length field and length (they are combined into a long). For variable length portion, each is aligned to 8-byte boundaries.
Instances of UnsafeArrayData act as pointers to row data stored in this format.
final class UnsafeMapData extends MapData with Externalizable with KryoSerializable
An Unsafe implementation of Map which is backed by raw memory instead of Java objects.
An Unsafe implementation of Map which is backed by raw memory instead of Java objects.
Currently we just use 2 UnsafeArrayData to represent UnsafeMapData, with extra 8 bytes at head to indicate the number of bytes of the unsafe key array. [unsafe key array numBytes] [unsafe key array] [unsafe value array]
Note that, user is responsible to guarantee that the key array does not have duplicated elements, otherwise the behavior is undefined.
abstract class UnsafeProjection extends Projection
A projection that returns UnsafeRow.
A projection that returns UnsafeRow.
CAUTION: the returned projection object should *not* be assumed to be thread-safe.
final class UnsafeRow extends InternalRow with Externalizable with KryoSerializable
An Unsafe implementation of Row which is backed by raw memory instead of Java objects.
An Unsafe implementation of Row which is backed by raw memory instead of Java objects.
Each tuple has three parts: [null-tracking bit set] [values] [variable length portion]
The null-tracking bit set is aligned to 8-byte word boundaries. It stores one bit per field.
In the values region, we store one 8-byte word per field. For fields that hold fixed-length primitive types, such as long, double, or int, we store the value directly in the word. For fields with non-primitive or variable-length values, we store a relative offset (w.r.t. the base address of the row) that points to the beginning of the variable-length field, and length (they are combined into a long).
Instances of UnsafeRow act as pointers to row data stored in this format.
case class UnscaledValue(child: Expression) extends UnaryExpression with NullIntolerant with Product with Serializable
Return the unscaled Long value of a Decimal, assuming it fits in a Long.
Return the unscaled Long value of a Decimal, assuming it fits in a Long. Note: this expression is internal and created only by the optimizer, we don't need to do type check for it.
case class UnwrapUDT(child: Expression) extends UnaryExpression with NonSQLExpression with Product with Serializable
Unwrap UDT data type column into its underlying type.
case class UpCast(child: Expression, target: AbstractDataType, walkedTypePath: Seq[String] = Nil) extends UnaryExpression with Unevaluable with Product with Serializable
Cast the child expression to the target data type, but will throw error if the cast might truncate, e.g.
Cast the child expression to the target data type, but will throw error if the cast might truncate, e.g. long -> int, timestamp -> data.
Note: target is AbstractDataType, so that we can put object DecimalType, which means we accept DecimalType with any valid precision/scale.
case class UpdateFields(structExpr: Expression, fieldOps: Seq[StructFieldsOperation]) extends Expression with Unevaluable with Product with Serializable
Updates fields in a struct.
case class Upper(child: Expression) extends UnaryExpression with String2StringExpression with NullIntolerant with Product with Serializable
A function that converts the characters of a string to uppercase.
A function that converts the characters of a string to uppercase.

Annotations
@ExpressionDescription()
case class UrlDecode(child: Expression) extends Expression with RuntimeReplaceable with UnaryLike[Expression] with ImplicitCastInputTypes with Product with Serializable

Annotations
@ExpressionDescription()
case class UrlEncode(child: Expression) extends Expression with RuntimeReplaceable with UnaryLike[Expression] with ImplicitCastInputTypes with Product with Serializable

Annotations
@ExpressionDescription()
trait UserDefinedExpression extends AnyRef
Common base trait for user-defined functions, including UDF/UDAF/UDTF of different languages and Hive function wrappers.
case class UserDefinedGenerator(elementSchema: StructType, function: (Row) ⇒ TraversableOnce[InternalRow], children: Seq[Expression]) extends Expression with Generator with CodegenFallback with Product with Serializable
A generator that produces its output using the provided lambda function.
case class Uuid(randomSeed: Option[Long] = None) extends LeafExpression with Nondeterministic with ExpressionWithRandomSeed with Product with Serializable

Annotations
@ExpressionDescription()
final class VariableLengthRowBasedKeyValueBatch extends RowBasedKeyValueBatch
An implementation of RowBasedKeyValueBatch in which key-value records have variable lengths.
An implementation of RowBasedKeyValueBatch in which key-value records have variable lengths.
The format for each record looks like this (in case of uaoSize = 4): [4 bytes total size = (klen + vlen + 4)] [4 bytes key size = klen] [UnsafeRow for key of length klen] [UnsafeRow for Value of length vlen] [8 bytes pointer to next] Thus, record length = 4 + 4 + klen + vlen + 8
case class VariableReference(originalNameParts: Seq[String], catalog: CatalogPlugin, identifier: Identifier, varDef: VariableDefinition, canFold: Boolean = true) extends LeafExpression with Product with Serializable
A resolved SQL variable, which contains all the information of it.
case class WeekDay(child: Expression) extends UnaryExpression with GetDateField with Product with Serializable

Annotations
@ExpressionDescription()
case class WeekOfYear(child: Expression) extends UnaryExpression with GetDateField with Product with Serializable

Annotations
@ExpressionDescription()
case class WidthBucket(value: Expression, minValue: Expression, maxValue: Expression, numBucket: Expression) extends QuaternaryExpression with ImplicitCastInputTypes with NullIntolerant with Product with Serializable
Returns the bucket number into which the value of this expression would fall after being evaluated.
Returns the bucket number into which the value of this expression would fall after being evaluated. Note that input arguments must follow conditions listed below; otherwise, the method will return null.
- numBucket must be greater than zero and be less than Long.MaxValue
- value, min, and max cannot be NaN
- min bound cannot equal max
- min and max must be finite
Note: If minValue > maxValue, a return value is as follows; if value > minValue, it returns 0. if value <= maxValue, it returns numBucket + 1. otherwise, it returns (numBucket * (minValue - value) / (minValue - maxValue)) + 1
value
is the expression to compute a bucket number in the histogram
minValue
is the minimum value of the histogram
maxValue
is the maximum value of the histogram
numBucket
is the number of buckets

Annotations
@ExpressionDescription()
case class WindowExpression(windowFunction: Expression, windowSpec: WindowSpecDefinition) extends Expression with Unevaluable with BinaryLike[Expression] with Product with Serializable
sealed trait WindowFrame extends Expression with Unevaluable
Represents a window frame.
trait WindowFunction extends Expression
A window function is a function that can only be evaluated in the context of a window operator.
sealed trait WindowFunctionType extends AnyRef
Case objects that describe whether a window function is a SQL window function or a Python user-defined window function.
sealed trait WindowSpec extends AnyRef
The trait of the Window Specification (specified in the OVER clause or WINDOW clause) for Window Functions.
case class WindowSpecDefinition(partitionSpec: Seq[Expression], orderSpec: Seq[SortOrder], frameSpecification: WindowFrame) extends Expression with WindowSpec with Unevaluable with DataTypeErrorsBase with Product with Serializable
The specification for a window function.
The specification for a window function.
partitionSpec
It defines the way that input rows are partitioned.
orderSpec
It defines the ordering of rows in a partition.
frameSpecification
It defines the window frame in a partition.
case class WindowSpecReference(name: String) extends WindowSpec with Product with Serializable
A Window specification reference that refers to the WindowSpecDefinition defined under the name name.
case class WindowTime(windowColumn: Expression) extends UnaryExpression with ImplicitCastInputTypes with Unevaluable with NonSQLExpression with Product with Serializable

Annotations
@ExpressionDescription()
case class WithField(name: String, valExpr: Expression) extends Expression with StructFieldsOperation with UnaryLike[Expression] with Product with Serializable
Add or replace a field by name.
Add or replace a field by name.
We extend Unevaluable here to ensure that UpdateFields can include it as part of its children, and thereby enable the analyzer to resolve and transform valExpr as necessary.
final class XXH64 extends AnyRef
xxHash64.
xxHash64. A high quality and fast 64 bit hash code by Yann Colet and Mathias Westerdahl. The class below is modelled like its Murmur3_x86_32 cousin.
This was largely based on the following (original) C and Java implementations: https://github.com/Cyan4973/xxHash/blob/master/xxhash.c https://github.com/OpenHFT/Zero-Allocation-Hashing/blob/master/src/main/java/net/openhft/hashing/XxHash_r39.java https://github.com/airlift/slice/blob/master/src/main/java/io/airlift/slice/XxHash64.java
case class XxHash64(children: Seq[Expression], seed: Long) extends HashExpression[Long] with Product with Serializable
A xxHash64 64-bit hash expression.
A xxHash64 64-bit hash expression.

Annotations
@ExpressionDescription()
case class Year(child: Expression) extends UnaryExpression with GetDateField with Product with Serializable

Annotations
@ExpressionDescription()
case class YearOfWeek(child: Expression) extends UnaryExpression with GetDateField with Product with Serializable
case class Years(child: Expression) extends PartitionTransformExpression with Product with Serializable
Expression for the v2 partition transform years.
case class ZipWith(left: Expression, right: Expression, function: Expression) extends Expression with HigherOrderFunction with CodegenFallback with TernaryLike[Expression] with Product with Serializable

Annotations
@ExpressionDescription()

Value Members

val EmptyRow: InternalRow
Used as input into expressions whose output does not depend on any input value.
object ArrayBinaryLike
object ArrayExists extends Serializable
object ArraySort extends Serializable
object ArraySortLike
object ArrayUnion extends Serializable
object ArraysZip extends Serializable
object Ascending extends SortDirection with Product with Serializable
object AssertTrue extends Serializable
object AttributeMap extends Serializable
Builds a map that is keyed by an Attribute's expression id.
Builds a map that is keyed by an Attribute's expression id. Using the expression id allows values to be looked up even when the attributes used differ cosmetically (i.e., the capitalization of the name, or the expected nullability).
object AttributeReferenceTreeBits
object AttributeSeq extends Serializable
object AttributeSet extends Serializable
object BaseGroupingSets
object BinaryArithmetic extends Serializable
object BinaryComparison
object BinaryExpression
object BinaryOperator
object BinaryPredicate extends Serializable
object BindReferences extends Logging
object BitwiseGetUtil
object BooleanLiteral
Extractor for retrieving Boolean literals.
object CallMethodViaReflection extends Serializable
object CaseKeyWhen
Case statements of the form "CASE a WHEN b THEN c [WHEN d THEN e]* [ELSE f] END".
Case statements of the form "CASE a WHEN b THEN c [WHEN d THEN e]* [ELSE f] END". When a = b, returns c; when a = d, returns e; else returns f.
object CaseWhen extends Serializable
Factory methods for CaseWhen.
object Cast extends QueryErrorsBase with Serializable
object CeilExpressionBuilder extends CeilFloorExpressionBuilderBase

Annotations
@ExpressionDescription()
object CodegenObjectFactoryMode extends Enumeration
Defines values for SQLConf config of fallback mode.
Defines values for SQLConf config of fallback mode. Use for test only.
object ContainsExpressionBuilder extends StringBinaryPredicateExpressionBuilderBase

Annotations
@ExpressionDescription()
object CreateArray extends Serializable
object CreateMap extends Serializable
object CreateStruct
Returns a Row containing the evaluation of all children expressions.
object Cube extends Serializable
object CurDateExpressionBuilder extends ExpressionBuilder

Annotations
@ExpressionDescription()
object CurrentRow extends LeafExpression with SpecialFrameBoundary with Product with Serializable
CURRENT ROW boundary.
object DatePart
object DatePartExpressionBuilder extends ExpressionBuilder

Annotations
@ExpressionDescription()
object DecimalLiteral
Extractor for and other utility methods for decimal literals.
object Decode extends Serializable
object Descending extends SortDirection with Product with Serializable
object DoubleLiteral
Extractor for retrieving Double literals.
object EndsWithExpressionBuilder extends StringBinaryPredicateExpressionBuilderBase

Annotations
@ExpressionDescription()
object Equality
An extractor that matches both standard 3VL equality and null-safe equality.
object EvalMode extends Enumeration
Expression evaluation modes.
Expression evaluation modes.
- LEGACY: the default evaluation mode, which is compliant to Hive SQL.
- ANSI: a evaluation mode which is compliant to ANSI SQL standard.
- TRY: a evaluation mode for try_* functions. It is identical to ANSI evaluation mode except for returning null result on errors.
object ExpectsInputTypes extends QueryErrorsBase
object ExplodeExpressionBuilder extends ExpressionBuilder

Annotations
@ExpressionDescription()
object ExplodeGeneratorBuilder extends ExplodeGeneratorBuilderBase

Annotations
@ExpressionDescription()
object ExplodeOuterGeneratorBuilder extends ExplodeGeneratorBuilderBase

Annotations
@ExpressionDescription()
object ExprId extends Serializable
object ExprUtils extends QueryErrorsBase
object ExpressionSet
object Extract extends Serializable
object ExtractIntervalPart extends Serializable
object ExtractValue
object Factorial extends Serializable
object FileSourceConstantMetadataAttribute
Matches the internal representation of a file source constant metadata attribute -- a member of the _metadata struct whose value is constant for a whole file (e.g.
Matches the internal representation of a file source constant metadata attribute -- a member of the _metadata struct whose value is constant for a whole file (e.g. file name). Values are usually appended to the output and not generated per row.
object FileSourceConstantMetadataStructField
A StructField that describes a file source constant metadata struct field -- a member of the _metadata struct whose value is constant for a whole file (e.g.
A StructField that describes a file source constant metadata struct field -- a member of the _metadata struct whose value is constant for a whole file (e.g. file name). Values are usually appended to the output and not generated per row.
object FileSourceGeneratedMetadataAttribute
Matches the internal representation of a file source generated metadata attribute -- a member of the metadata struct that maps to some internal column the scanner returns.
object FileSourceGeneratedMetadataStructField
A StructField that describes a file source generated metadata struct field -- a member of the metadata struct that maps to some internal column the scanner returns.
A StructField that describes a file source generated metadata struct field -- a member of the metadata struct that maps to some internal column the scanner returns.
- apply() wil create a file source generated metadata struct field - unapply() matches a file source generated metadata struct field, and returns its internal name
object FileSourceMetadataAttribute
The internal representation of the FileSourceMetadataAttribute, it sets metadata_col and file_source_metadata_col to true in AttributeReference's metadata.
The internal representation of the FileSourceMetadataAttribute, it sets metadata_col and file_source_metadata_col to true in AttributeReference's metadata. This is a super type of FileSourceConstantMetadataAttribute and FileSourceGeneratedMetadataAttribute.
- apply() will create a file source metadata attribute reference - unapply() will check if an attribute reference is any file source metadata attribute reference
object FloatLiteral
Extractor for retrieving Float literals.
object FloorExpressionBuilder extends CeilFloorExpressionBuilderBase

Annotations
@ExpressionDescription()
object GetStructFieldObject
A Scala extractor that extracts the child expression and struct field from a GetStructField.
A Scala extractor that extracts the child expression and struct field from a GetStructField. This is in contrast to the GetStructField case class extractor which returns the field ordinal instead of the field itself.
object GroupingAnalytics
object GroupingID extends Serializable
object GroupingSets extends Serializable
object Hex extends Serializable
object HiveHashFunction extends InterpretedHashFunction
object IdentityProjection extends Projection
An identity projection.
An identity projection. This returns the input row.
object IntegerLiteral
Extractor for retrieving Int literals.
object InterpretedMutableProjection
Helper functions for creating an InterpretedMutableProjection.
object InterpretedOrdering extends Serializable
object InterpretedSafeProjection
Helper functions for creating an InterpretedSafeProjection.
object InterpretedUnsafeProjection
Helper functions for creating an InterpretedUnsafeProjection.
object IsNotUnknown
object IsUnknown
IS UNKNOWN and IS NOT UNKNOWN are the same as IS NULL and IS NOT NULL, respectively, except that the input expression must be of a boolean type.
object LPadExpressionBuilder extends PadExpressionBuilderBase

Annotations
@ExpressionDescription()
object LambdaFunction extends Serializable
object Literal extends Serializable
object LiteralTreeBits
object LongLiteral
Extractor for retrieving Long literals.
object MakeDecimal extends Serializable
object MakeTimestampLTZExpressionBuilder extends ExpressionBuilder

Annotations
@ExpressionDescription()
object MakeTimestampNTZExpressionBuilder extends ExpressionBuilder

Annotations
@ExpressionDescription()
object Mask extends Serializable
object MaskExpressionBuilder extends ExpressionBuilder

Annotations
@ExpressionDescription()
object MetadataAttribute
The internal representation of the MetadataAttribute, it sets metadata_col to true in AttributeReference metadata - apply() will create a metadata attribute reference - unapply() will check if an attribute reference is the metadata attribute reference
object MetadataAttributeWithLogicalName
A MetadataAttribute that works even if the attribute was renamed to avoid a conflict with some column name in the schema.
A MetadataAttribute that works even if the attribute was renamed to avoid a conflict with some column name in the schema. See also LogicalPlan.getMetadataAttributeByName.
- apply() creates a logically-named attribute with the given physical name. - unapply() matches a MetadataAttribute and also returns its logical name.
object MetadataStructFieldWithLogicalName
object Murmur3HashFunction extends InterpretedHashFunction
object MutableProjection extends CodeGeneratorWithInterpretedFallback[Seq[Expression], MutableProjection]
The factory object for MutableProjection.
object NamePlaceholder extends LeafExpression with Unevaluable with Product with Serializable
An expression representing a not yet available attribute name.
An expression representing a not yet available attribute name. This expression is unevaluable and as its name suggests it is a temporary place holder until we're able to determine the actual attribute name.
object NamedExpression
object NonNullLiteral
An extractor that matches non-null literal values
object NullsFirst extends NullOrdering with Product with Serializable
object NullsLast extends NullOrdering with Product with Serializable
object Overlay extends Serializable
object ParseToTimestampLTZExpressionBuilder extends ExpressionBuilder
Parses a column to a timestamp with local time zone based on the supplied format.
Parses a column to a timestamp with local time zone based on the supplied format.

Annotations
@ExpressionDescription()
object ParseToTimestampNTZExpressionBuilder extends ExpressionBuilder
Parses a column to a timestamp without time zone based on the supplied format.
Parses a column to a timestamp without time zone based on the supplied format.

Annotations
@ExpressionDescription()
object ParseUrl extends Serializable
object Predicate extends CodeGeneratorWithInterpretedFallback[Expression, BasePredicate]
The factory object for BasePredicate.
object PythonUDF extends Serializable
Helper functions for PythonUDF
object RPadExpressionBuilder extends PadExpressionBuilderBase

Annotations
@ExpressionDescription()
object RaiseError extends Serializable
object Rand extends Serializable
object Randn extends Serializable
object RangeFrame extends FrameType with Product with Serializable
RangeFrame treats rows in a partition as groups of peers.
RangeFrame treats rows in a partition as groups of peers. All rows having the same ORDER BY ordering are considered as peers. Values used in a range frame are considered to be logical offsets. For example, assuming the value of the current row's ORDER BY expression expr is v, RANGE BETWEEN 1 PRECEDING AND 1 FOLLOWING represents a frame containing rows whose values expr are in the range of [v-1, v+1].
If ORDER BY clause is not defined, all rows in the partition are considered as peers of the current row.
object RegExpExtractBase
object RegExpReplace extends Serializable
object RegExpUtils
object Rollup extends Serializable
object RowFrame extends FrameType with Product with Serializable
RowFrame treats rows in a partition individually.
RowFrame treats rows in a partition individually. Values used in a row frame are considered to be physical offsets. For example, ROW BETWEEN 1 PRECEDING AND 1 FOLLOWING represents a 3-row frame, from the row that precedes the current row to the row that follows the current row.
object RowOrdering extends CodeGeneratorWithInterpretedFallback[Seq[SortOrder], BaseOrdering]
object SafeProjection extends CodeGeneratorWithInterpretedFallback[Seq[Expression], Projection]
A projection that could turn UnsafeRow into GenericInternalRow
object ScalarSubquery extends Serializable
object SchemaPruning extends SQLConfHelper
object SelectedField
A Scala extractor that builds a org.apache.spark.sql.types.StructField from a Catalyst complex type extractor.
A Scala extractor that builds a org.apache.spark.sql.types.StructField from a Catalyst complex type extractor. For example, consider a relation with the following schema:
```
root
|-- name: struct (nullable = true)
|    |-- first: string (nullable = true)
|    |-- last: string (nullable = true)
```
Further, suppose we take the select expression name.first. This will parse into an Alias(child, "first"). Ignoring the alias, child matches the following pattern:
```
GetStructFieldObject(
  AttributeReference("name", StructType(_), _, _),
  StructField("first", StringType, _, _))
```
SelectedField converts that expression into
```
StructField("name", StructType(Array(StructField("first", StringType))))
```
by mapping each complex type extractor to a org.apache.spark.sql.types.StructField with the same name as its child (or "parent" going right to left in the select expression) and a data type appropriate to the complex type extractor. In our example, the name of the child expression is "name" and its data type is a org.apache.spark.sql.types.StructType with a single string field named "first".
object Sequence extends Serializable
object SessionWindow extends Serializable
object Size extends Serializable
object SizeBasedWindowFunction extends Serializable
object SortOrder extends Serializable
object StartsWithExpressionBuilder extends StringBinaryPredicateExpressionBuilderBase

Annotations
@ExpressionDescription()
object StringLiteral
Extractor for retrieving String literals.
object StringTranslate extends Serializable
object StringTrim extends Serializable
object StringTrimLeft extends Serializable
object StringTrimRight extends Serializable
object SubExprUtils extends PredicateHelper
object SubqueryExpression
object SubtractDates extends Serializable
object SubtractTimestamps extends Serializable
object TimeWindow extends Serializable
object ToCharacterBuilder extends ExpressionBuilder
A function that converts decimal/datetime values to strings, returning NULL if the value fails to match the format string.
A function that converts decimal/datetime values to strings, returning NULL if the value fails to match the format string.

Annotations
@ExpressionDescription()
object TryToTimestampExpressionBuilder extends ExpressionBuilder
* Parses a column to a timestamp based on the supplied format.
* Parses a column to a timestamp based on the supplied format.

Annotations
@ExpressionDescription()
object UnBase64 extends Serializable
object UnaryExpression
object UnboundedFollowing extends LeafExpression with SpecialFrameBoundary with Product with Serializable
object UnboundedPreceding extends LeafExpression with SpecialFrameBoundary with Product with Serializable
UNBOUNDED boundary.
object UnresolvedNamedLambdaVariable extends Serializable
object UnsafeProjection extends CodeGeneratorWithInterpretedFallback[Seq[Expression], UnsafeProjection]
The factory object for UnsafeProjection.
object UnspecifiedFrame extends Expression with WindowFrame with LeafLike[Expression] with Product with Serializable
Used as a placeholder when a frame specification is not defined.
object UpdateFields extends Serializable
object UrlCodec
object V2ExpressionUtils extends SQLConfHelper with Logging
A utility class that converts public connector expressions into Catalyst expressions.
object VirtualColumn
object WidthBucket extends Serializable
object WindowExpression extends Serializable
object WindowFunctionType
object XxHash64Function extends InterpretedHashFunction

Packages

Standard Expressions

Named Expressions

Evaluation

expressions

package expressions

Standard Expressions

Named Expressions

Evaluation

Type Members

Value Members

Inherited from AnyRef

Inherited from Any

Ungrouped

Packages

Standard Expressions

Named Expressions

Evaluation

expressions 

package expressions

Standard Expressions

Named Expressions

Evaluation

Type Members

Value Members

Inherited from AnyRef

Inherited from Any

Ungrouped

expressions