Spark函数 - 技术栈

文章目录

- - !
  - !=
  - %
  - &
  - *
  - +
  - -
  - /
  - <
  - <<
  - <=
  - <=>
  - <>
  - =
  - ==
  - >
  - >=
  - >>
  - >>>
  - ^
  - |
  - ||
  - ~

!

! expr - Logical not.

Examples:

sql 复制代码

> SELECT ! true;
 false
> SELECT ! false;
 true
> SELECT ! NULL;
 NULL

Since: 1.0.0

!=

expr1 != expr2 - Returns true if expr1 is not equal to expr2, or false otherwise.

Arguments:

expr1, expr2 - the two expressions must be same type or can be casted to a common type, and must be a type that can be used in equality comparison. Map type is not supported. For complex types such array/struct, the data types of fields must be orderable.

Examples:

sql 复制代码

> SELECT 1 != 2;
 true
> SELECT 1 != '2';
 true
> SELECT true != NULL;
 NULL
> SELECT NULL != NULL;
 NULL

Since: 1.0.0

%

expr1 % expr2, or mod(expr1, expr2) - Returns the remainder after expr1/expr2.

Examples:

sql 复制代码

> SELECT 2 % 1.8;
 0.2
> SELECT MOD(2, 1.8);
 0.2

Since: 1.0.0

&

expr1 & expr2 - Returns the result of bitwise AND of expr1 and expr2.

Examples:

sql 复制代码

> SELECT 3 & 5;
 1

Since: 1.4.0

*

expr1 * expr2 - Returns expr1*expr2.

Examples:

sql 复制代码

> SELECT 2 * 3;
 6

Since: 1.0.0

+

expr1 + expr2 - Returns expr1+expr2.

Examples:

sql 复制代码

> SELECT 1 + 2;
 3

Since: 1.0.0

-

expr1 - expr2 - Returns expr1-expr2.

Examples:

sql 复制代码

> SELECT 2 - 1;
 1

Since: 1.0.0

/

expr1 / expr2 - Returns expr1/expr2. It always performs floating point division.

Examples:

sql 复制代码

> SELECT 3 / 2;
 1.5
> SELECT 2L / 2L;
 1.0

Since: 1.0.0

<

expr1 < expr2 - Returns true if expr1 is less than expr2.

Arguments:

expr1, expr2 - the two expressions must be same type or can be casted to a common type, and must be a type that can be ordered. For example, map type is not orderable, so it is not supported. For complex types such array/struct, the data types of fields must be orderable.

Examples:

sql 复制代码

> SELECT 1 < 2;
 true
> SELECT 1.1 < '1';
 false
> SELECT to_date('2009-07-30 04:17:52') < to_date('2009-07-30 04:17:52');
 false
> SELECT to_date('2009-07-30 04:17:52') < to_date('2009-08-01 04:17:52');
 true
> SELECT 1 < NULL;
 NULL

Since: 1.0.0

<<

base << exp - Bitwise left shift.

Examples:

sql 复制代码

> SELECT shiftleft(2, 1);
 4
> SELECT 2 << 1;
 4

Note:

<< operator is added in Spark 4.0.0 as an alias for shiftleft.

Since: 4.0.0

<=

expr1 <= expr2 - Returns true if expr1 is less than or equal to expr2.

Arguments:

expr1, expr2 - the two expressions must be same type or can be casted to a common type, and must be a type that can be ordered. For example, map type is not orderable, so it is not supported. For complex types such array/struct, the data types of fields must be orderable.

Examples:

sql 复制代码

> SELECT 2 <= 2;
 true
> SELECT 1.0 <= '1';
 true
> SELECT to_date('2009-07-30 04:17:52') <= to_date('2009-07-30 04:17:52');
 true
> SELECT to_date('2009-07-30 04:17:52') <= to_date('2009-08-01 04:17:52');
 true
> SELECT 1 <= NULL;
 NULL

Since: 1.0.0

<=>

expr1 <=> expr2 - Returns same result as the EQUAL(=) operator for non-null operands, but returns true if both are null, false if one of the them is null.

Arguments:

expr1, expr2 - the two expressions must be same type or can be casted to a common type, and must be a type that can be used in equality comparison. Map type is not supported. For complex types such array/struct, the data types of fields must be orderable.

Examples:

sql 复制代码

> SELECT 2 <=> 2;
 true
> SELECT 1 <=> '1';
 true
> SELECT true <=> NULL;
 false
> SELECT NULL <=> NULL;
 true

Since: 1.1.0

<>

expr1 != expr2 - Returns true if expr1 is not equal to expr2, or false otherwise.

Arguments:

expr1, expr2 - the two expressions must be same type or can be casted to a common type, and must be a type that can be used in equality comparison. Map type is not supported. For complex types such array/struct, the data types of fields must be orderable.

Examples:

sql 复制代码

> SELECT 1 != 2;
 true
> SELECT 1 != '2';
 true
> SELECT true != NULL;
 NULL
> SELECT NULL != NULL;
 NULL

Since: 1.0.0

=

expr1 = expr2 - Returns true if expr1 equals expr2, or false otherwise.

Arguments:

expr1, expr2 - the two expressions must be same type or can be casted to a common type, and must be a type that can be used in equality comparison. Map type is not supported. For complex types such array/struct, the data types of fields must be orderable.

Examples:

sql 复制代码

> SELECT 2 = 2;
 true
> SELECT 1 = '1';
 true
> SELECT true = NULL;
 NULL
> SELECT NULL = NULL;
 NULL

Since: 1.0.0

==

expr1 == expr2 - Returns true if expr1 equals expr2, or false otherwise.

Arguments:

expr1, expr2 - the two expressions must be same type or can be casted to a common type, and must be a type that can be used in equality comparison. Map type is not supported. For complex types such array/struct, the data types of fields must be orderable.

Examples:

sql 复制代码

> SELECT 2 == 2;
 true
> SELECT 1 == '1';
 true
> SELECT true == NULL;
 NULL
> SELECT NULL == NULL;
 NULL

Since: 1.0.0

>

expr1 > expr2 - Returns true if expr1 is greater than expr2.

Arguments:

expr1, expr2 - the two expressions must be same type or can be casted to a common type, and must be a type that can be ordered. For example, map type is not orderable, so it is not supported. For complex types such array/struct, the data types of fields must be orderable.

Examples:

sql 复制代码

> SELECT 2 > 1;
 true
> SELECT 2 > 1.1;
 true
> SELECT to_date('2009-07-30 04:17:52') > to_date('2009-07-30 04:17:52');
 false
> SELECT to_date('2009-07-30 04:17:52') > to_date('2009-08-01 04:17:52');
 false
> SELECT 1 > NULL;
 NULL

Since: 1.0.0

>=

expr1 >= expr2 - Returns true if expr1 is greater than or equal to expr2.

Arguments:

expr1, expr2 - the two expressions must be same type or can be casted to a common type, and must be a type that can be ordered. For example, map type is not orderable, so it is not supported. For complex types such array/struct, the data types of fields must be orderable.

Examples:

sql 复制代码

> SELECT 2 >= 1;
 true
> SELECT 2.0 >= '2.1';
 false
> SELECT to_date('2009-07-30 04:17:52') >= to_date('2009-07-30 04:17:52');
 true
> SELECT to_date('2009-07-30 04:17:52') >= to_date('2009-08-01 04:17:52');
 false
> SELECT 1 >= NULL;
 NULL

Since: 1.0.0

>>

base >> expr - Bitwise (signed) right shift.

Examples:

sql 复制代码

> SELECT shiftright(4, 1);
 2
> SELECT 4 >> 1;
 2

Note:

>> operator is added in Spark 4.0.0 as an alias for shiftright.

Since: 4.0.0

>>>

base >>> expr - Bitwise unsigned right shift.

Examples:

sql 复制代码

> SELECT shiftrightunsigned(4, 1);
 2
> SELECT 4 >>> 1;
 2

Note:

>>> operator is added in Spark 4.0.0 as an alias for shiftrightunsigned.

Since: 4.0.0

^

expr1 ^ expr2 - Returns the result of bitwise exclusive OR of expr1 and expr2.

Examples:

sql 复制代码

> SELECT 3 ^ 5;
 6

Since: 1.4.0

|

expr1 | expr2 - Returns the result of bitwise OR of expr1 and expr2.

Examples:

sql 复制代码

> SELECT 3 | 5;
 7

Since: 1.4.0

||

expr1 || expr2 - Returns the concatenation of expr1 and expr2.

Examples:

sql 复制代码

> SELECT 'Spark' || 'SQL';
 SparkSQL
> SELECT array(1, 2, 3) || array(4, 5) || array(6);
 [1,2,3,4,5,6]

Note:

|| for arrays is available since 2.4.0.

Since: 2.3.0

~

~ expr - Returns the result of bitwise NOT of expr.

Examples:

sql 复制代码

> SELECT ~ 0;
 -1

Since: 1.4.0