conditions
cast_string_to_boolean(column_or_name)
Casts a column of string values to boolean values.
This function converts specific string representations of boolean values
to their corresponding boolean types. The recognized string values for
False
are "False", "false", "F", "f", and "0". The recognized string
values for True
are "True", "true", "T", "t", and "1". Any other values
will be converted to None.
Parameters:
Name | Type | Description | Default |
---|---|---|---|
column
|
Column
|
The input column containing string values to be cast. |
required |
Returns:
Name | Type | Description |
---|---|---|
Column |
Column
|
A column with boolean values where recognized strings are |
Column
|
converted to their corresponding boolean values, and unrecognized |
|
Column
|
strings are converted to None. |
Source code in pysparky/functions/cast.py
to_timestamps(column_or_name, formats)
Converts a column with date/time strings into a timestamp column by trying multiple formats.
This function iterates over a list of date/time formats and attempts to parse the input column
using each format. The first format that successfully parses the value is used. If no format succeeds,
the result for that row is NULL
.
Parameters:
column_or_name : ColumnOrName The input Spark column containing date/time strings to be converted to timestamp format. or the column name
list[str]
A list of date/time format strings to try. Formats should follow the pattern
conventions of java.text.SimpleDateFormat
, such as "yyyy-MM-dd", "MM/dd/yyyy", etc.
Returns:
Column
A Spark Column of type timestamp. If none of the formats match for a row, the value will be NULL
.