Commit a3c0303
[SPARK-54442][SQL] Add numeric conversion functions for TIME type
### What changes were proposed in this pull request?
This PR adds six numeric conversion functions for the TIME type, mirroring the existing pattern for TIMESTAMP:
Constructor Functions (Numeric → TIME):
- time_from_seconds(seconds) - Supports fractional seconds via NumericType
- time_from_millis(millis) - IntegralType input
- time_from_micros(micros) - IntegralType input
Extractor Functions (TIME → Numeric):
- time_to_seconds(time) - Returns DECIMAL(14,6) to preserve fractional seconds
- time_to_millis(time) - Returns BIGINT
- time_to_micros(time) - Returns BIGINT
eg
```
-- Constructor functions (Numeric → TIME)
SELECT time_from_seconds(52200); -- 14:30:00
SELECT time_from_seconds(52200.123456); -- 14:30:00.123456
SELECT time_from_millis(52200123); -- 14:30:00.123
SELECT time_from_micros(52200123456); -- 14:30:00.123456
-- Extractor functions (TIME → Numeric)
SELECT time_to_seconds(TIME'14:30:00.123456'); -- 52200.123456
SELECT time_to_millis(TIME'14:30:00.123'); -- 52200123
SELECT time_to_micros(TIME'14:30:00.123456'); -- 52200123456
```
### Why are the changes needed?
The TIME type lacks numeric conversion functions, making it difficult to:
- Create TIME values from numeric representations (common in data ingestion)
- Extract numeric values for calculations or external system integration
TIMESTAMP has equivalent functions (timestamp_seconds(), unix_seconds(), etc.), and TIME should achieve feature parity.
### Does this PR introduce _any_ user-facing change?
Yes, adds six new SQL functions:
### How was this patch tested?
Unit Tests (TimeExpressionsSuite.scala):
SQL Integration Tests (time.sql)
### Was this patch authored or co-authored using generative AI tooling?
Yes.
Generated-by: Claude 3.5 Sonnet
AI assistance was used for:
- Code pattern analysis and design discussions
- Implementation guidance following Spark conventions
- Test case generation and organization
- Documentation and examples
### Additional context
Q: Why does time_to_seconds() return DECIMAL(14,6) instead of BIGINT?
A: To preserve fractional seconds and enable exact round-trip conversions:
```
SELECT time_to_seconds(time_from_seconds(52200.123456));
-- Returns: 52200.123456 (exact match) ✓
```
If we returned BIGINT, fractional seconds would be lost.
Q: Why does time_from_seconds() accept NumericType instead of just IntegralType?
A: To support fractional seconds for maximum flexibility:
```
SELECT time_from_seconds(52200.5); -- DECIMAL: 14:30:00.5
SELECT time_from_seconds(52200.5::float); -- FLOAT: 14:30:00.5
```
This mirrors timestamp_seconds() which also accepts NumericType.
Closes apache#53147 from vinodkc/br_time_numeric_conversion.
Authored-by: vinodkc <[email protected]>
Signed-off-by: Dongjoon Hyun <[email protected]>1 parent 58b074d commit a3c0303
File tree
11 files changed
+1434
-2
lines changed- python/pyspark/sql
- connect/functions
- functions
- sql
- api/src/main/scala/org/apache/spark/sql
- catalyst/src
- main/scala/org/apache/spark/sql/catalyst
- analysis
- expressions
- test/scala/org/apache/spark/sql/catalyst/expressions
- core/src/test/resources
- sql-functions
- sql-tests
- analyzer-results
- inputs
- results
11 files changed
+1434
-2
lines changed| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
3945 | 3945 | | |
3946 | 3946 | | |
3947 | 3947 | | |
| 3948 | + | |
| 3949 | + | |
| 3950 | + | |
| 3951 | + | |
| 3952 | + | |
| 3953 | + | |
| 3954 | + | |
| 3955 | + | |
| 3956 | + | |
| 3957 | + | |
| 3958 | + | |
| 3959 | + | |
| 3960 | + | |
| 3961 | + | |
| 3962 | + | |
| 3963 | + | |
| 3964 | + | |
| 3965 | + | |
| 3966 | + | |
| 3967 | + | |
| 3968 | + | |
| 3969 | + | |
| 3970 | + | |
| 3971 | + | |
| 3972 | + | |
| 3973 | + | |
| 3974 | + | |
| 3975 | + | |
| 3976 | + | |
| 3977 | + | |
| 3978 | + | |
| 3979 | + | |
| 3980 | + | |
| 3981 | + | |
| 3982 | + | |
| 3983 | + | |
| 3984 | + | |
| 3985 | + | |
| 3986 | + | |
| 3987 | + | |
| 3988 | + | |
| 3989 | + | |
3948 | 3990 | | |
3949 | 3991 | | |
3950 | 3992 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
249 | 249 | | |
250 | 250 | | |
251 | 251 | | |
| 252 | + | |
| 253 | + | |
| 254 | + | |
| 255 | + | |
| 256 | + | |
| 257 | + | |
252 | 258 | | |
253 | 259 | | |
254 | 260 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
24958 | 24958 | | |
24959 | 24959 | | |
24960 | 24960 | | |
| 24961 | + | |
| 24962 | + | |
| 24963 | + | |
| 24964 | + | |
| 24965 | + | |
| 24966 | + | |
| 24967 | + | |
| 24968 | + | |
| 24969 | + | |
| 24970 | + | |
| 24971 | + | |
| 24972 | + | |
| 24973 | + | |
| 24974 | + | |
| 24975 | + | |
| 24976 | + | |
| 24977 | + | |
| 24978 | + | |
| 24979 | + | |
| 24980 | + | |
| 24981 | + | |
| 24982 | + | |
| 24983 | + | |
| 24984 | + | |
| 24985 | + | |
| 24986 | + | |
| 24987 | + | |
| 24988 | + | |
| 24989 | + | |
| 24990 | + | |
| 24991 | + | |
| 24992 | + | |
| 24993 | + | |
| 24994 | + | |
| 24995 | + | |
| 24996 | + | |
| 24997 | + | |
| 24998 | + | |
| 24999 | + | |
| 25000 | + | |
| 25001 | + | |
| 25002 | + | |
| 25003 | + | |
| 25004 | + | |
| 25005 | + | |
| 25006 | + | |
| 25007 | + | |
| 25008 | + | |
| 25009 | + | |
| 25010 | + | |
| 25011 | + | |
| 25012 | + | |
| 25013 | + | |
| 25014 | + | |
| 25015 | + | |
| 25016 | + | |
| 25017 | + | |
| 25018 | + | |
| 25019 | + | |
| 25020 | + | |
| 25021 | + | |
| 25022 | + | |
| 25023 | + | |
| 25024 | + | |
| 25025 | + | |
| 25026 | + | |
| 25027 | + | |
| 25028 | + | |
| 25029 | + | |
| 25030 | + | |
| 25031 | + | |
| 25032 | + | |
| 25033 | + | |
| 25034 | + | |
| 25035 | + | |
| 25036 | + | |
| 25037 | + | |
| 25038 | + | |
| 25039 | + | |
| 25040 | + | |
| 25041 | + | |
| 25042 | + | |
| 25043 | + | |
| 25044 | + | |
| 25045 | + | |
| 25046 | + | |
| 25047 | + | |
| 25048 | + | |
| 25049 | + | |
| 25050 | + | |
| 25051 | + | |
| 25052 | + | |
| 25053 | + | |
| 25054 | + | |
| 25055 | + | |
| 25056 | + | |
| 25057 | + | |
| 25058 | + | |
| 25059 | + | |
| 25060 | + | |
| 25061 | + | |
| 25062 | + | |
| 25063 | + | |
| 25064 | + | |
| 25065 | + | |
| 25066 | + | |
| 25067 | + | |
| 25068 | + | |
| 25069 | + | |
| 25070 | + | |
| 25071 | + | |
| 25072 | + | |
| 25073 | + | |
| 25074 | + | |
| 25075 | + | |
| 25076 | + | |
| 25077 | + | |
| 25078 | + | |
| 25079 | + | |
| 25080 | + | |
| 25081 | + | |
| 25082 | + | |
| 25083 | + | |
| 25084 | + | |
| 25085 | + | |
| 25086 | + | |
| 25087 | + | |
| 25088 | + | |
| 25089 | + | |
| 25090 | + | |
| 25091 | + | |
| 25092 | + | |
| 25093 | + | |
| 25094 | + | |
| 25095 | + | |
| 25096 | + | |
| 25097 | + | |
| 25098 | + | |
| 25099 | + | |
| 25100 | + | |
| 25101 | + | |
| 25102 | + | |
| 25103 | + | |
| 25104 | + | |
| 25105 | + | |
| 25106 | + | |
| 25107 | + | |
| 25108 | + | |
| 25109 | + | |
| 25110 | + | |
| 25111 | + | |
| 25112 | + | |
| 25113 | + | |
| 25114 | + | |
| 25115 | + | |
| 25116 | + | |
24961 | 25117 | | |
24962 | 25118 | | |
24963 | 25119 | | |
| |||
Lines changed: 49 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
6910 | 6910 | | |
6911 | 6911 | | |
6912 | 6912 | | |
| 6913 | + | |
| 6914 | + | |
| 6915 | + | |
| 6916 | + | |
| 6917 | + | |
| 6918 | + | |
| 6919 | + | |
| 6920 | + | |
| 6921 | + | |
| 6922 | + | |
| 6923 | + | |
| 6924 | + | |
| 6925 | + | |
| 6926 | + | |
| 6927 | + | |
| 6928 | + | |
| 6929 | + | |
| 6930 | + | |
| 6931 | + | |
| 6932 | + | |
| 6933 | + | |
| 6934 | + | |
| 6935 | + | |
| 6936 | + | |
| 6937 | + | |
| 6938 | + | |
| 6939 | + | |
| 6940 | + | |
| 6941 | + | |
| 6942 | + | |
| 6943 | + | |
| 6944 | + | |
| 6945 | + | |
| 6946 | + | |
| 6947 | + | |
| 6948 | + | |
| 6949 | + | |
| 6950 | + | |
| 6951 | + | |
| 6952 | + | |
| 6953 | + | |
| 6954 | + | |
| 6955 | + | |
| 6956 | + | |
| 6957 | + | |
| 6958 | + | |
| 6959 | + | |
| 6960 | + | |
| 6961 | + | |
6913 | 6962 | | |
6914 | 6963 | | |
6915 | 6964 | | |
| |||
Lines changed: 6 additions & 0 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
701 | 701 | | |
702 | 702 | | |
703 | 703 | | |
| 704 | + | |
| 705 | + | |
| 706 | + | |
| 707 | + | |
| 708 | + | |
| 709 | + | |
704 | 710 | | |
705 | 711 | | |
706 | 712 | | |
| |||
0 commit comments