You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
Commit 053f392
Browse filesBrowse the repository at this point in the historyBrowse files
Update the Widen and decimal plan for the PRs since 2026-10-26
WD1 to WD4, CD3, CD3b and the decimal Interleave kernel merged, and
CD4, CD5 and WD6 are open drafts. The plan now lists CD3b (#10939) and
the Interleave kernel (CD4p, #10940) as dependencies of CD4, and CD3b
as a dependency of CD6 and CD7. CD7 merges right after CD5 (#10905).
D26 moves to CD6, because CD4 (#10946) adds no serialization lowering:
the default writer executes a Widen, and reads of old files are the
first ordinary source of one. The CD6 and WR1 briefs now carry that
choice. D35 no longer makes WD6 wait for WD5. #10908 joins the
follow-up issues, and the plan suggests settling the mechanism of
Signed-off-by: Connor Tsui <connor.tsui20@gmail.com>
#10748 before WD5.
Copy file name to clipboardExpand all lines: research/decimal/04-implementation.md
+23-12Lines changed: 23 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -66,19 +66,21 @@ planning review on 2026-10-23.
66
66
| FX1 | Strict decoding of stored `DecimalType` and `PType` values, in three commits | None | Medium |
67
67
| FX2 | Dropped. #10798 documents the cast-rule contract instead (see the FX2 brief) | None | None |
68
68
| FX3 | Dictionary `fill_null` no longer truncates the fill code | None | Small |
69
-
| WD1 | In-memory integer `Widen`, with `encode`, the deprecated `encode_values`, the serialization workaround and a flatten rule | None | Large |
69
+
| WD1 | In-memory integer `Widen`, with `encode`, the deprecated `encode_values` and a flatten rule | None | Large |
70
70
| WD2 | Benchmarks for compute on integer `Widen`| WD1 | Medium |
71
71
| WD3 | Aggregates run on the child | WD2 | Medium |
72
72
| WD4 | Arithmetic, `between`, `zip` and casts run on the child | WD3 | Large |
73
73
| WD5 | Structural encodings move above `Widen`, and dictionary codes use the child | WD4, FX3 | Medium |
74
-
| WD6 | The existing narrowing call sites move to `encode_values`| WD1. It merges after WD5 (D35) | Medium |
74
+
| WD6 | The existing narrowing call sites move to `encode_values`| WD1 (D35) | Medium |
75
75
| CD1 | Decimal benchmarks | None | Medium |
76
76
| CD2 | Compat fixtures for old decimal forms | None | Medium |
77
77
| CD3 | One width function, imports that check and copy, and the shared helpers | None | Medium |
78
-
| CD4 | Decimal `Widen` in memory | WD1, WD3, WD4, CD1, CD3 | Large |
78
+
| CD3b | One scan for the range of valid decimal values (`valid_range`), shared by CD4, CD6 and CD7 | CD3 | Small |
79
+
| CD4p | Decimal `Interleave` execution, so that CD4 can test its fallbacks | None | Medium |
80
+
| CD4 | Decimal `Widen` in memory | WD1, WD3, WD4, CD1, CD3, CD3b, CD4p | Large |
79
81
| CD5 | DecimalByteParts decodes to the canonical width | CD1, CD3, released CD2 fixtures | Medium |
80
-
| CD6 | The `vortex.decimal` read plugin converts old nodes| FX1, CD3, CD4, released CD2 fixtures | Medium |
81
-
| CD7 | One DecimalByteParts scheme on the canonical array| WD1, CD1 | Medium |
82
+
| CD6 | The `vortex.decimal` read plugin converts old nodes, and decides D26 | FX1, CD3, CD3b, CD4, released CD2 fixtures | Medium |
83
+
| CD7 | One DecimalByteParts scheme on the canonical array. It merges right after CD5 | WD1, CD1, CD3b| Medium |
82
84
| CD8a | Every decimal array has the canonical width | CD3, CD5, CD6, CD7 | Large |
83
85
| CD8b |`DecimalArray::try_new` checks every valid value, with an `ExecutionCtx`| CD8a | Large |
84
86
| CD8c | Remove the code for mixed decimal widths | CD8a | Medium |
@@ -101,7 +103,8 @@ brief or a decision changes.
101
103
102
104
Each solid arrow is a hard dependency. Some arrows also follow from other arrows. The graph keeps
103
105
them, so that the direct needs of each PR show. The dashed arrows are order rules, not code
104
-
dependencies: the release gate of the compat fixtures (D31), and WD6 merging after WD5 (D35).
106
+
dependencies: the release gate of the compat fixtures (D31), and CD7 merging right after CD5, so
107
+
that the old scheme does not split decimals from their storage width for long (#10905).
105
108
106
109
```mermaid
107
110
flowchart TD
@@ -122,6 +125,8 @@ flowchart TD
122
125
CD2["CD2 Compat fixtures"]
123
126
REL(["Release with CD2"])
124
127
CD3["CD3 Width and imports"]
128
+
CD3b["CD3b Valid range scan"]
129
+
CD4p["CD4p Decimal Interleave"]
125
130
CD4["CD4 Decimal Widen"]
126
131
CD5["CD5 DBP decoding"]
127
132
CD6["CD6 Read plugin"]
@@ -139,12 +144,17 @@ flowchart TD
139
144
WD1 --> WD2 --> WD3 --> WD4 --> WD5
140
145
FX3 --> WD5
141
146
WD1 --> WD6
142
-
WD5 -.-> WD6
143
147
CD2 --> REL
144
148
REL -.-> CD5
145
149
REL -.-> CD6
146
150
CD1 --> CD4
147
151
CD3 --> CD4
152
+
CD3 --> CD3b
153
+
CD3b --> CD4
154
+
CD4p --> CD4
155
+
CD3b --> CD6
156
+
CD3b --> CD7
157
+
CD5 -.-> CD7
148
158
WD1 --> CD4
149
159
WD3 --> CD4
150
160
WD4 --> CD4
@@ -178,10 +188,10 @@ files. The column "Shared files" names the pairs that do not, so run those one a
178
188
179
189
| Wave | PRs | Note | Shared files |
180
190
| --- | --- | --- | --- |
181
-
| 1 | FX1, FX3, WD1, CD1, CD2, CD3 | No dependencies. The next regular release after CD2 publishes the fixtures | CD2 calls `smallest_decimal_value_type`, which CD3 removes. The second of the two to merge adapts |
182
-
| 2 | WD2, WD6, CD7, CD5 | WD6 merges after WD5 (D35). CD7 waits for CD1. CD5 waits for the release with CD2 | CD5 and CD7: the crate doc in `encodings/decimal-byte-parts/src/lib.rs`, the tests in `assemble.rs`, and `benches/dbp_assemble.rs`. CD7 also changes the signature of `split_decimal`. WD6 and CD7: the golden snapshots of `vortex-btrblocks`|
191
+
| 1 | FX1, FX3, WD1, CD1, CD2, CD3, CD4p| No dependencies. The next regular release after CD2 publishes the fixtures | CD2 calls `smallest_decimal_value_type`, which CD3 removes. The second of the two to merge adapts |
192
+
| 2 | WD2, WD6, CD3b, CD7, CD5 | WD6 needs only WD1 (D35). CD7 waits for CD1 and CD3b, and it merges right after CD5. CD5 waits for the release with CD2 | CD5 and CD7: the crate doc in `encodings/decimal-byte-parts/src/lib.rs`, the tests in `assemble.rs`, and `benches/dbp_assemble.rs`. CD7 also changes the signature of `split_decimal`. WD6 and CD7: the golden snapshots of `vortex-btrblocks`|
183
193
| 3 | WD3, then WD4 | One stack. Read the CodSpeed reports with the rule in the NR brief ("CodSpeed on stacked PRs") | None |
184
-
| 4 | WD5, CD4 | FX3 is merged (`dcb4f50ae2`) | Both edit `vortex-array/src/arrays/widen/rules.rs`|
194
+
| 4 | WD5, CD4 | FX3 is merged (`dcb4f50ae2`). Settle the mechanism question of #10748 before WD5, so that WD5 and phase 4a of the extension notes can share it| Both edit `vortex-array/src/arrays/widen/rules.rs`|
185
195
| 5 | CD6 | Waits for FX1, CD4 and the release with CD2 | None |
186
196
| 6 | CD8a | The last in-memory step that the wire change needs | None |
187
197
| 7 | CD8b, CD8c, WR1 | Each needs only CD8a | CD8b adds a `ctx` parameter to `DecimalArray::new` and `try_new`, which CD8c and WR1 call. CD8b and CD8c both edit `vortex-array/src/builders/decimal.rs` and the same tests. Land CD8b first, or rebase the others on it |
@@ -192,7 +202,7 @@ publishes the CD2 fixtures also gates CD5 and CD6, so CD2 is worth landing in wa
192
202
193
203
## Rules for every PR
194
204
195
-
- No PR adds temporary code (D11). The serialization workaround of D26, if CD4 adds it, is the one
205
+
- No PR adds temporary code (D11). The serialization workaround of D26, if CD6 adds it, is the one
196
206
documented exception. It stays until `vortex.widen` is in `core`.
197
207
- Benchmarks land before the changes that they cover (D19): WD2 before WD3 to WD5, and CD1 before
198
208
CD4, CD5 and CD7. The dependency table lists these as hard dependencies.
@@ -220,11 +230,12 @@ PRs, and each one has its own issue. Two are closed.
220
230
| Issue | Problem |
221
231
| --- | --- |
222
232
|#10748| Generic structural encodings always sit above other encodings. WD5 does this for `Widen` only |
223
-
|#10749| Decimal `scalar_at` and `min_max` panic on a stored value outside the precision, which a file read can produce |
233
+
|#10749| Decimal `scalar_at` and `min_max` panic on a stored value outside the precision, which a file read can produce. #10767 fixes both and is in review|
224
234
|#10750| Closed as not planned. The push-down cast rules return no wrong result, and #10798 documents them as a valid form |
225
235
|#10751| Closed by #10757. The FoR scheme now declares and refines its bit-packing child |
226
236
|#10754| The compat harness: no CI job runs `--mode all`, one failing fixture stops the whole check, and the README describes `--git-ref` wrongly |
227
237
|#10756| Corrupt ALP-RD, RLE and RunEnd metadata can panic a file read |
238
+
|#10908|`ZipExecuteAdaptor` dispatches only on the `if_true` branch, so a `Widen` in the `if_false` branch of `zip` does not run on the child. Found in the review of WD4 |
228
239
229
240
The stale versioning docs needed no issue, because #10747 fixes them.
0 commit comments