fix(multimodal): composite transparent images for Kimi-K3 - #1991
Conversation
Kimi-K2.5 and Kimi-K3 both encode images with MoonViT and differ only in their preprocessing parameters, but SMG registered `kimi-k3` against `KimiK25Processor` directly, leaving nowhere to express a K3-specific difference. The reference implementation instead factors the NaViT resize/patchify helpers into a shared `media_utils` module and keeps a thin per-model processor class on top. Mirror that split, using the same shape `qwen_vl_base` already provides for the Qwen VL family: - Move the MoonViT pipeline into `processors::moonvit` as `MoonVitProcessorBase`, parameterized by `MoonVitConfig`. - Reduce `KimiK25Processor` to a thin wrapper that `Deref`s to the base. - Add `KimiK3Processor` as a second wrapper with `model_name: "kimi-k3"`. - Point the `kimi-k3` / `kimi_k3` registrations at it. This commit is a no-op: `KimiK3Processor` reuses K2.5's parameters, so both processors still produce byte-identical output. A test asserts that directly, which anchors the behavior change that follows. Signed-off-by: key4ng <rukeyang@gmail.com>
K3's reference processor composites alpha-carrying images over a background before the NaViT resize and patchify; SMG dropped the alpha channel instead. That is not the same operation. A fully transparent PNG pixel usually stores RGB (0,0,0) under its alpha, so dropping alpha normalizes it to -1.0 — solid black — where the model was trained to see the chessboard the checkpoint describes. Logos, icons, screenshots with rounded corners, and anything else served as RGBA reached the encoder with its background inverted. Add the compositing step to the MoonViT base and let K3 drive it: - `transforms`: `TransparentBgConfig` / `TransparentBgPattern` / `TransparentBgFillStage` deserialize straight from a checkpoint, and `fill_transparent_bg` blends `a*img + (1-a)*bg` with the reference's truncating uint8 cast. - `transforms::resize_straight_alpha`: our SIMD resizer premultiplies by alpha by default, which drags RGB toward zero wherever alpha is partial. Pillow convolves channels independently, so compositing after a resize needs the non-premultiplied variant. - `PreProcessorConfig::transparent_bg()` lifts the two keys out of `media_proc_cfg` and returns them together, since the fill stage is only meaningful alongside a config. - `KimiK3Processor` resolves them per request rather than hardcoding them, matching the reference, which falls back to dropping alpha when `transparent_bg_config` is absent. K2.5 declares `transparent_bg: None` because its reference class has no transparency handling at all. The fill stage is load-bearing, not cosmetic: a chessboard is generated at the resolution of the image it lands on, so `before_resize` and `after_resize` produce different square sizes relative to the content. K3's checkpoint ships `after_resize`. Opaque images are untouched — they skip the compositing branch entirely, and a test asserts a declared background does not perturb them. Out of scope, tracked separately: K3's checkpoint also raises `in_patch_limit` to 65536, which changes token accounting. Signed-off-by: key4ng <rukeyang@gmail.com>
|
Caution The consumer version of Gemini Code Assist on GitHub has been sunset. All code review activity has officially ceased. |
📝 WalkthroughWalkthroughAdds shared MoonViT image preprocessing, transparent-background compositing and resizing, Kimi K3 processor registration, and Kimi K2.5 delegation to the shared pipeline. Configuration parsing now exposes transparency settings and fill-stage defaults. ChangesKimi MoonViT preprocessing
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant ModelRegistry
participant KimiK3Processor
participant PreProcessorConfig
participant MoonVitProcessorBase
participant EncoderInputs
ModelRegistry->>KimiK3Processor: select processor for kimi-k3
KimiK3Processor->>PreProcessorConfig: read transparency settings
KimiK3Processor->>MoonVitProcessorBase: preprocess with resolved config
MoonVitProcessorBase->>EncoderInputs: return patches and grid metadata
Suggested reviewers: Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Comment |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Clean, well-tested PR. Reviewed the full diff across all 7 changed files.
Refactoring (commit 1): The extraction of MoonVitProcessorBase into moonvit.rs is a faithful code move — the K2.5 wrapper becomes a pure Deref delegate and the opaque_images_match_k25_byte_for_byte test locks this as a no-op.
Bug fix (commit 2): The compositing logic in fill_transparent_bg correctly implements α·fg + (1−α)·bg with the truncating as u8 cast matching NumPy's astype(np.uint8). The resize_straight_alpha path correctly disables premultiplication via use_alpha(false) (fast_image_resize 6.x defaults to true, so the existing resize() path is unchanged). The BeforeResize/AfterResize stage dispatch handles the chessboard-resolution distinction correctly.
Design boundary: K2.5 hardcodes transparent_bg: None at construction and delegates directly to the base — transparency config in the PreProcessorConfig is intentionally invisible to it. K3's resolved() reads it per-request via Cow, which is the right place since the registry hands out default-constructed processors. This mirrors the Qwen VL family's pattern.
Edge cases verified: opaque images skip compositing; zero square size clamps to 1; missing transparent_bg_fill_stage defaults to BeforeResize; fill stage distinction is asserted by fill_stage_paints_the_board_at_its_own_resolution.
12 new tests, 0 issues found (0 🔴, 0 🟡, 0 🟣).
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 4
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/multimodal/src/vision/preprocessor_config.rs`:
- Around line 442-454: Update transparent_bg to distinguish an absent
transparent_bg_config from a present but unparseable value, and log the
deserialization error when parsing fails. Preserve returning None only when the
key is absent, while valid configs should continue constructing TransparentBg
with the existing stage behavior.
In `@crates/multimodal/src/vision/processors/kimi_k3.rs`:
- Around line 17-21: Update KimiK3Processor’s request-time resolution flow,
including resolved() and the registry-created KimiK3Processor::new() path, to
resolve in_patch_limit and patch_limit_on_one_side from the current preprocessor
configuration just as transparent_bg is resolved. Ensure
from_preprocessor_config participates in the request path so K3 uses the
checkpoint’s 65536 patch budget instead of the default K2.5 limit, while
preserving existing transparency handling.
In `@crates/multimodal/src/vision/transforms.rs`:
- Around line 129-149: Update fill_transparent_bg to return an empty RgbImage
before calling chunks_exact when the decoded image width or height is zero,
preserving the existing non-alpha conversion and compositing behavior for
non-empty images.
- Around line 366-395: Update the doc comment for resize_straight_alpha to refer
to ResizeOptions::use_alpha instead of ResizeOptions::mul_div_alpha. Leave the
implementation and all other documentation unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: ASSERTIVE
Plan: Pro
Run ID: fb98a171-34a1-4948-a85a-3fe7946fab15
📒 Files selected for processing (7)
crates/multimodal/src/vision/preprocessor_config.rscrates/multimodal/src/vision/processor.rscrates/multimodal/src/vision/processors/kimi_k25.rscrates/multimodal/src/vision/processors/kimi_k3.rscrates/multimodal/src/vision/processors/mod.rscrates/multimodal/src/vision/processors/moonvit.rscrates/multimodal/src/vision/transforms.rs
| /// The checkpoint's alpha-flattening behavior, if it declares any. | ||
| /// | ||
| /// `None` means the checkpoint asks for no compositing, matching the | ||
| /// reference's behavior when `transparent_bg_config` is absent. The fill | ||
| /// stage is only meaningful alongside a config, so it is read here rather | ||
| /// than exposed on its own. | ||
| pub fn transparent_bg(&self) -> Option<transforms::TransparentBg> { | ||
| let config = self.get_extra::<transforms::TransparentBgConfig>("transparent_bg_config")?; | ||
| let stage = self | ||
| .get_extra::<transforms::TransparentBgFillStage>("transparent_bg_fill_stage") | ||
| .unwrap_or_default(); | ||
| Some(transforms::TransparentBg { config, stage }) | ||
| } |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
A malformed transparent_bg_config silently degrades to alpha-drop.
get_extra discards deserialization errors, so a checkpoint that ships "pattern": "chess_board" (or any type mismatch) yields None here, and K3 then renders transparent regions as solid black with no signal anywhere. Log when the key is present but unparseable so the misconfiguration is diagnosable.
♻️ Surface the parse failure
pub fn transparent_bg(&self) -> Option<transforms::TransparentBg> {
- let config = self.get_extra::<transforms::TransparentBgConfig>("transparent_bg_config")?;
+ let raw = self.extra.get("transparent_bg_config")?;
+ let config = match serde_json::from_value::<transforms::TransparentBgConfig>(raw.clone()) {
+ Ok(config) => config,
+ Err(e) => {
+ tracing::warn!(error = %e, "unparseable transparent_bg_config; alpha will be dropped");
+ return None;
+ }
+ };
let stage = self
.get_extra::<transforms::TransparentBgFillStage>("transparent_bg_fill_stage")
.unwrap_or_default();
Some(transforms::TransparentBg { config, stage })
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| /// The checkpoint's alpha-flattening behavior, if it declares any. | |
| /// | |
| /// `None` means the checkpoint asks for no compositing, matching the | |
| /// reference's behavior when `transparent_bg_config` is absent. The fill | |
| /// stage is only meaningful alongside a config, so it is read here rather | |
| /// than exposed on its own. | |
| pub fn transparent_bg(&self) -> Option<transforms::TransparentBg> { | |
| let config = self.get_extra::<transforms::TransparentBgConfig>("transparent_bg_config")?; | |
| let stage = self | |
| .get_extra::<transforms::TransparentBgFillStage>("transparent_bg_fill_stage") | |
| .unwrap_or_default(); | |
| Some(transforms::TransparentBg { config, stage }) | |
| } | |
| /// The checkpoint's alpha-flattening behavior, if it declares any. | |
| /// | |
| /// `None` means the checkpoint asks for no compositing, matching the | |
| /// reference's behavior when `transparent_bg_config` is absent. The fill | |
| /// stage is only meaningful alongside a config, so it is read here rather | |
| /// than exposed on its own. | |
| pub fn transparent_bg(&self) -> Option<transforms::TransparentBg> { | |
| let raw = self.extra.get("transparent_bg_config")?; | |
| let config = match serde_json::from_value::<transforms::TransparentBgConfig>(raw.clone()) { | |
| Ok(config) => config, | |
| Err(e) => { | |
| tracing::warn!(error = %e, "unparseable transparent_bg_config; alpha will be dropped"); | |
| return None; | |
| } | |
| }; | |
| let stage = self | |
| .get_extra::<transforms::TransparentBgFillStage>("transparent_bg_fill_stage") | |
| .unwrap_or_default(); | |
| Some(transforms::TransparentBg { config, stage }) | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/multimodal/src/vision/preprocessor_config.rs` around lines 442 - 454,
Update transparent_bg to distinguish an absent transparent_bg_config from a
present but unparseable value, and log the deserialization error when parsing
fails. Preserve returning None only when the key is absent, while valid configs
should continue constructing TransparentBg with the existing stage behavior.
| //! Known remaining difference, deliberately left for a follow-up change: K3's | ||
| //! checkpoint raises `in_patch_limit` to 65536, so large images should keep | ||
| //! roughly 4x the visual tokens they get from K2.5's budget. Reading the | ||
| //! budget from the checkpoint touches token accounting on both models and is | ||
| //! tracked separately from this module's transparency handling. |
There was a problem hiding this comment.
📐 Maintainability & Code Quality | 🔵 Trivial
Documented gap: only transparency is re-resolved per request, not the patch budget.
resolved() lifts transparent_bg from the request config, but in_patch_limit/patch_limit_on_one_side stay at the default-constructed values because the registry hands out KimiK3Processor::new(). So from_preprocessor_config's limit reading never runs on the request path and K3 keeps K2.5's 16384 budget, as the module doc notes.
Want me to open a follow-up issue to track lifting the patch budget through the same call-time resolution?
Also applies to: 96-115
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/multimodal/src/vision/processors/kimi_k3.rs` around lines 17 - 21,
Update KimiK3Processor’s request-time resolution flow, including resolved() and
the registry-created KimiK3Processor::new() path, to resolve in_patch_limit and
patch_limit_on_one_side from the current preprocessor configuration just as
transparent_bg is resolved. Ensure from_preprocessor_config participates in the
request path so K3 uses the checkpoint’s 65536 patch budget instead of the
default K2.5 limit, while preserving existing transparency handling.
| pub fn fill_transparent_bg(image: &DynamicImage, config: TransparentBgConfig) -> RgbImage { | ||
| if !image.color().has_alpha() { | ||
| return image.to_rgb8(); | ||
| } | ||
|
|
||
| let rgba = image.to_rgba8(); | ||
| let (width, height) = rgba.dimensions(); | ||
| let mut out = Vec::with_capacity(width as usize * height as usize * 3); | ||
|
|
||
| for (y, row) in rgba.as_raw().chunks_exact(width as usize * 4).enumerate() { | ||
| for (x, px) in row.chunks_exact(4).enumerate() { | ||
| let bg = f32::from(config.background_at(x as u32, y as u32)); | ||
| let alpha = f32::from(px[3]) / 255.0; | ||
| let inv = 1.0 - alpha; | ||
| // numpy's `.astype(np.uint8)` truncates rather than rounds, and so | ||
| // does an `as` cast; keep them consistent. | ||
| out.push((alpha * f32::from(px[0]) + inv * bg) as u8); | ||
| out.push((alpha * f32::from(px[1]) + inv * bg) as u8); | ||
| out.push((alpha * f32::from(px[2]) + inv * bg) as u8); | ||
| } | ||
| } |
There was a problem hiding this comment.
🩺 Stability & Availability | 🟡 Minor | ⚡ Quick win
chunks_exact panics on a zero-width image.
chunks_exact(width as usize * 4) panics ("chunk size must be non-zero") when width == 0. The caller in moonvit.rs only checks has_alpha(), so a degenerate 0-width/0-height decode reaches here and panics on the request path instead of producing an empty/aligned canvas.
🛡️ Proposed guard
let rgba = image.to_rgba8();
let (width, height) = rgba.dimensions();
+ if width == 0 || height == 0 {
+ return RgbImage::new(width, height);
+ }
let mut out = Vec::with_capacity(width as usize * height as usize * 3);📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
| pub fn fill_transparent_bg(image: &DynamicImage, config: TransparentBgConfig) -> RgbImage { | |
| if !image.color().has_alpha() { | |
| return image.to_rgb8(); | |
| } | |
| let rgba = image.to_rgba8(); | |
| let (width, height) = rgba.dimensions(); | |
| let mut out = Vec::with_capacity(width as usize * height as usize * 3); | |
| for (y, row) in rgba.as_raw().chunks_exact(width as usize * 4).enumerate() { | |
| for (x, px) in row.chunks_exact(4).enumerate() { | |
| let bg = f32::from(config.background_at(x as u32, y as u32)); | |
| let alpha = f32::from(px[3]) / 255.0; | |
| let inv = 1.0 - alpha; | |
| // numpy's `.astype(np.uint8)` truncates rather than rounds, and so | |
| // does an `as` cast; keep them consistent. | |
| out.push((alpha * f32::from(px[0]) + inv * bg) as u8); | |
| out.push((alpha * f32::from(px[1]) + inv * bg) as u8); | |
| out.push((alpha * f32::from(px[2]) + inv * bg) as u8); | |
| } | |
| } | |
| pub fn fill_transparent_bg(image: &DynamicImage, config: TransparentBgConfig) -> RgbImage { | |
| if !image.color().has_alpha() { | |
| return image.to_rgb8(); | |
| } | |
| let rgba = image.to_rgba8(); | |
| let (width, height) = rgba.dimensions(); | |
| if width == 0 || height == 0 { | |
| return RgbImage::new(width, height); | |
| } | |
| let mut out = Vec::with_capacity(width as usize * height as usize * 3); | |
| for (y, row) in rgba.as_raw().chunks_exact(width as usize * 4).enumerate() { | |
| for (x, px) in row.chunks_exact(4).enumerate() { | |
| let bg = f32::from(config.background_at(x as u32, y as u32)); | |
| let alpha = f32::from(px[3]) / 255.0; | |
| let inv = 1.0 - alpha; | |
| // numpy's `.astype(np.uint8)` truncates rather than rounds, and so | |
| // does an `as` cast; keep them consistent. | |
| out.push((alpha * f32::from(px[0]) + inv * bg) as u8); | |
| out.push((alpha * f32::from(px[1]) + inv * bg) as u8); | |
| out.push((alpha * f32::from(px[2]) + inv * bg) as u8); | |
| } | |
| } |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/multimodal/src/vision/transforms.rs` around lines 129 - 149, Update
fill_transparent_bg to return an empty RgbImage before calling chunks_exact when
the decoded image width or height is zero, preserving the existing non-alpha
conversion and compositing behavior for non-empty images.
| /// Resize without premultiplying by alpha, matching `PIL.Image.resize`. | ||
| /// | ||
| /// Pillow convolves every channel independently, alpha included. Our SIMD | ||
| /// resizer premultiplies by default (`ResizeOptions::mul_div_alpha`), which | ||
| /// changes RGB wherever alpha is partial and would diverge from any reference | ||
| /// pipeline that composites *after* the resize. | ||
| pub fn resize_straight_alpha( | ||
| image: &DynamicImage, | ||
| width: u32, | ||
| height: u32, | ||
| filter: FilterType, | ||
| ) -> DynamicImage { | ||
| resize_inner(image, width, height, filter, false) | ||
| } | ||
|
|
||
| fn resize_inner( | ||
| image: &DynamicImage, | ||
| width: u32, | ||
| height: u32, | ||
| filter: FilterType, | ||
| premultiply_alpha: bool, | ||
| ) -> DynamicImage { | ||
| let pixel_type = match image.pixel_type() { | ||
| Some(pt) => pt, | ||
| None => return image.resize_exact(width, height, filter), | ||
| }; | ||
| let mut dst = FirImage::new(width, height, pixel_type); | ||
| let options = ResizeOptions::new().resize_alg(to_fir_algorithm(filter)); | ||
| let options = ResizeOptions::new() | ||
| .resize_alg(to_fir_algorithm(filter)) | ||
| .use_alpha(premultiply_alpha); |
There was a problem hiding this comment.
🎯 Functional Correctness | 🔵 Trivial | 💤 Low value
🧩 Analysis chain
🌐 Web query:
fast_image_resize 6.0 ResizeOptions use_alpha default premultiply alpha
💡 Result:
In the fast_image_resize 6.0 library, the ResizeOptions struct controls the resizing process, including how alpha channels are handled [1][2]. The field controlling alpha channel consideration is mul_div_alpha [1]. When enabled, the resizer automatically multiplies the source image's color channels by its alpha channel before resizing and divides the destination image's color channels by the alpha channel after resizing [3]. Key details regarding mul_div_alpha: - Default Value: The default value for mul_div_alpha is true [1][2]. - Purpose: It enables or disables the consideration of the alpha channel during the resize process [1]. - Method: You can use the use_alpha(bool) method on ResizeOptions to set this value [1][2]. - Supported Types: By default, the Resizer performs this multiplication and division for images with U8x2, U8x4, U16x2, and U16x4 pixel types [3]. If you explicitly set mul_div_alpha to true (or rely on the default), the library ensures that color channels are premultiplied by alpha prior to convolution-based filtering to avoid dark fringes or incorrect color bleeding, and then unpremultiplied in the final result [2][3].
Citations:
- 1: https://docs.rs/fast_image_resize/latest/fast_image_resize/struct.ResizeOptions.html
- 2: https://docs.rs/fast_image_resize/latest/src/fast_image_resize/resizer.rs.html
- 3: https://github.com/Cykooz/fast_image_resize/blob/master/CHANGELOG.md
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Inspect the relevant file around the cited lines.
sed -n '330,430p' crates/multimodal/src/vision/transforms.rs
echo '---'
# Find all references to fast_image_resize alpha options in the repo.
rg -n "use_alpha|mul_div_alpha|ResizeOptions" crates/multimodal/src -SRepository: lightseekorg/smg
Length of output: 4431
🏁 Script executed:
#!/bin/bash
set -euo pipefail
# Read docs for the fast_image_resize API from the local cargo registry if available.
# This is read-only and only inspects installed documentation/source.
python3 - <<'PY'
import os, glob, textwrap, pathlib, json, sys
home = os.path.expanduser("~")
candidates = []
for base in [
os.path.join(home, ".cargo", "registry", "src"),
os.path.join(home, ".cargo", "registry", "cache"),
]:
if os.path.exists(base):
candidates.append(base)
print("\n".join(candidates))
PYRepository: lightseekorg/smg
Length of output: 155
Fix the doc comment API name. use_alpha(true) is fine here; the comment should refer to ResizeOptions::use_alpha, not ResizeOptions::mul_div_alpha.
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@crates/multimodal/src/vision/transforms.rs` around lines 366 - 395, Update
the doc comment for resize_straight_alpha to refer to ResizeOptions::use_alpha
instead of ResizeOptions::mul_div_alpha. Leave the implementation and all other
documentation unchanged.
|
Superseded by #1984, which implements the same transparency fix plus the |
1 participant
Description
Problem
Kimi-K3's reference vision processor composites alpha-carrying images over a configured background before the NaViT resize and patchify (
kimi_k3_vision_processing.py#L110-L114):SMG registered
kimi-k3againstKimiK25Processor, which has no such step — it drops the alpha channel viato_rgb8(). Dropping alpha is not the same operation as compositing over a background. A fully transparent PNG pixel usually stores RGB(0,0,0)beneath its alpha, so dropping alpha normalizes it to-1.0— solid black — where the model was trained to see the chessboard the checkpoint describes.Anything served as RGBA reached the encoder with its background inverted: logos, icons, diagrams exported with transparency, screenshots with rounded corners. Opaque JPEG/PNG traffic was unaffected, which is why MMMU and OCRBench did not surface it.
Only the gRPC worker path is affected. On the HTTP/OpenAI proxy path the engine preprocesses, so SMG's Rust never runs.
Solution
Two commits, so the refactor is reviewable separately from the behavior change.
1.
refactor(multimodal)— a provable no-op. K2.5 and K3 share the MoonViT stack and differ only in parameters, but there was nowhere to express a K3-specific difference. The reference factors its NaViT helpers into a sharedmedia_utilsmodule with a thin per-model processor class on top; this mirrors that split using the shapeqwen_vl_basealready provides for the Qwen VL family. A test asserts both processors still produce byte-identical output.2.
fix(multimodal)— the compositing step. All values come from the checkpoint, never hardcoded, matching the reference's fallback to dropping alpha whentransparent_bg_configis absent. Hardcoded checkpoint values are the bug class that caused this incident in the first place.The fill stage is load-bearing, not cosmetic: a chessboard is generated at the resolution of the image it lands on, so
before_resizeandafter_resizeyield different square sizes relative to the content. K3's checkpoint shipsafter_resize.Changes
processors/moonvit.rs(new):MoonVitProcessorBase+MoonVitConfighold the shared resize → pad → normalize → patchify pipeline, moved verbatim out ofkimi_k25.rs.processors/kimi_k25.rs: reduced to a thin wrapper thatDerefs to the base. Declarestransparent_bg: None— its reference class has no transparency handling at all, so a config carrying the keys must not change its output.processors/kimi_k3.rs(new): second thin wrapper,model_name: "kimi-k3", resolving the checkpoint's transparency settings per request.transforms.rs:TransparentBgConfig/TransparentBgPattern/TransparentBgFillStagedeserialize straight from apreprocessor_config.json;fill_transparent_bgblendsa*img + (1-a)*bgwith the reference's truncating uint8 cast.transforms.rs:resize_straight_alpha. Our SIMD resizer premultiplies by alpha by default, dragging RGB toward zero wherever alpha is partial. Pillow convolves channels independently, so compositing after a resize requires the non-premultiplied variant.preprocessor_config.rs: liftstransparent_bg_config/transparent_bg_fill_stageout ofmedia_proc_cfg;transparent_bg()returns them together, since the stage is only meaningful alongside a config.processor.rs:kimi-k3/kimi_k3now resolve toKimiK3Processor.Test Plan
cargo test -p llm-multimodal— 274 lib tests + all integration suites (incl. 81 golden tests) pass, 0 failures. 12 new tests:opaque_images_match_k25_byte_for_bytetransparent_pixels_composite_over_the_configured_background+1.0; before this PR it was-1.0transparency_config_does_not_reach_k25opaque_images_ignore_the_transparency_configfill_stage_paints_the_board_at_its_own_resolutionbefore_resize≠after_resizeunder a 4× downscalechessboard_background_matches_reference_create_chessboard_background, bothsquare_on_top_leftphasesfill_transparent_bg_blends_partial_alphastraight_alpha_resize_keeps_rgba_and_ignores_premultiplytest_parse_kimi_k3_transparency_settingstest_transparency_fill_stage_defaults_without_explicit_keybefore_resize, as the reference doesPlus
fill_transparent_bg_is_identity_without_alpha,fill_transparent_bg_survives_zero_square_size,transparent_bg_config_fills_missing_fields.Before → after for a fully transparent 56×56 input with K3's config, mean = std = 0.5:
-1.0+1.0/+0.569Deliberately out of scope
Kept separate so this PR stays reviewable and bisectable:
in_patch_limit: K3's checkpoint raises it to 65536; SMG uses K2.5's 16384. Large images get ~4× fewer visual tokens than they should (4000×3000 → 4200 instead of 15444). Changes token accounting on both models, so it needs its own PR.Checklist
cargo +nightly fmtpassescargo clippy --all-targets --all-features -- -D warningspassesSummary by CodeRabbit