fix: handle ModelPatcher-wrapped upscale model (ComfyUI #15063) - #13
Open
cuzelac wants to merge 2 commits into
Open
fix: handle ModelPatcher-wrapped upscale model (ComfyUI #15063)#13cuzelac wants to merge 2 commits into
cuzelac wants to merge 2 commits into
Conversation
- ComfyUI 0.28.0 wraps LatentUpsampler in a ModelPatcher, which is not callable and exposes no .parameters()/.to()/.cpu() - resolve the handle via _resolve_upscale_model(); legacy bare-module inputs still work - delegate residency to load_models_gpu instead of manual .to()/.cpu(), which fought the memory manager - model_dtype() returns None for LatentUpsampler, so fall back to parameter dtype to preserve the bf16 cast Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Claude <noreply@anthropic.com>
|
Confirmed for me / by me: git checkout fix/upsampler-modelpatcher in the forked repo solved the problem for me. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
ComfyUI is converting the latent upsampler to DynamicVRAM, which changes
LATENT_UPSCALE_MODELfrom a barenn.Moduleinto aModelPatcher. LTX Latent Upsampler (Tiled) crashes on builds that have that change. This PR makes the node accept either shape.Please note the version nuance before trying to reproduce — you may not hit this on a released tag (details below).
The crash
The node fails before doing any work, so there's no partial-output workaround on an affected build.
Which builds are affected
This is the part worth checking first, since it determines whether you can reproduce:
f8a3fd9— "upscalers: convert latent_upsampler model to DynamicVram" (#15063, 2026-07-24) — wraps theLatentUpsamplerin aCoreModelPatcher, which becomesModelPatcherDynamicwhen DynamicVRAM is on. This commit is currently on ComfyUI master.So the shape depends on the build, not cleanly on the version string. I hit the crash on a Comfy Desktop install that carried
f8a3fd9while reporting0.28.0; after updating to v0.28.3 the same install went back to a bare module. If you test on v0.28.3 you will not see the crash — that doesn't mean it isn't real, it means the change hasn't reached your tag yet.Separately, this isn't purely a future concern: even on builds that do have
f8a3fd9, the Hunyuan 720p/1080p branches ofLatentUpscaleModelLoaderstill return a bare module. Both shapes are reachable simultaneously.Root cause
From
comfy_extras/nodes_hunyuan.pyon master, the LTX branch of the loader:A
ModelPatcheris not callable and exposes no.parameters(),.to(),.cpu(), or.state_dict(). The node relied on all four, so line 106 is only the first of six break points — the others aremodule_size(upscale_model),upscale_model.to(device), the threeupscale_model(...)forward calls, andupscale_model.cpu().What changed
All access goes through a new
_resolve_upscale_model()helper returning(module, device, dtype, load_fn, unload_fn):.model, uses.load_device, delegates residency tomodel_management.load_models_gpu.free_memory+.to(device)/.cpu().Detection is
hasattr(upscale_model, "model") and hasattr(upscale_model, "load_device").load_deviceis a ModelPatcher-ism thatLatentUpsampler(a plainnn.Module) doesn't carry, so it can't misfire.The bare-module path is behaviourally identical to the current code — same dtype source, same device, same total memory reservation, same load/unload calls. No minimum ComfyUI version bump is needed, and nothing changes for users on current tags.
The tiling and cosine-window blending math is untouched.
Two things worth your attention
1. Deliberate divergence from stock on dtype. On the patcher path,
patcher.model_dtype()returnsNoneforLatentUpsampler(noget_dtypemethod). ComfyUI's own node passes that straight into.to(dtype=None), a no-op leaving latents at input dtype. This PR instead falls back tonext(module.parameters()).dtype, preserving the bf16 cast this node has always applied to the un-normalized latents. That's intentional, not an oversight.2. On the patcher path the model stays resident between runs. Dropping the manual
.cpu()is a behaviour change beyond the crash fix — manual eviction fights the memory manager, andload_models_gpuhandles offload itself. The bare-module path keeps the original explicit eviction. Happy to revisit if you'd rather force eviction in both.Verification
Live: hit the original crash on every run, in a two-pass LTX i2v workflow (RTX 5090, PyTorch 2.10.0+cu130) on a build carrying
f8a3fd9; the node completes with this patch.Isolated, against a
CoreModelPatcherwrapping a stand-in 2× upsampler:dtype=bfloat16,(1,4,2,6,6)→(1,4,2,12,12)dtype=float32, correct shape(1,4,2,40,24)→(1,4,2,80,48), weight accumulatormin=max=1.0000(1,4,2,8,8)→(1,4,2,16,16)The weight accumulator at exactly 1.0 confirms the blending math is unaffected.
Coverage caveat: the isolated test exercises
ModelPatcher, notModelPatcherDynamic— the dynamic subclass only instantiates under the full DynamicVRAM startup path. Every attribute used here is inherited unchanged fromModelPatcher, and the live run above was onModelPatcherDynamic(that's what the traceback names), so both are covered between the two.Commits
Split so you can take the fix without the release metadata:
fix(upsampler): handle ModelPatcher-wrapped upscale model— the fix, plus a compatibility note inCLAUDE.mdchore: bump version to 1.9.6—__init__.py,pyproject.toml, README version historyDrop the second commit if you'd rather assign the version yourself; I have no attachment to
1.9.6.