Skip to content

Add FLUX Fill support #7780

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 7 commits into from
Mar 19, 2025
Merged

Add FLUX Fill support #7780

merged 7 commits into from
Mar 19, 2025

Conversation

RyanJDick
Copy link
Contributor

@RyanJDick RyanJDick commented Mar 13, 2025

Summary

This PR adds support for FLUX Fill (https://huggingface.co/black-forest-labs/FLUX.1-Fill-dev).

Note: FLUX Fill expects a guidance value of ~30.0.

Example

Workflow: flux_fill.json
image

Image 1
Original
Image 1
Mask
Image 1
Result

Related Issues / Discussions

N/A

QA Instructions

  • Test FLUX Fill model install from Starter Models.
  • Test FLUX Fill happy path workflow.
  • Test that FLUX dev model install is still correctly probed.
  • Test that an error is raised if a FLUX Fill model is used with a FLUX Control LoRA
  • Compatibility with other features. These all technically 'work', but none work very well.
    • 'Standard' inpainting + FLUX Fill inpainting simultaneously
    • FLUX Fill and FLUX Redux
    • FLUX Fill and controlnet
    • FLUX Fill and ip_adapter

Merge Plan

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

@github-actions github-actions bot added python PRs that change python files invocations PRs that change invocations backend PRs that change backend files labels Mar 13, 2025
@github-actions github-actions bot added the frontend PRs that change frontend files label Mar 14, 2025
@RyanJDick RyanJDick marked this pull request as ready for review March 14, 2025 14:49
@psychedelicious psychedelicious merged commit 9cc2232 into main Mar 19, 2025
15 checks passed
@psychedelicious psychedelicious deleted the ryan/flux-fill branch March 19, 2025 03:45
psychedelicious added a commit that referenced this pull request Mar 23, 2025
In #7780 we added FLUX Fill support, and needed the probe to be able to distinguish between "normal" FLUX models and FLUX Fill models.

Logic was added to the probe to check a particular state dict key (input channels), which should be 384 for FLUX Fill and 64 for other FLUX models.

The new logic was stricter and instead of falling back on the "normal" variant, it raised when an unexpected value for input channels was detected.

This caused failures to probe for BNB-NF4 quantized FLUX Dev/Schnell, which apparently only have 1 input channel.

After checking a variety of FLUX models, I loosened the strictness of the variant probing logic to only special-case the new FLUX Fill model, and otherwise fall back to returning the "normal" variant. This better matches the old behaviour and fixes the import errors.

Closes #7822
psychedelicious added a commit that referenced this pull request Mar 24, 2025
In #7780 we added FLUX Fill support, and needed the probe to be able to distinguish between "normal" FLUX models and FLUX Fill models.

Logic was added to the probe to check a particular state dict key (input channels), which should be 384 for FLUX Fill and 64 for other FLUX models.

The new logic was stricter and instead of falling back on the "normal" variant, it raised when an unexpected value for input channels was detected.

This caused failures to probe for BNB-NF4 quantized FLUX Dev/Schnell, which apparently only have 1 input channel.

After checking a variety of FLUX models, I loosened the strictness of the variant probing logic to only special-case the new FLUX Fill model, and otherwise fall back to returning the "normal" variant. This better matches the old behaviour and fixes the import errors.

Closes #7822
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend PRs that change backend files frontend PRs that change frontend files invocations PRs that change invocations python PRs that change python files
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants