Skip to content

feat(LoRA): support AI Toolkit LoRA for FLUX #8071

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

keturn
Copy link
Contributor

@keturn keturn commented May 30, 2025

Summary

https://github.com/ostris/ai-toolkit is "an all in one training suite for diffusion models."

and I guess LoRA saved by it don't look exactly like the ones we support already.

Related Issues / Discussions

QA Instructions

Merge Plan

N/A

Checklist

  • The PR has a short but descriptive title, suitable for a changelog
  • Tests added / updated (if applicable)
  • Documentation added / updated (if applicable)
  • Updated What's New copy (if doing a release after this PR)

@github-actions github-actions bot added python PRs that change python files backend PRs that change backend files labels May 30, 2025
@keturn
Copy link
Contributor Author

keturn commented May 30, 2025

Current state: LoRA install but don't run.

@github-actions github-actions bot added the python-tests PRs that change python tests label May 31, 2025
@keturn keturn force-pushed the feat/aitoolkit-lora branch from 43e4b2e to 0ca7a05 Compare June 1, 2025 00:06
@keturn keturn marked this pull request as ready for review June 1, 2025 00:09
software = json.loads(metadata.get("software", "{}"))
except json.JSONDecodeError:
return False
return software.get("name") == "ai-toolkit"
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this might not be what we need, as I have another LoRA here that reports the same metadata 'software': '{"name": "ai-toolkit", "repo": "https://github.com/ostris/ai-toolkit", "version": "0.1.0"}', but it uses transformer.single_transformer_blocks keys instead of diffusion_model.single_blocks keys.

to avoid disrupting already-working LoRA
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backend PRs that change backend files python PRs that change python files python-tests PRs that change python tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant