Skip to content

Conversation

sayakpaul
Copy link
Member

@sayakpaul sayakpaul commented Sep 12, 2025

What does this PR do?

As discussed with @DN6, this is a POC PR to evaluate the idea of a pipeline-specific Mixin class that would hold common methods shared across different task-specific pipelines.

Dhruv, some things to note when you review:

  • There's only a few differences between encode_prompt() of QwenImage and encode_prompt() of QwenImageEdit: https://www.diffchecker.com/z7nJMsP4/. I believe we can at least turn the _get_qwen_prompt_embeds() method into one but readability may be compromised due to that. Hence, I have chosen to have QwenImagePipelineMixin and QwenImageEditPipelineMixin. Both of them subclass from QwenImageMixin, differing only in encode_prompt() and _get_qwen_prompt_embeds(). Kept encode_prompt() varying while unifying the _get_qwen_prompt_embeds() method in QwenImageMixin. Comments are in line.
  • There are few methods that are shared by 2/3 pipelines such as prepare_image() is shared by QwenImageControlNetInpaintPipeline and QwenImageControlNetPipeline but not others. We can choose to move them to pipeline_qwen_utils.py but I am not strongly opinionated on this. Same applies for methods like prepare_mask_latents(). My preference here is to keep them as is for now.

@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

Copy link
Collaborator

@DN6 DN6 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looking good 👍🏽 Still on the fence on whether we use multiple levels of inheritance with QwenImageMixin. But other than that this looks good to me.

Perhaps we try with to do the same with Flux and SDXL to see if some common patterns show up for these Mixins.

return latents


class QwenImagePipelineMixin(QwenImageMixin):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can also add additional Mixins here such as LoRA

@sayakpaul
Copy link
Member Author

Looking good 👍🏽 Still on the fence on whether we use multiple levels of inheritance with QwenImageMixin. But other than that this looks good to me.

With you. I am also on the fence. Since the mixins for QwenImage and QwenImageEdit primarily differ in encode_prompt(), I decided to design it like that. We can still rewrite the encode_prompt() method in a single mixin so that it can deal with the images, but it could complicate things. It will also end up changing the signature of the method, leaving room for breaking changes.

But we can definitely uniformize the _get_qwen_prompt_embeds() method as it's private anyway. WDYT?

Perhaps we try with to do the same with Flux and SDXL to see if some common patterns show up for these Mixins.

Yes, I will work on that.


return split_result

def _get_qwen_prompt_embeds(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Decided to unify it to further reduce LOC. It's a private method anyway, so, we should be safe.



class QwenImagePipelineMixin(QwenImageMixin):
def encode_prompt(
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Only differing encode_prompt() to not introduce any breaking change.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants