Skip to content

Use pre-built wheel for flash-attn on Linux & Resolve version mismatch for torchaudio#1848

Open
2vibeornot2vibe wants to merge 3 commits into
deepbeepmeep:mainfrom
2vibeornot2vibe:patch-2
Open

Use pre-built wheel for flash-attn on Linux & Resolve version mismatch for torchaudio#1848
2vibeornot2vibe wants to merge 3 commits into
deepbeepmeep:mainfrom
2vibeornot2vibe:patch-2

Conversation

@2vibeornot2vibe

@2vibeornot2vibe 2vibeornot2vibe commented May 30, 2026

Copy link
Copy Markdown

Since this project uses Python 3.11 + CUDA 13.0 + PyTorch 2.10, and there's no pre-built flash-attn wheel for this combination on PyPI, it will default to building the package, which will then fail due to build isolation issues. Even with build isolation disabled, it will still take a very long time. The windows config already uses a pre-built wheel, so this PR essentially does the same for Linux.

@2vibeornot2vibe

Copy link
Copy Markdown
Author

I added two more patches:

  1. The CUDA 13.0 config only specified the version number for torch, but not for torchvision and torchaudio, which now causes version mismatch on Linux. This PR specifies the version number for all 3 packages to prevent that from occurring in the future.
  2. I also updated the CUDA 12.8 config to torch 2.10.0, which is the same as the Dockerfile. I've already tested it on my Tesla V100 setup and it works well.

@2vibeornot2vibe 2vibeornot2vibe changed the title Use pre-built wheel for flash-attn on Linux Use pre-built wheel for flash-attn on Linux & Resolve version mismatch for torchaudio May 30, 2026
@deepbeepmeep

Copy link
Copy Markdown
Owner

thx but you cant replace pytorch 2.7 with pytorch 2.10 as it invalidates all the precompiled wheels for the various kernels provided with wangp: there is no combination python 3.10 / pytorch 2.10 / cuda 12.8.
I did that change nevertheless:
"cmd": "torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 --index-url https://download.pytorch.org/whl/cu130"

@2vibeornot2vibe

Copy link
Copy Markdown
Author

thx but you cant replace pytorch 2.7 with pytorch 2.10 as it invalidates all the precompiled wheels for the various kernels provided with wangp: there is no combination python 3.10 / pytorch 2.10 / cuda 12.8. I did that change nevertheless: "cmd": "torch==2.10.0 torchvision==0.25.0 torchaudio==2.10.0 --index-url https://download.pytorch.org/whl/cu130"

Thanks for the reply. The reason I updated the CUDA 12.8 config to pytorch 2.10 was that I saw only the GTX_10 profile (aka pre-Turin cards) used it. And this profile doesn't install any kernels (they aren't supported anyways).
As for the python 3.10 conflict, my third commit did change the default python version for the GTX_10 profile to 3.11, which does have a pytorch 2.10 wheel. This combination works very well on Tesla V100 with sdpa attention.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants