Build docker container for ROCm #1595

fenfir · 2024-01-15T21:38:35Z

Description

This PR changes the Dockerfile to build for HipBLAS

Notes for Reviewers

I'm not sure about referencing the rocm/dev-ubuntu-22.04:6.0-complete container, I couldn't find anything publicly available that included the libs. I have one locally that I will look at publishing.

Signed commits

[ X] Yes, I signed my commits.

netlify · 2024-01-15T21:38:40Z

✅ Deploy Preview for localai canceled.

Name	Link
🔨 Latest commit	`752033f`
🔍 Latest deploy log	https://app.netlify.com/sites/localai/deploys/65ce423513be02000848ee71

fenfir · 2024-01-15T21:46:43Z

I also updated the go builder to from bullseye to bookworm , there was a glibc dependency for ROCm that didn't work on bullseye

jamiemoller · 2024-01-16T09:17:30Z

does this actually build?
looking at #1592 atm

fenfir · 2024-01-16T15:19:20Z

It works for me on llama.cpp at least, I havent tested the rest yet. It doesn't include stablediffusion IIRC. I'm still testing is why I still have it WIP

hiepxanh · 2024-01-19T03:35:39Z

great work, I love to see this one

jamiemoller · 2024-01-19T04:44:41Z

it does build however I still seem to have the same issue as I had when following the arch build instructions in the localai wiki. where the model will load without a problem, then for some reason the api query is never executed and the model just sits there in vram, this issue also persists when running this method in CPU mode also :|

fenfir · 2024-01-23T19:17:52Z

This works with stablediffusion now. The diffusers backend is still giving me trouble

fenfir · 2024-01-24T00:03:47Z

This works with diffusers now

fenfir · 2024-01-24T00:11:24Z

@mudler I removed the WIP header,. Is there a good way to test this? I did some manual testing and it seems to work.

Dockerfile

mudler · 2024-01-24T18:45:42Z

@mudler I removed the WIP header,. Is there a good way to test this? I did some manual testing and it seems to work.

for acceleration tests we have a specific workflow but for now is disabled as it waits for HW (#1252 ).

However that doesn't cover container e2e tests (which are still manual) for all the backends

Dockerfile

mudler · 2024-02-08T17:15:12Z

@fenfir with #1689 I'm actually making the base image configurable - maybe it is now as easy as pointing to the ROCm images? any chance you can give that a shot?

mudler · 2024-02-14T22:39:55Z

@fenfir I've updated the PR and based the Dockerfile directly on the rocm images, can you try it out?

Signed-off-by: mudler <[email protected]>

mudler · 2024-02-16T14:08:35Z

ok, let's give a shot to the images on master

….0 by renovate (#18546) This PR contains the following updates: | Package | Update | Change | |---|---|---| | [docker.io/localai/localai](https://togithub.com/mudler/LocalAI) | minor | `v2.8.2-cublas-cuda11-ffmpeg-core` -> `v2.9.0-cublas-cuda11-ffmpeg-core` | | [docker.io/localai/localai](https://togithub.com/mudler/LocalAI) | minor | `v2.8.2-cublas-cuda11-core` -> `v2.9.0-cublas-cuda11-core` | | [docker.io/localai/localai](https://togithub.com/mudler/LocalAI) | minor | `v2.8.2-cublas-cuda12-ffmpeg-core` -> `v2.9.0-cublas-cuda12-ffmpeg-core` | | [docker.io/localai/localai](https://togithub.com/mudler/LocalAI) | minor | `v2.8.2-cublas-cuda12-core` -> `v2.9.0-cublas-cuda12-core` | | [docker.io/localai/localai](https://togithub.com/mudler/LocalAI) | minor | `v2.8.2-ffmpeg-core` -> `v2.9.0-ffmpeg-core` | | [docker.io/localai/localai](https://togithub.com/mudler/LocalAI) | minor | `v2.8.2` -> `v2.9.0` | --- > [!WARNING] > Some dependencies could not be looked up. Check the Dependency Dashboard for more information. --- ### Release Notes <details> <summary>mudler/LocalAI (docker.io/localai/localai)</summary> ### [`v2.9.0`](https://togithub.com/mudler/LocalAI/releases/tag/v2.9.0) [Compare Source](https://togithub.com/mudler/LocalAI/compare/v2.8.2...v2.9.0) This release brings many enhancements, fixes, and a special thanks to the community for the amazing work and contributions! We now have sycl images for Intel GPUs, ROCm images for AMD GPUs,and much more: - You can find the AMD GPU images tags between the container images available - look for `hipblas`. For example, [master-hipblas-ffmpeg-core](https://quay.io/repository/go-skynet/local-ai/tag/master-hipblas-ffmpeg-core). Thanks to [@fenfir](https://togithub.com/fenfir) for this nice contribution! - Intel GPU images are tagged with `sycl`. You can find images with two flavors, sycl-f16 and sycl-f32 respectively. For example, [master-sycl-f16](https://quay.io/repository/go-skynet/local-ai/tag/master-sycl-f16-core). Work is in progress to support also diffusers and transformers on Intel GPUs. - Thanks to [@christ66](https://togithub.com/christ66) first efforts in supporting the Assistant API were made, and we are planning to support the Assistant API! Stay tuned for more! - Now LocalAI supports the Tools API endpoint - it also supports the (now deprecated) functions API call as usual. We now also have support for SSE with function calling. See [https://github.com/mudler/LocalAI/pull/1726](https://togithub.com/mudler/LocalAI/pull/1726) for more - Support for Gemma models - did you hear? Google released OSS models and LocalAI supports it already! - Thanks to [@dave-gray101](https://togithub.com/dave-gray101) in [https://github.com/mudler/LocalAI/pull/1728](https://togithub.com/mudler/LocalAI/pull/1728) to put efforts in refactoring parts of the code - we are going to support soon more ways to interface with LocalAI, and not only restful api! ##### Support the project First off, a massive thank you to each and every one of you who've chipped in to squash bugs and suggest cool new features for LocalAI. Your help, kind words, and brilliant ideas are truly appreciated - more than words can say! And to those of you who've been heros, giving up your own time to help out fellow users on Discord and in our repo, you're absolutely amazing. We couldn't have asked for a better community. Just so you know, LocalAI doesn't have the luxury of big corporate sponsors behind it. It's all us, folks. So, if you've found value in what we're building together and want to keep the momentum going, consider showing your support. A little shoutout on your favorite social platforms using [@LocalAI_OSS](https://twitter.com/LocalAI_API) and [@mudler_it](https://twitter.com/mudler_it) or joining our sponsorship program can make a big difference. Also, if you haven't yet joined our Discord, come on over! Here's the link: https://discord.gg/uJAeKSAGDy Every bit of support, every mention, and every star adds up and helps us keep this ship sailing. Let's keep making LocalAI awesome together! Thanks a ton, and here's to more exciting times ahead with LocalAI! 🚀 ##### What's Changed ##### Bug fixes 🐛 - Add TTS dependency for cuda based builds fixes [#1727](https://togithub.com/mudler/LocalAI/issues/1727) by [@blob42](https://togithub.com/blob42) in [https://github.com/mudler/LocalAI/pull/1730](https://togithub.com/mudler/LocalAI/pull/1730) ##### Exciting New Features 🎉 - Build docker container for ROCm by [@fenfir](https://togithub.com/fenfir) in [https://github.com/mudler/LocalAI/pull/1595](https://togithub.com/mudler/LocalAI/pull/1595) - feat(tools): support Tool calls in the API by [@mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1715](https://togithub.com/mudler/LocalAI/pull/1715) - Initial implementation of upload files api. by [@christ66](https://togithub.com/christ66) in [https://github.com/mudler/LocalAI/pull/1703](https://togithub.com/mudler/LocalAI/pull/1703) - feat(tools): Parallel function calling by [@mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1726](https://togithub.com/mudler/LocalAI/pull/1726) - refactor: move part of api packages to core by [@dave-gray101](https://togithub.com/dave-gray101) in [https://github.com/mudler/LocalAI/pull/1728](https://togithub.com/mudler/LocalAI/pull/1728) - deps(llama.cpp): update, support Gemma models by [@mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1734](https://togithub.com/mudler/LocalAI/pull/1734) ##### 👒 Dependencies - deps(llama.cpp): update by [@mudler](https://togithub.com/mudler) in [https://github.com/mudler/LocalAI/pull/1714](https://togithub.com/mudler/LocalAI/pull/1714) - ⬆️ Update ggerganov/llama.cpp by [@localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1740](https://togithub.com/mudler/LocalAI/pull/1740) ##### Other Changes - ⬆️ Update docs version mudler/LocalAI by [@localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1718](https://togithub.com/mudler/LocalAI/pull/1718) - ⬆️ Update ggerganov/llama.cpp by [@localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1705](https://togithub.com/mudler/LocalAI/pull/1705) - Update README.md by [@lunamidori5](https://togithub.com/lunamidori5) in [https://github.com/mudler/LocalAI/pull/1739](https://togithub.com/mudler/LocalAI/pull/1739) - ⬆️ Update ggerganov/llama.cpp by [@localai-bot](https://togithub.com/localai-bot) in [https://github.com/mudler/LocalAI/pull/1750](https://togithub.com/mudler/LocalAI/pull/1750) ##### New Contributors - [@fenfir](https://togithub.com/fenfir) made their first contribution in [https://github.com/mudler/LocalAI/pull/1595](https://togithub.com/mudler/LocalAI/pull/1595) - [@christ66](https://togithub.com/christ66) made their first contribution in [https://github.com/mudler/LocalAI/pull/1703](https://togithub.com/mudler/LocalAI/pull/1703) - [@blob42](https://togithub.com/blob42) made their first contribution in [https://github.com/mudler/LocalAI/pull/1730](https://togithub.com/mudler/LocalAI/pull/1730) **Full Changelog**: mudler/LocalAI@v2.8.2...v2.9.0 </details> --- ### Configuration 📅 **Schedule**: Branch creation - "before 10pm on monday" in timezone Europe/Amsterdam, Automerge - At any time (no schedule defined). 🚦 **Automerge**: Enabled. ♻ **Rebasing**: Whenever PR becomes conflicted, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about these updates again. --- - [ ] If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://togithub.com/renovatebot/renovate).

fenfir force-pushed the docker_build_rocm branch from 94326aa to 3797117 Compare January 15, 2024 21:41

fenfir changed the title ~~[WIP] Dockerfile changes to build for ROCm~~ Dockerfile changes to build for ROCm Jan 24, 2024

fenfir changed the title ~~Dockerfile changes to build for ROCm~~ Build docker container for ROCm Jan 24, 2024

mudler reviewed Jan 24, 2024

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

mudler reviewed Jan 24, 2024

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

mudler added roadmap high prio labels Jan 31, 2024

mudler assigned fenfir Jan 31, 2024

mudler added the enhancement New feature or request label Jan 31, 2024

mudler mentioned this pull request Jan 31, 2024

feat(llama.cpp): Vulkan, Kompute, SYCL #1647

Open

4 tasks

mudler reviewed Feb 6, 2024

View reviewed changes

Dockerfile Outdated Show resolved Hide resolved

fenfir and others added 5 commits February 14, 2024 23:29

Dockerfile changes to build for ROCm

36f5854

Adjust linker flags for ROCm

53352c0

Update conda env for diffusers and transformers to use ROCm pytorch

8fbb453

Update transformers conda env for ROCm

2a8477c

ci: build hipblas images

e70b386

mudler force-pushed the docker_build_rocm branch from eff6fb2 to e70b386 Compare February 14, 2024 22:39

fixup rebase

97fd421

mudler added 2 commits February 15, 2024 17:53

use self-hosted

853d8d1

Signed-off-by: mudler <[email protected]>

specify LD_LIBRARY_PATH only when BUILD_TYPE=hipblas

752033f

mudler mentioned this pull request Feb 15, 2024

Better Support for AMD and ROCM via docker containers. #1592

Closed

mudler self-assigned this Feb 15, 2024

mudler approved these changes Feb 16, 2024

View reviewed changes

mudler merged commit fb0a4c5 into mudler:master Feb 16, 2024
19 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build docker container for ROCm #1595

Build docker container for ROCm #1595

fenfir commented Jan 15, 2024

netlify bot commented Jan 15, 2024 •

edited

Loading

fenfir commented Jan 15, 2024

jamiemoller commented Jan 16, 2024

fenfir commented Jan 16, 2024

hiepxanh commented Jan 19, 2024

jamiemoller commented Jan 19, 2024

fenfir commented Jan 23, 2024

fenfir commented Jan 24, 2024

fenfir commented Jan 24, 2024

mudler commented Jan 24, 2024 •

edited

Loading

mudler commented Feb 8, 2024

mudler commented Feb 14, 2024

mudler commented Feb 16, 2024

Build docker container for ROCm #1595

Build docker container for ROCm #1595

Conversation

fenfir commented Jan 15, 2024

netlify bot commented Jan 15, 2024 • edited Loading

✅ Deploy Preview for localai canceled.

fenfir commented Jan 15, 2024

jamiemoller commented Jan 16, 2024

fenfir commented Jan 16, 2024

hiepxanh commented Jan 19, 2024

jamiemoller commented Jan 19, 2024

fenfir commented Jan 23, 2024

fenfir commented Jan 24, 2024

fenfir commented Jan 24, 2024

mudler commented Jan 24, 2024 • edited Loading

mudler commented Feb 8, 2024

mudler commented Feb 14, 2024

mudler commented Feb 16, 2024

netlify bot commented Jan 15, 2024 •

edited

Loading

mudler commented Jan 24, 2024 •

edited

Loading