v2.20.1- P2P Global community pools! #3363

mudler · 2024-08-23T08:29:53Z

mudler
Aug 23, 2024
Maintainer

TL;DR

🌍 Explorer & Community: Explore global community pools at explorer.localai.io
👀 Demo instance available: Test out LocalAI at demo.localai.io
🤗 Integration: Hugging Face Local apps now include LocalAI
🐛 Bug Fixes: Diffusers and hipblas issues resolved
🎨 New Feature: FLUX-1 image generation support
🏎️ Strict Mode: Stay compliant with OpenAI’s latest API changes
💪 Multiple P2P Clusters: Run multiple clusters within the same network
🧪 Deprecation Notice: gpt4all.cpp and petals backends deprecated

🌍 Explorer and Global Community Pools

Now you can share your LocalAI instance with the global community or explore available instances by visiting explorer.localai.io. This decentralized network powers our demo instance, creating a truly collaborative AI experience.

How It Works

Using the Explorer, you can easily share or connect to clusters. For detailed instructions on creating new clusters or connecting to existing ones, check out our documentation.

👀 Demo Instance Now Available

Curious about what LocalAI can do? Dive right in with our live demo at demo.localai.io! Thanks to our generous sponsors, this instance is publicly available and configured via peer-to-peer (P2P) networks. If you'd like to connect, follow the instructions here.

🤗 Hugging Face Integration

I am excited to announce that LocalAI is now integrated within Hugging Face’s local apps! This means you can select LocalAI directly within Hugging Face to build and deploy models with the power and flexibility of our platform. Experience seamless integration with a single click!

This integration was made possible through this PR.

🎨 FLUX-1 Image Generation Support

FLUX-1 lands in LocalAI! With this update, LocalAI can now generate stunning images using FLUX-1, even in federated mode. Whether you're experimenting with new designs or creating production-quality visuals, FLUX-1 has you covered.

Try it out at demo.localai.io and see what LocalAI + FLUX-1 can do!

🐛 Diffusers and hipblas Fixes

Great news for AMD users! If you’ve encountered issues with the Diffusers backend or hipblas, those bugs have been resolved. We’ve transitioned to uv for managing Python dependencies, ensuring a smoother experience. For more details, check out Issue #1592.

🏎️ Strict Mode for API Compliance

To stay up to date with OpenAI’s latest changes, now LocalAI have support as well for Strict Mode ( https://openai.com/index/introducing-structured-outputs-in-the-api/ ). This new feature ensures compatibility with the most recent API updates, enforcing stricter JSON outputs using BNF grammar rules.

To activate, simply set strict: true in your API calls, even if it’s disabled in your configuration.

Key Notes:

Setting strict: true enables grammar enforcement, even if disabled in your config.
If format_type is set to json_schema, BNF grammars will be automatically generated from the schema.

🛑 Disable Gallery

Need to streamline your setup? You can now disable the gallery endpoint using LOCALAI_DISABLE_GALLERY_ENDPOINT. For more options, check out the full list of commands with --help.

🌞 P2P and Federation Enhancements

Several enhancements have been made to improve your experience with P2P and federated clusters:

Load Balancing by Default: This feature is now enabled by default (disable it with LOCALAI_RANDOM_WORKER if needed).
Target Specific Workers: Directly target workers in federated mode using LOCALAI_TARGET_WORKER.

💪 Run Multiple P2P Clusters in the Same Network

You can now run multiple clusters within the same network by specifying a network ID via CLI. This allows you to logically separate clusters while using the same shared token. Just set LOCALAI_P2P_NETWORK_ID to a UUID that matches across instances.

Please note, while this offers segmentation, it’s not fully secure—anyone with the network token can view available services within the network.

🧪 Deprecation Notice: `gpt4all.cpp` and `petals` Backends

As we continue to evolve, we are officially deprecating the gpt4all.cpp and petals backends. The newer llama.cpp offers a superior set of features and better performance, making it the preferred choice moving forward.

From this release onward, gpt4all models in ggml format are no longer compatible. Additionally, the petals backend has been deprecated as well. LocalAI’s new P2P capabilities now offer a comprehensive replacement for these features.

What's Changed

New Contributors

@jermeyhu made their first contribution in fix(docs): Refer to the OpenAI documentation to update the openai-functions docu… #3317

Full Changelog: v2.19.4...v2.20.0

This discussion was created from the release v2.20.1.

yakovw · 2024-08-25T05:13:42Z

yakovw
Aug 25, 2024

Everything looks wonderful and good, and I really want to download a new version and use it
What is the format of the files? I can't find an EXE installation file anywhere, why?

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

v2.20.1- P2P Global community pools! #3363

{{title}}

Replies: 1 comment

{{title}}

Select a reply

v2.20.1- P2P Global community pools! #3363

mudler Aug 23, 2024 Maintainer

TL;DR

🌍 Explorer and Global Community Pools

How It Works

👀 Demo Instance Now Available

🤗 Hugging Face Integration

🎨 FLUX-1 Image Generation Support

🐛 Diffusers and hipblas Fixes

🏎️ Strict Mode for API Compliance

Key Notes:

🛑 Disable Gallery

🌞 P2P and Federation Enhancements

💪 Run Multiple P2P Clusters in the Same Network

🧪 Deprecation Notice: gpt4all.cpp and petals Backends

What's Changed

Breaking Changes 🛠

Bug fixes 🐛

Exciting New Features 🎉

🧠 Models

📖 Documentation and examples

👒 Dependencies

Other Changes

New Contributors

Replies: 1 comment

yakovw Aug 25, 2024

mudler
Aug 23, 2024
Maintainer

🧪 Deprecation Notice: `gpt4all.cpp` and `petals` Backends

yakovw
Aug 25, 2024