LocalAI v3.4.0 — Release Notes & Engineering Impact

<h1 align="center"> <br> <img height="300" src="https://raw.githubusercontent.com/mudler/LocalAI/refs/heads/master/core/http/static/logo.png"> <br> <br> 🚀 LocalAI 3.4.0 </h1> ## What’s New in LocalAI 3.4.0 🎉 - WebUI improvements: now size can be set during image generation - New backends: [KittenTTS](github.com/KittenML/KittenTTS), [kokoro](https://github.com/hexgrad/kokoro) and [dia](https://github.com/nari-labs/dia) now are available as backends and models can be installed directly from the gallery Note: these backends needs to be warmed up during the first call to download the model files. - Support for reasoning effort in the OpenAI chat completion - Diffusers backend now is available for l4t images and devices - During backend installation from the CLI can be supplied alias and name (`--alias` and --name`) to override configurations - Backends now can be sideloaded from the system: you can drag-and-drop the backends in the backends folder and they will just work! ## The Complete Local Stack for Privacy-First AI <table> <tr> <td width="30%" valign="top" align="center"> <a href="https://github.com/mudler/LocalAI"> <img src="https://raw.githubusercontent.com/mudler/LocalAI/refs/heads/master/core/http/static/logo.png" width="200" alt="LocalAI Logo"> <h3>LocalAI</h3> </a> </td> <td width="70%" valign="top"> <p>The free, Open Source OpenAI alternative. Acts as a drop-in replacement REST API compatible with OpenAI specifications for local AI inferencing. No GPU required.</p> <p><em>Link:</em> <a href="https://github.com/mudler/LocalAI">https://github.com/mudler/LocalAI</a></p> </td> </tr> <tr> <td width="30%" valign="top" align="center"> <a href="https://github.com/mudler/LocalAGI"> <img src="https://raw.githubusercontent.com/mudler/LocalAGI/refs/heads/main/webui/react-ui/public/logo_2.png" width="200" alt="LocalAGI Logo"> <h3>LocalAGI</h3> </a> </td> <td width="70%" valign="top"> <p>A powerful Local AI agent management platform. Serves as a drop-in replacement for OpenAI's Responses API, supercharged with advanced agentic capabilities and a no-code UI.</p> <p><em>Link:</em> <a href="https://github.com/mudler/LocalAGI">https://github.com/mudler/LocalAGI</a></p> </td> </tr> <tr> <td width="30%" valign="top" align="center"> <a href="https://github.com/mudler/LocalRecall"> <img src="https://raw.githubusercontent.com/mudler/LocalRecall/refs/heads/main/static/localrecall_horizontal.png" width="200" alt="LocalRecall Logo"> <h3>LocalRecall</h3> </a> </td> <td width="70%" valign="top"> <p>A RESTful API and knowledge base management system providing persistent memory and storage capabilities for AI agents. Designed to work alongside LocalAI and LocalAGI.</p> <p><em>Link:</em> <a href="https://github.com/mudler/LocalRecall">https://github.com/mudler/LocalRecall</a></p> </td> </tr> </table> ## Thank you! ❤️ A massive **THANK YOU** to our incredible community and our sponsors! LocalAI has over **34,500 stars**, and LocalAGI has already rocketed past **1k+ stars**! As a reminder, LocalAI is real FOSS (Free and Open Source Software) and its sibling projects are community-driven and not backed by VCs or a company. We rely on contributors donating their spare time and our sponsors to provide us the hardware! If you love open-source, privacy-first AI, please consider starring the repos, contributing code, reporting bugs, or spreading the word! 👉 **Check out the reborn LocalAGI v2 today:** [https://github.com/mudler/LocalAGI](https://github.com/mudler/LocalAGI) ## Full changelog :point_down: <details> <summary> :point_right: Click to expand :point_left: </summary> ## What's Changed ### Bug fixes :bug: * fix(llama.cpp): do not default to linear rope by @mudler in https://github.com/mudler/LocalAI/pull/5982 ### Exciting New Features 🎉 * feat(webui): allow to specify image size by @mudler in https://github.com/mudler/LocalAI/pull/5976 * feat(backends): add KittenTTS by @mudler in https://github.com/mudler/LocalAI/pull/5977 * feat(kokoro): complete kokoro integration by @mudler in https://github.com/mudler/LocalAI/pull/5978 * feat: add reasoning effort and metadata to template by @mudler in https://github.com/mudler/LocalAI/pull/5981 * feat(transformers): add support to Dia by @mudler in https://github.com/mudler/LocalAI/pull/5991 * feat(diffusers): add builds for nvidia-l4t by @mudler in https://github.com/mudler/LocalAI/pull/6004 * feat(backends install): allow to specify name and alias during manual installation by @mudler in https://github.com/mudler/LocalAI/pull/5971 ### 🧠 Models * chore(models): add gpt-oss-20b by @mudler in https://github.com/mudler/LocalAI/pull/5973 * chore(models): add gpt-oss-120b by @mudler in https://github.com/mudler/LocalAI/pull/5974 * feat(models): add support to qwen-image by @mudler in https://github.com/mudler/LocalAI/pull/5975 * chore(model gallery): add openai_gpt-oss-20b-neo by @mudler in https://github.com/mudler/LocalAI/pull/5986 * fix(harmony): improve template by adding reasoning effort and system_prompt by @mudler in https://github.com/mudler/LocalAI/pull/5985 * chore(model gallery): add qwen_qwen3-4b-instruct-2507 by @mudler in https://github.com/mudler/LocalAI/pull/5987 * chore(model gallery): add qwen_qwen3-4b-thinking-2507 by @mudler in https://github.com/mudler/LocalAI/pull/5988 * chore(model gallery): add huihui-ai_huihui-gpt-oss-20b-bf16-abliterated by @mudler in https://github.com/mudler/LocalAI/pull/5995 * chore(model gallery): add openai-gpt-oss-20b-abliterated-uncensored-neo-imatrix by @mudler in https://github.com/mudler/LocalAI/pull/5996 * chore(model gallery): add tarek07_nomad-llama-70b by @mudler in https://github.com/mudler/LocalAI/pull/5997 * chore: add Dia to the model gallery, fix backend by @mudler in https://github.com/mudler/LocalAI/pull/5998 * chore(model gallery): add chatterbox by @mudler in https://github.com/mudler/LocalAI/pull/5999 * chore(model gallery): add outetts by @mudler in https://github.com/mudler/LocalAI/pull/6000 * chore(model gallery): add impish_nemo_12b by @mudler in https://github.com/mudler/LocalAI/pull/6007 * chore(model-gallery): :arrow_up: update checksum by @localai-bot in https://github.com/mudler/LocalAI/pull/6010 ### 👒 Dependencies * chore(deps): bump edgevpn by @mudler in https://github.com/mudler/LocalAI/pull/6001 ### Other Changes * docs: :arrow_up: update docs version mudler/LocalAI by @localai-bot in https://github.com/mudler/LocalAI/pull/5967 * chore: :arrow_up: Update ggml-org/llama.cpp to `41613437ffee0dbccad684fc744788bc504ec213` by @localai-bot in https://github.com/mudler/LocalAI/pull/5968 * chore(deps): bump torch and diffusers by @mudler in https://github.com/mudler/LocalAI/pull/5970 * chore(deps): bump torch and sentence-transformers by @mudler in https://github.com/mudler/LocalAI/pull/5969 * chore: :arrow_up: Update ggml-org/llama.cpp to `fd1234cb468935ea087d6929b2487926c3afff4b` by @localai-bot in https://github.com/mudler/LocalAI/pull/5972 * chore: :arrow_up: Update ggml-org/llama.cpp to `e725a1a982ca870404a9c4935df52466327bbd02` by @localai-bot in https://github.com/mudler/LocalAI/pull/5984 * feat(swagger): update swagger by @localai-bot in https://github.com/mudler/LocalAI/pull/5983 * chore: :arrow_up: Update ggml-org/llama.cpp to `a0552c8beef74e843bb085c8ef0c63f9ed7a2b27` by @localai-bot in https://github.com/mudler/LocalAI/pull/5992 * chore: :arrow_up: Update ggml-org/whisper.cpp to `4245c77b654cd384ad9f53a4a302be716b3e5861` by @localai-bot in https://github.com/mudler/LocalAI/pull/5993 * docs: update links in documentation by @lnnt in https://github.com/mudler/LocalAI/pull/5994 * chore: :arrow_up: Update ggml-org/llama.cpp to `cd6983d56d2cce94ecb86bb114ae8379a609073c` by @localai-bot in https://github.com/mudler/LocalAI/pull/6003 * fix(l4t-diffusers): add sentencepiece by @mudler in https://github.com/mudler/LocalAI/pull/6005 * chore: :arrow_up: Update ggml-org/llama.cpp to `79c1160b073b8148a404f3dd2584be1606dccc66` by @localai-bot in https://github.com/mudler/LocalAI/pull/6006 * chore: :arrow_up: Update ggml-org/whisper.cpp to `b02242d0adb5c6c4896d59ac86d9ec9fe0d0fe33` by @localai-bot in https://github.com/mudler/LocalAI/pull/6009 * chore: :arrow_up: Update ggml-org/llama.cpp to `be48528b068111304e4a0bb82c028558b5705f05` by @localai-bot in https://github.com/mudler/LocalAI/pull/6012 </details> ## New Contributors * @lnnt made their first contribution in https://github.com/mudler/LocalAI/pull/5994 **Full Changelog**: https://github.com/mudler/LocalAI/compare/v3.3.2...v3.4.0