LocalAI v2.18.0 — Release Notes & Engineering Impact

 ## :star: Highlights Here’s a quick overview of what’s new in 2.18.0: - :whale: Support for models in OCI registry (includes ollama) - 🌋 Support for llama.cpp with vulkan (container images only for now) - :speaking_head: the transcription endpoint now can also translate with `translate` - :gear: Adds `repeat_last_n` and `properties_order` as model configurations - :arrow_up: **CUDA 12.5 Upgrade**: we are now tracking the latest CUDA version (12.5). - :gem: Gemma 2 model support! ### 🐋 Support for OCI Images and Ollama Models You can now specify models using `oci://` and `ollama://` prefixes in your YAML config files. Here’s an example for Ollama models: ```yaml parameters: model: ollama://... ``` Start the Ollama model directly with: ``` local-ai run ollama://gemma:2b ``` Or download only the model by using: ``` local-ai models install ollama://gemma:2b ``` For standard OCI images, use the `oci://` prefix. To build a compatible container image, use `docker` for example. Your Dockerfile should look like this: ```Dockerfile FROM scratch COPY ./my_gguf_file.gguf / ``` You can actually use it to store also other model types (for instance safetensors files for stable diffusion) and YAML config files ! ### 🌋 Vulkan Support for Llama.cpp We’ve introduced Vulkan support for Llama.cpp! Check out our new image tags `latest-vulkan-ffmpeg-core` and `v2.18.0-vulkan-ffmpeg-core`. ### 🗣️ Transcription and Translation Our transcription endpoint now supports translation! Simply add `translate: true` to your transcription requests to translate the transcription to English. ### ⚙️ Enhanced Model Configuration We’ve added new configuration options `repeat_last_n` and `properties_order` to give you more control. Here’s how you can set them up in your model YAML file: ```yaml # Force JSON to return properties in the specified order function: grammar: properties_order: "name,arguments" ``` And for setting `repeat_last_n` (specific to Llama.cpp): ```yaml parameters: repeat_last_n: 64 ``` ### :gem: Gemma 2!  Google has just dropped gemma 2 models ([blog post here](https://blog.google/technology/developers/google-gemma-2/)), you can already install and run gemma 2 models in LocalAI with ``` local-ai run gemma-2-27b-it local-ai run gemma-2-9b-it ``` --- <!-- Release notes generated using configuration in .github/release.yml at master --> ## What's Changed ### Bug fixes :bug: * fix(install.sh): correctly handle systemd service installation by @mudler in https://github.com/mudler/LocalAI/pull/2627 * fix(worker): use dynaload for single binaries by @mudler in https://github.com/mudler/LocalAI/pull/2620 * fix(install.sh): fix version typo by @mudler in https://github.com/mudler/LocalAI/pull/2645 * fix(install.sh): move ARCH detection so it works also for mac by @mudler in https://github.com/mudler/LocalAI/pull/2646 * fix(cli): remove duplicate alias by @mudler in https://github.com/mudler/LocalAI/pull/2654 ### Exciting New Features 🎉 * feat: Upgrade to CUDA 12.5 by @reneleonhardt in https://github.com/mudler/LocalAI/pull/2601 * feat(oci): support OCI images and Ollama models by @mudler in https://github.com/mudler/LocalAI/pull/2628 * feat(whisper): add translate option by @mudler in https://github.com/mudler/LocalAI/pull/2649 * feat(vulkan): add vulkan support to the llama.cpp backend by @mudler in https://github.com/mudler/LocalAI/pull/2648 * feat(ui): allow to select between all the available models in the chat by @mudler in https://github.com/mudler/LocalAI/pull/2657 * feat(build): only build llama.cpp relevant targets by @mudler in https://github.com/mudler/LocalAI/pull/2659 * feat(options): add `repeat_last_n` by @mudler in https://github.com/mudler/LocalAI/pull/2660 * feat(grammar): expose properties_order by @mudler in https://github.com/mudler/LocalAI/pull/2662 ### 🧠 Models * models(gallery): add l3-umbral-mind-rp-v1.0-8b-iq-imatrix by @mudler in https://github.com/mudler/LocalAI/pull/2608 * models(gallery): :arrow_up: update checksum by @localai-bot in https://github.com/mudler/LocalAI/pull/2607 * models(gallery): add llama-3-sec-chat by @mudler in https://github.com/mudler/LocalAI/pull/2611 * models(gallery): add llama-3-cursedstock-v1.8-8b-iq-imatrix by @mudler in https://github.com/mudler/LocalAI/pull/2612 * models(gallery): add llama3-8b-darkidol-1.1-iq-imatrix by @mudler in https://github.com/mudler/LocalAI/pull/2613 * models(gallery): add magnum-72b-v1 by @mudler in https://github.com/mudler/LocalAI/pull/2614 * models(gallery): add qwen2-1.5b-ita by @mudler in https://github.com/mudler/LocalAI/pull/2615 * models(gallery): add hermes-2-theta-llama-3-70b by @mudler in https://github.com/mudler/LocalAI/pull/2626 * models(gallery): :arrow_up: update checksum by @localai-bot in https://github.com/mudler/LocalAI/pull/2630 * models(gallery): add dark-idol-1.2 by @mudler in https://github.com/mudler/LocalAI/pull/2663 * models(gallery): add einstein v7 qwen2 by @mudler in https://github.com/mudler/LocalAI/pull/2664 * models(gallery): add arcee-spark by @mudler in https://github.com/mudler/LocalAI/pull/2665 * models(gallery): add gemma2-9b-it and gemma2-27b-it by @mudler in https://github.com/mudler/LocalAI/pull/2670 ### 📖 Documentation and examples * docs: update to include installer and update advanced YAML options by @mudler in https://github.com/mudler/LocalAI/pull/2631 * feat(swagger): update swagger by @localai-bot in https://github.com/mudler/LocalAI/pull/2651 * feat(swagger): update swagger by @localai-bot in https://github.com/mudler/LocalAI/pull/2666 * telegram-bot example: Update LocalAI version (fixes #2638) by @greygoo in https://github.com/mudler/LocalAI/pull/2640 ### 👒 Dependencies * :arrow_up: Update docs version mudler/LocalAI by @localai-bot in https://github.com/mudler/LocalAI/pull/2605 * :arrow_up: Update ggerganov/llama.cpp by @localai-bot in https://github.com/mudler/LocalAI/pull/2606 * :arrow_up: Update ggerganov/llama.cpp by @localai-bot in https://github.com/mudler/LocalAI/pull/2617 * :arrow_up: Update ggerganov/llama.cpp by @localai-bot in https://github.com/mudler/LocalAI/pull/2629 * :arrow_up: Update ggerganov/llama.cpp by @localai-bot in https://github.com/mudler/LocalAI/pull/2632 * deps(llama.cpp): bump to latest, update build variables by @mudler in https://github.com/mudler/LocalAI/pull/2669 * :arrow_up: Update ggerganov/llama.cpp by @localai-bot in https://github.com/mudler/LocalAI/pull/2652 * :arrow_up: Update ggerganov/llama.cpp by @localai-bot in https://github.com/mudler/LocalAI/pull/2671 ### Other Changes * ci: bump parallel jobs by @mudler in https://github.com/mudler/LocalAI/pull/2633 * chore: fix go.mod module by @sozercan in https://github.com/mudler/LocalAI/pull/2635 * rf: centralize base64 image handling and secscan cleanup by @dave-gray101 in https://github.com/mudler/LocalAI/pull/2595 * refactor: gallery inconsistencies by @mudler in https://github.com/mudler/LocalAI/pull/2647 ## New Contributors * @greygoo made their first contribution in https://github.com/mudler/LocalAI/pull/2640 **Full Changelog**: https://github.com/mudler/LocalAI/compare/v2.17.1...v2.18.0