LocalAI v1.40.0 — Release Notes & Engineering Impact

This release is a preparation before v2 - the efforts now will be to refactor, polish and add new backends. Follow up on: https://github.com/mudler/LocalAI/issues/1126 ## Hot topics This release now brings the `llama-cpp` backend which is a c++ backend tied to llama.cpp. It follows more closely and tracks recent versions of llama.cpp. It is not feature compatible with the current `llama` backend but plans are to sunset the current `llama` backend in favor of this one. This one will be probably be the latest release containing the older `llama` backend written in go and c++. The major improvement with this change is that there are less layers that could be expose to potential bugs - and as well it ease out maintenance as well. ### Support for ROCm/HIPBLAS This release bring support for AMD thanks to @65a . See more details in https://github.com/mudler/LocalAI/pull/1100 ### More CLI commands Thanks to @jespino now the local-ai binary has more subcommands allowing to manage the gallery or try out directly inferencing, check it out! ## What's Changed ### Bug fixes :bug: * fix(openai): Populate ID and Created fields in OpenAI compatible responses by @jespino in https://github.com/mudler/LocalAI/pull/1164 * Fix backend/cpp/llama CMakeList.txt on OSX by @dave-gray101 in https://github.com/mudler/LocalAI/pull/1212 ### Exciting New Features 🎉 * feat(llama.cpp): enable ROCm/HIPBLAS support by @65a in https://github.com/mudler/LocalAI/pull/1100 * Cleaning up `examples/` models and starter `.env` files by @jamesbraza in https://github.com/mudler/LocalAI/pull/1124 * Update docker-compose.yaml by @scenaristeur in https://github.com/mudler/LocalAI/pull/1131 * Feats: bruno example, gallery improvements for new scraper by @dave-gray101 in https://github.com/mudler/LocalAI/pull/1161 * feat(cli): Adding models subcommand with list and install subcommands by @jespino in https://github.com/mudler/LocalAI/pull/1165 * feat: add tts command line subcommand by @jespino in https://github.com/mudler/LocalAI/pull/1169 * Adding transcript subcommand by @jespino in https://github.com/mudler/LocalAI/pull/1171 * feat: llama.cpp gRPC C++ backend by @mudler in https://github.com/mudler/LocalAI/pull/1170 * feat(metrics): Adding initial support for prometheus metrics by @jespino in https://github.com/mudler/LocalAI/pull/1176 * ci: add GPU tests by @mudler in https://github.com/mudler/LocalAI/pull/1095 ### 👒 Dependencies * fix(deps): update module github.com/onsi/gomega to v1.28.0 by @renovate in https://github.com/mudler/LocalAI/pull/1113 * :arrow_up: Update go-skynet/go-llama.cpp by @localai-bot in https://github.com/mudler/LocalAI/pull/1106 * fix(deps): update github.com/tmc/langchaingo digest to e16b777 by @renovate in https://github.com/mudler/LocalAI/pull/1101 * fix(deps): update github.com/go-skynet/go-llama.cpp digest to 79f9587 by @renovate in https://github.com/mudler/LocalAI/pull/1085 * fix(deps): update module github.com/shirou/gopsutil/v3 to v3.23.9 by @renovate in https://github.com/mudler/LocalAI/pull/1120 * fix(deps): update module github.com/sashabaranov/go-openai to v1.15.4 by @renovate in https://github.com/mudler/LocalAI/pull/1122 * fix(deps): update module github.com/rs/zerolog to v1.31.0 by @renovate in https://github.com/mudler/LocalAI/pull/1102 * :arrow_up: Update go-skynet/go-llama.cpp by @localai-bot in https://github.com/mudler/LocalAI/pull/1130 * fix(deps): update github.com/go-skynet/go-llama.cpp digest to 6018c9d by @renovate in https://github.com/mudler/LocalAI/pull/1129 * :arrow_up: Update go-skynet/go-llama.cpp by @localai-bot in https://github.com/mudler/LocalAI/pull/1136 * fix(deps): update github.com/go-skynet/go-llama.cpp digest to 1676dcd by @renovate in https://github.com/mudler/LocalAI/pull/1135 * fix(deps): update github.com/nomic-ai/gpt4all/gpt4all-bindings/golang digest to 56c0d28 by @renovate in https://github.com/mudler/LocalAI/pull/1140 * fix(deps): update module github.com/onsi/ginkgo/v2 to v2.13.0 by @renovate in https://github.com/mudler/LocalAI/pull/1152 * fix(deps): update module google.golang.org/grpc to v1.58.3 by @renovate in https://github.com/mudler/LocalAI/pull/1160 * fix(deps): update github.com/go-skynet/go-llama.cpp digest to aeba71e by @renovate in https://github.com/mudler/LocalAI/pull/1155 * fix(deps): update github.com/nomic-ai/gpt4all/gpt4all-bindings/golang digest to 10f9b49 by @renovate in https://github.com/mudler/LocalAI/pull/1158 * fix(deps): update module github.com/sashabaranov/go-openai to v1.16.0 by @renovate in https://github.com/mudler/LocalAI/pull/1159 * fix(deps): update github.com/nomic-ai/gpt4all/gpt4all-bindings/golang digest to 22de3c5 by @renovate in https://github.com/mudler/LocalAI/pull/1172 * fix(deps): update github.com/tmc/langchaingo digest to a02d4fd by @renovate in https://github.com/mudler/LocalAI/pull/1175 * fix(deps): update module github.com/gofiber/fiber/v2 to v2.50.0 by @renovate in https://github.com/mudler/LocalAI/pull/1177 * fix(deps): update github.com/nomic-ai/gpt4all/gpt4all-bindings/golang digest to 9a19c74 by @renovate in https://github.com/mudler/LocalAI/pull/1179 * fix(deps): update github.com/tmc/langchaingo digest to c636b3d by @renovate in https://github.com/mudler/LocalAI/pull/1188 * fix(deps): update module google.golang.org/grpc to v1.59.0 by @renovate in https://github.com/mudler/LocalAI/pull/1189 * chore(deps): update actions/checkout action to v4 by @renovate in https://github.com/mudler/LocalAI/pull/1006 * feat(llama.cpp): update by @mudler in https://github.com/mudler/LocalAI/pull/1200 * :arrow_up: Update go-skynet/go-llama.cpp by @localai-bot in https://github.com/mudler/LocalAI/pull/1156 * fix(deps): update github.com/nomic-ai/gpt4all/gpt4all-bindings/golang digest to c25dc51 by @renovate in https://github.com/mudler/LocalAI/pull/1191 * :arrow_up: Update ggerganov/llama.cpp by @localai-bot in https://github.com/mudler/LocalAI/pull/1204 * fix(deps): update module github.com/onsi/gomega to v1.28.1 by @renovate in https://github.com/mudler/LocalAI/pull/1205 ### Other Changes * fix(deps): update github.com/nomic-ai/gpt4all/gpt4all-bindings/golang digest to 6711bdd by @renovate in https://github.com/mudler/LocalAI/pull/1079 * ci: cleanup worker by @mudler in https://github.com/mudler/LocalAI/pull/1166 * docs(examples): Add mistral example by @mudler in https://github.com/mudler/LocalAI/pull/1214 * feat(llama.cpp): Bump llama.cpp, adapt grpc server by @mudler in https://github.com/mudler/LocalAI/pull/1211 * cleanup: drop bloomz and ggllm as now supported by llama.cpp by @mudler in https://github.com/mudler/LocalAI/pull/1217 * ci: use self-hosted to build container images by @mudler in https://github.com/mudler/LocalAI/pull/1206 * ci: run only cublas on selfhosted by @mudler in https://github.com/mudler/LocalAI/pull/1224 * :arrow_up: Update ggerganov/llama.cpp by @localai-bot in https://github.com/mudler/LocalAI/pull/1207 ## New Contributors * @65a made their first contribution in https://github.com/mudler/LocalAI/pull/1100 * @localai-bot made their first contribution in https://github.com/mudler/LocalAI/pull/1106 * @scenaristeur made their first contribution in https://github.com/mudler/LocalAI/pull/1131 * @jespino made their first contribution in https://github.com/mudler/LocalAI/pull/1164 **Full Changelog**: https://github.com/mudler/LocalAI/compare/v1.30.0...v1.40.0