vLLM v0.8.1 — Release Notes & Engineering Impact

This release contains important bug fixes for v0.8.0. We highly recommend upgrading! * V1 Fixes * Ensure using int64 for sampled token ids (#15065) * Fix long dtype in topk sampling (#15049) * Refactor Structured Output for multiple backends (#14694) * Fix size calculation of processing cache (#15114) * Optimize Rejection Sampler with Triton Kernels (#14930) * Fix oracle for device checking (#15104) * TPU * Fix chunked prefill with padding (#15037) * Enhanced CI/CD (#15054, 14974) * Model * Re-enable Gemma3 for V1 (#14980) * Embedding model support LoRA (#14935) * Pixtral: Remove layer instantiation duplication (#15053) ## What's Changed * [Bugfix] Fix interface for Olmo2 on V1 by @ywang96 in https://github.com/vllm-project/vllm/pull/14976 * [CI/Build] Use `AutoModelForImageTextToText` to load image models in tests by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/14945 * [V1] Guard Against Main Thread Usage by @robertgshaw2-redhat in https://github.com/vllm-project/vllm/pull/14972 * [V1] TPU - Fix CI/CD runner for V1 and remove V0 tests by @alexm-redhat in https://github.com/vllm-project/vllm/pull/14974 * [Bugfix] Fix bnb quantization for models with both HF-format and Mistral-format weights by @tristanleclercq in https://github.com/vllm-project/vllm/pull/14950 * [Neuron] trim attention kernel tests to fit trn1.2x instance by @liangfu in https://github.com/vllm-project/vllm/pull/14988 * [Doc][V1] Fix V1 APC doc by @shen-shanshan in https://github.com/vllm-project/vllm/pull/14920 * [Kernels] LoRA - Retire SGMV and BGMV Kernels by @varun-sundar-rabindranath in https://github.com/vllm-project/vllm/pull/14685 * [Mistral-Small 3.1] Update docs and tests by @patrickvonplaten in https://github.com/vllm-project/vllm/pull/14977 * [Misc] Embedding model support LoRA by @jeejeelee in https://github.com/vllm-project/vllm/pull/14935 * [Bugfix] torchrun compatibility by @hiyouga in https://github.com/vllm-project/vllm/pull/14899 * [Bugfix][Frontend] Fix validation of `logprobs` in `ChatCompletionRequest` by @schoennenbeck in https://github.com/vllm-project/vllm/pull/14352 * [Misc][Docs] fix the comments of KV_T and CACHE_T in CALL_RESHAPE_AND_CACHE_XX macros by @yangsijia-serena in https://github.com/vllm-project/vllm/pull/14347 * [Bugfix] Loosen type check to avoid errors in V1 by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/15021 * [Bugfix] Register serializers for V0 MQ Engine by @simon-mo in https://github.com/vllm-project/vllm/pull/15009 * [TPU][V1][Bugfix] Fix chunked prefill with padding by @NickLucche in https://github.com/vllm-project/vllm/pull/15037 * MI325 configs, fused_moe_kernel bugfix by @ekuznetsov139 in https://github.com/vllm-project/vllm/pull/14987 * [MODEL] Add support for Zamba2 models by @yury-tokpanov in https://github.com/vllm-project/vllm/pull/13185 * [Bugfix] Fix broken CPU quantization due to triton import by @Isotr0py in https://github.com/vllm-project/vllm/pull/15038 * [Bugfix] Fix LoRA extra vocab size by @jeejeelee in https://github.com/vllm-project/vllm/pull/15047 * [V1] Refactor Structured Output for multiple backends by @russellb in https://github.com/vllm-project/vllm/pull/14694 * [V1][Spec Decode] Optimize Rejection Sampler with Triton Kernels by @WoosukKwon in https://github.com/vllm-project/vllm/pull/14930 * [V1] TPU - CI/CD use smaller model by @alexm-redhat in https://github.com/vllm-project/vllm/pull/15054 * fix long dtype in topk sampling by @chujiezheng in https://github.com/vllm-project/vllm/pull/15049 * [Doc] Minor v1_user_guide update by @JenZhao in https://github.com/vllm-project/vllm/pull/15064 * [Misc][V1] Skip device checking if not available by @comaniac in https://github.com/vllm-project/vllm/pull/15061 * [Model] Pixtral: Remove layer instantiation duplication by @juliendenize in https://github.com/vllm-project/vllm/pull/15053 * [Model] Remove duplicated message check in Mistral chat completion request by @b8zhong in https://github.com/vllm-project/vllm/pull/15069 * [Core] Update dtype detection and defaults by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/14858 * [V1] Ensure using int64 for sampled token ids by @WoosukKwon in https://github.com/vllm-project/vllm/pull/15065 * [Bugfix] Re-enable Gemma3 for V1 by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/14980 * [CI][Intel GPU] update XPU dockerfile and CI script by @jikunshang in https://github.com/vllm-project/vllm/pull/15109 * [V1][Bugfix] Fix oracle for device checking by @ywang96 in https://github.com/vllm-project/vllm/pull/15104 * [Misc] Avoid unnecessary HF `do_rescale` warning when passing dummy data by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/15107 * [Bugfix] Fix size calculation of processing cache by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/15114 * [Doc] Update tip info on using latest transformers when creating a custom Dockerfile by @MarcCote in https://github.com/vllm-project/vllm/pull/15070 * [Misc][Benchmark] Add support for different `tokenizer_mode` by @aarnphm in https://github.com/vllm-project/vllm/pull/15040 * [Bugfix] Adjust mllama to regional compilation by @jkaniecki in https://github.com/vllm-project/vllm/pull/15112 * [Doc] Update the "the first vLLM China Meetup" slides link to point to the first page by @imkero in https://github.com/vllm-project/vllm/pull/15134 * [Frontend] Remove custom_cache_manager by @fulvius31 in https://github.com/vllm-project/vllm/pull/13791 * [V1] Minor V1 async engine test refactor by @andoorve in https://github.com/vllm-project/vllm/pull/15075 ## New Contributors * @tristanleclercq made their first contribution in https://github.com/vllm-project/vllm/pull/14950 * @hiyouga made their first contribution in https://github.com/vllm-project/vllm/pull/14899 * @ekuznetsov139 made their first contribution in https://github.com/vllm-project/vllm/pull/14987 * @yury-tokpanov made their first contribution in https://github.com/vllm-project/vllm/pull/13185 * @juliendenize made their first contribution in https://github.com/vllm-project/vllm/pull/15053 * @MarcCote made their first contribution in https://github.com/vllm-project/vllm/pull/15070 * @jkaniecki made their first contribution in https://github.com/vllm-project/vllm/pull/15112 * @fulvius31 made their first contribution in https://github.com/vllm-project/vllm/pull/13791 **Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.8.0...v0.8.1