vLLM v0.6.3.post1 — Release Notes & Engineering Impact

## Highlights ### New Models * Support Ministral 3B and Ministral 8B via interleaved attention (#9414) * Support multiple and interleaved images for Llama3.2 (#9095) * Support VLM2Vec, the first multimodal embedding model in vLLM (#9303) ### Important bug fix * Fix chat API continuous usage stats (#9357) * Fix vLLM UsageInfo and logprobs None AssertionError with empty token_ids (#9034) * Fix Molmo text-only input bug (#9397) * Fix CUDA 11.8 Build (#9386) * Fix `_version.py` not found issue (#9375) ### Other Enhancements * Remove block manager v1 and make block manager v2 default (#8704) * Spec Decode Optimize ngram lookup performance (#9333) ## What's Changed * [TPU] Fix TPU SMEM OOM by Pallas paged attention kernel by @WoosukKwon in https://github.com/vllm-project/vllm/pull/9350 * [Frontend] merge beam search implementations by @LunrEclipse in https://github.com/vllm-project/vllm/pull/9296 * [Model] Make llama3.2 support multiple and interleaved images by @xiangxu-google in https://github.com/vllm-project/vllm/pull/9095 * [Bugfix] Clean up some cruft in mamba.py by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/9343 * [Frontend] Clarify model_type error messages by @stevegrubb in https://github.com/vllm-project/vllm/pull/9345 * [Doc] Fix code formatting in spec_decode.rst by @mgoin in https://github.com/vllm-project/vllm/pull/9348 * [Bugfix] Update InternVL input mapper to support image embeds by @hhzhang16 in https://github.com/vllm-project/vllm/pull/9351 * [BugFix] Fix chat API continuous usage stats by @njhill in https://github.com/vllm-project/vllm/pull/9357 * pass ignore_eos parameter to all benchmark_serving calls by @gracehonv in https://github.com/vllm-project/vllm/pull/9349 * [Misc] Directly use compressed-tensors for checkpoint definitions by @mgoin in https://github.com/vllm-project/vllm/pull/8909 * [Bugfix] Fix vLLM UsageInfo and logprobs None AssertionError with empty token_ids by @CatherineSue in https://github.com/vllm-project/vllm/pull/9034 * [Bugfix][CI/Build] Fix CUDA 11.8 Build by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/9386 * [Bugfix] Molmo text-only input bug fix by @mrsalehi in https://github.com/vllm-project/vllm/pull/9397 * [Misc] Standardize RoPE handling for Qwen2-VL by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/9250 * [Model] VLM2Vec, the first multimodal embedding model in vLLM by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/9303 * [CI/Build] Test VLM embeddings by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/9406 * [Core] Rename input data types by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/8688 * [Misc] Consolidate example usage of OpenAI client for multimodal models by @ywang96 in https://github.com/vllm-project/vllm/pull/9412 * [Model] Support SDPA attention for Molmo vision backbone by @Isotr0py in https://github.com/vllm-project/vllm/pull/9410 * Support mistral interleaved attn by @patrickvonplaten in https://github.com/vllm-project/vllm/pull/9414 * [Kernel][Model] Improve continuous batching for Jamba and Mamba by @mzusman in https://github.com/vllm-project/vllm/pull/9189 * [Model][Bugfix] Add FATReLU activation and support for openbmb/MiniCPM-S-1B-sft by @streaver91 in https://github.com/vllm-project/vllm/pull/9396 * [Performance][Spec Decode] Optimize ngram lookup performance by @LiuXiaoxuanPKU in https://github.com/vllm-project/vllm/pull/9333 * [CI/Build] mypy: Resolve some errors from checking vllm/engine by @russellb in https://github.com/vllm-project/vllm/pull/9267 * [Bugfix][Kernel] Prevent integer overflow in fp8 dynamic per-token quantize kernel by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/9425 * [BugFix] [Kernel] Fix GPU SEGV occurring in int8 kernels by @rasmith in https://github.com/vllm-project/vllm/pull/9391 * Add notes on the use of Slack by @terrytangyuan in https://github.com/vllm-project/vllm/pull/9442 * [Kernel] Add Exllama as a backend for compressed-tensors by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/9395 * [Misc] Print stack trace using `logger.exception` by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/9461 * [misc] CUDA Time Layerwise Profiler by @LucasWilkinson in https://github.com/vllm-project/vllm/pull/8337 * [Bugfix] Allow prefill of assistant response when using `mistral_common` by @sasha0552 in https://github.com/vllm-project/vllm/pull/9446 * [TPU] Call torch._sync(param) during weight loading by @WoosukKwon in https://github.com/vllm-project/vllm/pull/9437 * [Hardware][CPU] compressed-tensor INT8 W8A8 AZP support by @bigPYJ1151 in https://github.com/vllm-project/vllm/pull/9344 * [Core] Deprecating block manager v1 and make block manager v2 default by @KuntaiDu in https://github.com/vllm-project/vllm/pull/8704 * [CI/Build] remove .github from .dockerignore, add dirty repo check by @dtrifiro in https://github.com/vllm-project/vllm/pull/9375 ## New Contributors * @gracehonv made their first contribution in https://github.com/vllm-project/vllm/pull/9349 * @streaver91 made their first contribution in https://github.com/vllm-project/vllm/pull/9396 **Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.6.3...v0.6.3.post1