vLLM v0.6.1.post1 — Release Notes & Engineering Impact

## Highlights This release features important bug fixes and enhancements for - Pixtral models. (#8415, #8425, #8399, #8431) - Chunked scheduling has been turned off for vision models. Please replace `--max_num_batched_tokens 16384` with `--max-model-len 16384` - Multistep scheduling. (#8417, #7928, #8427) - Tool use. (#8423, #8366) Also * support multiple images for qwen-vl (#8247) * removes `engine_use_ray` (#8126) * add engine option to return only deltas or final output (#7381) * add bitsandbytes support for Gemma2 (#8338) ## What's Changed * [MISC] Dump model runner inputs when crashing by @comaniac in https://github.com/vllm-project/vllm/pull/8305 * [misc] remove engine_use_ray by @youkaichao in https://github.com/vllm-project/vllm/pull/8126 * [TPU] Use Ray for default distributed backend by @WoosukKwon in https://github.com/vllm-project/vllm/pull/8389 * Fix the AMD weight loading tests by @mgoin in https://github.com/vllm-project/vllm/pull/8390 * [Bugfix]: Fix the logic for deciding if tool parsing is used by @tomeras91 in https://github.com/vllm-project/vllm/pull/8366 * [Gemma2] add bitsandbytes support for Gemma2 by @blueyo0 in https://github.com/vllm-project/vllm/pull/8338 * [Misc] Raise error when using encoder/decoder model with cpu backend by @kevin314 in https://github.com/vllm-project/vllm/pull/8355 * [Misc] Use RoPE cache for MRoPE by @WoosukKwon in https://github.com/vllm-project/vllm/pull/8396 * [torch.compile] hide slicing under custom op for inductor by @youkaichao in https://github.com/vllm-project/vllm/pull/8384 * [Hotfix][VLM] Fixing max position embeddings for Pixtral by @ywang96 in https://github.com/vllm-project/vllm/pull/8399 * [Bugfix] Fix InternVL2 inference with various num_patches by @Isotr0py in https://github.com/vllm-project/vllm/pull/8375 * [Model] Support multiple images for qwen-vl by @alex-jw-brooks in https://github.com/vllm-project/vllm/pull/8247 * [BugFix] lazy init _copy_stream to avoid torch init wrong gpu instance by @lnykww in https://github.com/vllm-project/vllm/pull/8403 * [BugFix] Fix Duplicate Assignment of Class Variable in Hermes2ProToolParser by @vegaluisjose in https://github.com/vllm-project/vllm/pull/8423 * [Bugfix] Offline mode fix by @joerunde in https://github.com/vllm-project/vllm/pull/8376 * [multi-step] add flashinfer backend by @SolitaryThinker in https://github.com/vllm-project/vllm/pull/7928 * [Core] Add engine option to return only deltas or final output by @njhill in https://github.com/vllm-project/vllm/pull/7381 * [Bugfix] multi-step + flashinfer: ensure cuda graph compatible by @alexm-neuralmagic in https://github.com/vllm-project/vllm/pull/8427 * [Hotfix][Core][VLM] Disable chunked prefill by default and prefix caching for multimodal models by @ywang96 in https://github.com/vllm-project/vllm/pull/8425 * [CI/Build] Disable multi-node test for InternVL2 by @ywang96 in https://github.com/vllm-project/vllm/pull/8428 * [Hotfix][Pixtral] Fix multiple images bugs by @patrickvonplaten in https://github.com/vllm-project/vllm/pull/8415 * [Bugfix] Fix weight loading issue by rename variable. by @wenxcs in https://github.com/vllm-project/vllm/pull/8293 * [Misc] Update Pixtral example by @ywang96 in https://github.com/vllm-project/vllm/pull/8431 * [BugFix] fix group_topk by @dsikka in https://github.com/vllm-project/vllm/pull/8430 * [Core] Factor out input preprocessing to a separate class by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/7329 * [Bugfix] Mapping physical device indices for e2e test utils by @ShangmingCai in https://github.com/vllm-project/vllm/pull/8290 * [Bugfix] Bump fastapi and pydantic version by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/8435 * [CI/Build] Update pixtral tests to use JSON by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/8436 * [Bugfix] Fix async log stats by @alexm-neuralmagic in https://github.com/vllm-project/vllm/pull/8417 * [bugfix] torch profiler bug for single gpu with GPUExecutor by @SolitaryThinker in https://github.com/vllm-project/vllm/pull/8354 * bump version to v0.6.1.post1 by @simon-mo in https://github.com/vllm-project/vllm/pull/8440 ## New Contributors * @blueyo0 made their first contribution in https://github.com/vllm-project/vllm/pull/8338 * @lnykww made their first contribution in https://github.com/vllm-project/vllm/pull/8403 * @vegaluisjose made their first contribution in https://github.com/vllm-project/vllm/pull/8423 **Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.6.1...v0.6.1.post1