vLLM v0.5.0.post1 — Release Notes & Engineering Impact

# Highlights * Add initial TPU integration (#5292) * Fix crashes when using FlashAttention backend (#5478) * Fix issues when using num_devices < num_available_devices (#5473) ## What's Changed * [CI/Build] Add `is_quant_method_supported` to control quantization test configurations by @mgoin in https://github.com/vllm-project/vllm/pull/5253 * Revert "[CI/Build] Add `is_quant_method_supported` to control quantization test configurations" by @simon-mo in https://github.com/vllm-project/vllm/pull/5463 * [CI] Upgrade codespell version. by @rkooo567 in https://github.com/vllm-project/vllm/pull/5381 * [Hardware] Initial TPU integration by @WoosukKwon in https://github.com/vllm-project/vllm/pull/5292 * [Bugfix] Add device assertion to TorchSDPA by @bigPYJ1151 in https://github.com/vllm-project/vllm/pull/5402 * [ci] Add AMD, Neuron, Intel tests for AWS CI and turn off default soft fail for GPU tests by @khluu in https://github.com/vllm-project/vllm/pull/5464 * [Kernel] Vectorized FP8 quantize kernel by @comaniac in https://github.com/vllm-project/vllm/pull/5396 * [Bugfix] TYPE_CHECKING for MultiModalData by @kimdwkimdw in https://github.com/vllm-project/vllm/pull/5444 * [Frontend] [Core] Support for sharded tensorized models by @tjohnson31415 in https://github.com/vllm-project/vllm/pull/4990 * [misc] add hint for AttributeError by @youkaichao in https://github.com/vllm-project/vllm/pull/5462 * [Doc] Update debug docs by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/5438 * [Bugfix] Fix typo in scheduler.py (requeset -> request) by @mgoin in https://github.com/vllm-project/vllm/pull/5470 * [Frontend] Add "input speed" to tqdm postfix alongside output speed by @mgoin in https://github.com/vllm-project/vllm/pull/5425 * [Bugfix] Fix wrong multi_modal_input format for CPU runner by @Isotr0py in https://github.com/vllm-project/vllm/pull/5451 * [Core][Distributed] add coordinator to reduce code duplication in tp and pp by @youkaichao in https://github.com/vllm-project/vllm/pull/5293 * [ci] Use sccache to build images by @khluu in https://github.com/vllm-project/vllm/pull/5419 * [Bugfix]if the content is started with ":"(response of ping), client should i… by @sywangyi in https://github.com/vllm-project/vllm/pull/5303 * [Kernel] `w4a16` support for `compressed-tensors` by @dsikka in https://github.com/vllm-project/vllm/pull/5385 * [CI/Build][REDO] Add is_quant_method_supported to control quantization test configurations by @mgoin in https://github.com/vllm-project/vllm/pull/5466 * [Kernel] Tune Qwen2MoE kernel configurations with tp2,4 by @wenyujin333 in https://github.com/vllm-project/vllm/pull/5497 * [Hardware][Intel] Optimize CPU backend and add more performance tips by @bigPYJ1151 in https://github.com/vllm-project/vllm/pull/4971 * [Docs] Add 4th meetup slides by @WoosukKwon in https://github.com/vllm-project/vllm/pull/5509 * [Misc] Add vLLM version getter to utils by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/5098 * [CI/Build] Simplify OpenAI server setup in tests by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/5100 * [Doc] Update LLaVA docs by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/5437 * [Kernel] Factor out epilogues from cutlass kernels by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/5391 * [MISC] Remove FP8 warning by @comaniac in https://github.com/vllm-project/vllm/pull/5472 * Seperate dev requirements into lint and test by @Yard1 in https://github.com/vllm-project/vllm/pull/5474 * Revert "[Core] Remove unnecessary copies in flash attn backend" by @Yard1 in https://github.com/vllm-project/vllm/pull/5478 * [misc] fix format.sh by @youkaichao in https://github.com/vllm-project/vllm/pull/5511 * [CI/Build] Disable test_fp8.py by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/5508 * [Kernel] Disable CUTLASS kernels for fp8 by @tlrmchlsmth in https://github.com/vllm-project/vllm/pull/5505 * Add `cuda_device_count_stateless` by @Yard1 in https://github.com/vllm-project/vllm/pull/5473 * [Hardware][Intel] Support CPU inference with AVX2 ISA by @DamonFool in https://github.com/vllm-project/vllm/pull/5452 * [Bugfix]typofix by @AllenDou in https://github.com/vllm-project/vllm/pull/5507 * bump version to v0.5.0.post1 by @simon-mo in https://github.com/vllm-project/vllm/pull/5522 ## New Contributors * @kimdwkimdw made their first contribution in https://github.com/vllm-project/vllm/pull/5444 * @sywangyi made their first contribution in https://github.com/vllm-project/vllm/pull/5303 **Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.5.0...v0.5.0.post1