# vLLM v0.5.5 — Release & Upgrade Intelligence
- **Product:** vLLM (https://vllm.ai)
- **Version:** v0.5.5
- **Released:** Aug 23, 2024
- **Impact Score:** 5/10
- **Breaking Changes:** Not verified
- **Canonical Analysis:** https://changelogs.us/vllm/updates/v0.5.5/

## Executive Summary (TL;DR)
vLLM v0.5.5. Source highlight: We introduced a new mode that schedule multiple GPU steps in advance, reducing CPU overhead (#7000, #7387, #7452, #7703). Initial result shows 20% improvements in QPS for a single GPU running 8B and 30B models. You can set `--num-scheduler-steps 8` as a parameter to the API server (via `vllm serve`)

## New Features
- We introduced a new mode that schedule multiple GPU steps in advance, reducing CPU overhead (#7000, #7387, #7452, #7703). Initial result shows 20% improvements in QPS for a single GPU running 8B and 30B models. You can set `--num-scheduler-steps 8` as a parameter to the API server (via `vllm serve`) or `AsyncLLMEngine`. We are working on expanding the coverage to `LLM` class and aiming to turning it on by default
- AMD: Add fp8 Linear Layer for rocm (#7210)
- Expand MoE weight loading + Add Fused Marlin MoE Kernel (#7527)
- [Build] Add initial conditional testing spec
- [Kernel] Add per-tensor and per-token AZP epilogues
- [distributed][misc] add specialized method for cuda platform

## Improvements
- Various enhancements:
- Use flashinfer sampling kernel when avaiable, leading to 7% decoding throughput speedup (#7137)
- Reduce Python allocations, leading to 24% throughput speedup (#7162, 7364)
- Improvements to the zeromq based decoupled frontend (#7570, #7716, #7484)
- Improvements to vision models:
- Enhancements to TPU support: load time W8A16 quantization (#7005), optimized rope (#7635), and support multi-host inference (#7457).

## Bug Fixes
- [Bugfix] Specify device when loading LoRA and embedding tensors
- [BugFix] Fix DeepSeek remote code
- [ BugFix ] Fix ZMQ when `VLLM_PORT` is set
- [Bugfix] add gguf dependency
- [SpecDecode] [Minor] Fix spec decode sampler tests
- [Bugfix] Fix GPTQ and GPTQ Marlin CPU Offloading

## Upgrade Command
```bash
python -m pip install 'vllm==0.5.5'
```