# vLLM v0.5.4 — Release & Upgrade Intelligence
- **Product:** vLLM (https://vllm.ai)
- **Version:** v0.5.4
- **Released:** Aug 05, 2024
- **Impact Score:** 5/10
- **Breaking Changes:** Not verified
- **Canonical Analysis:** https://changelogs.us/vllm/updates/v0.5.4/

## Executive Summary (TL;DR)
vLLM v0.5.4. Source highlight: Enhanced pipeline parallelism support for DeepSeek v2 (#6519), Qwen (#6974), Qwen2 (#6924), and Nemotron (#6863)

## New Features
- Added H2O Danube3-4b (#6451)
- Added Nemotron models (Nemotron-3, Nemotron-4, Minitron) (#6611)
- Enhancements to speculative decoding: FlashInfer in DraftModelRunner (#6926), observability (#6963), and benchmarks (#6964)
- Refactor the punica kernel based on Triton (#5036)
- Support for guided decoding for offline LLM (#6878)
- [Misc] Add ignored layers for `fp8` quantization

## Improvements
- Enhanced pipeline parallelism support for DeepSeek v2 (#6519), Qwen (#6974), Qwen2 (#6924), and Nemotron (#6863)
- Enhanced vision language model support for InternVL2 (#6514, #7067), BLIP-2 (#5920), MiniCPM-V (#4087, #7122).
- TPU enhancements: collective communication, TP for async engine, faster compile time (#6891, #6933, #6856, #6813, #5871)
- Separated OpenAI Server's HTTP request handling and model inference loop with `zeromq`. This brought 20% speedup over time to first token and 2x speedup over inter token latency. (#6883)
- Used Python's native array data structure speedup padding. This bring 15% throughput enhancement in large batch size scenarios. (#6779)
- Reduce unnecessary compute when logprobs=None. This reduced latency of get log probs from ~30ms to ~5ms in large batch size scenarios. (#6532)

## Bug Fixes
- [doc][distributed] fix doc argument order
- [Bugfix] Fix a log error in chunked prefill
- [BugFix] Fix RoPE error in Llama 3.1
- Bump `transformers` version for Llama 3.1 hotfix and patch Chameleon
- [Bugfix] StatLoggers: cache spec decode metrics when they get collected.
- [Bugfix] fix flashinfer cudagraph capture for PP

## Upgrade Command
```bash
python -m pip install 'vllm==0.5.4'
```