# vLLM v0.4.0 — Release & Upgrade Intelligence
- **Product:** vLLM (https://vllm.ai)
- **Version:** v0.4.0
- **Released:** Mar 30, 2024
- **Impact Score:** 5/10
- **Breaking Changes:** Not verified
- **Canonical Analysis:** https://changelogs.us/vllm/updates/v0.4.0/

## Executive Summary (TL;DR)
vLLM v0.4.0. Source highlight: New models: Command+R(#3433), Qwen2 MoE(#3346), DBRX(#3660), XVerse (#3610), Jais (#3183).

## New Features
- Automatic prefix caching (#2762, #3703) supporting long system prompt to be automatically cached across requests. Use the flag `--enable-prefix-caching` to turn it on.
- Support `json_object` in OpenAI server for arbitrary JSON, `--use-delay` flag to improve time to first token across many requests, and `min_tokens` to EOS suppression.
- Progress in chunked prefill scheduler (#3236, #3538), and speculative decoding (#3103).
- Add Automatic Prefix Caching
- Add vLLM version info to logs and openai API server
- [DOC] add setup document to support neuron backend

## Improvements
- Improved Neuron support for AWS Inferentia.
- Enhance lora tests with more layer and rank variations
- Dynamic scheduler delay to improve ITL performance
- [Core] Improve detokenization performance for prefill

## Bug Fixes
- Custom all reduce kernel has been re-enabled after more robustness fixes.
- Replaced cupy dependency due to its bugs.
- allow user chose log level by --log-level instead of fixed 'info'.
- [FIX] Fix styles in automatic prefix caching & add a automatic prefix caching benchmark
- [Minor Fix] Remove unused code in benchmark_prefix_caching.py
- [Minor fix] The domain dns.google may cause a socket.gaierror exception

## Upgrade Command
```bash
python -m pip install 'vllm==0.4.0'
```