# vLLM v0.5.0 — Release & Upgrade Intelligence
- **Product:** vLLM (https://vllm.ai)
- **Version:** v0.5.0
- **Released:** Jun 11, 2024
- **Impact Score:** 7/10
- **Breaking Changes:** Source flagged
- **Canonical Analysis:** https://changelogs.us/vllm/updates/v0.5.0/

## Executive Summary (TL;DR)
vLLM v0.5.0. Source highlight: FP8 support is ready for testing. By quantizing the portion model weights to 8 bit precision float point, the inference speed gets 1.5x boost. Please try it out and let us know your thoughts! (#5352, #5388, #5159, #5238, #5294, #5183, #5144, #5231)

## Breaking Changes & Deprecations
- [Misc][Breaking] Change FP8 checkpoint format from act_scale -> input_scale by @mgoin in https://github.com/vllm-project/vllm/pull/5353

## New Features
- FP8 support is ready for testing. By quantizing the portion model weights to 8 bit precision float point, the inference speed gets 1.5x boost. Please try it out and let us know your thoughts! (#5352, #5388, #5159, #5238, #5294, #5183, #5144, #5231)
- Add OpenAI Vision API support. Currently only LLaVA and LLaVA-NeXT are supported. We are working on adding more models in the next release. (#5237, #5383, #4199, #5374, #4197)
- Speculative Decoding and Automatic Prefix Caching is also ready for testing, we plan to turn them on by default in upcoming releases. (#5400, #5157, #5137, #5324)
- Default to multiprocessing backend for single-node distributed case (#5230)
- Support bitsandbytes quantization and QLoRA (#4776)
- [Feature][Kernel] Support bitsandbytes quantization and QLoRA

## Improvements
- Improvements to the Intel CPU CI (#4113, #5241)
- [Kernel] Enhance MoE benchmarking & tuning script
- [CI/Test] improve robustness of test by replacing del with context manager (hf_runner)
- [CI/Test] improve robustness of test by replacing del with context manager (vllm_runner)
- [Misc] Improve error message when LoRA parsing fails

## Bug Fixes
- Debugging tips documentation (#5409, #5430)
- [Minor] Fix the path typo in loader.py: save_sharded_states.py -> save_sharded_state.py
- [Bugfix] Fix call to init_logger in openai server
- [Bugfix] Remove deprecated @abstractproperty
- [Bugfix]: Fix issues related to prefix caching example (#5177)
- [BugFix] Prevent `LLM.encode` for non-generation Models

## Upgrade Command
```bash
python -m pip install 'vllm==0.5.0'
```