vLLM v0.7.2 — Release Notes & Engineering Impact
vLLM v0.7.2. Source highlight: Qwen2.5-VL is now supported in vLLM. Please note that it requires a source installation from Hugging Face `transformers` library at the moment (#12604)
vllm release archive — Page 2
91 tracked entries. Older history is imported progressively; see source coverage.
vLLM v0.7.1 — Release Notes & Engineering Impact
vLLM v0.7.1. Source highlight: This release features MLA optimization for Deepseek family of models. Compared to v0.7.0 released this Monday, we offer ~3x the generation throughput, ~10x the memory capacity for tokens, and horizontal context scalability with pipeline parallelism
vLLM v0.7.0 — Release Notes & Engineering Impact
vLLM v0.7.0. Source highlight: vLLM's V1 engine is ready for testing! This is a rewritten engine designed for performance and architectural simplicity. You can turn it on by setting environment variable `VLLM_USE_V1=1`. See our blog for more details. (44 commits).
vLLM v0.6.6.post1 — Release Notes & Engineering Impact
vLLM v0.6.6.post1. Source highlight: This release restore functionalities for other quantized MoEs, which was introduced as part of initial DeepSeek V3 support 🙇 .
vLLM v0.6.6 — Release Notes & Engineering Impact
vLLM v0.6.6. Source highlight: Support Deepseek V3 (#11523, #11502) model.
vLLM v0.6.5 — Release Notes & Engineering Impact
vLLM v0.6.5. Source highlight: Significant progress on the V1 engine refactor and multimodal support: New model executable interfaces for text-only and multimodal models, multiprocessing, improved configuration handling, and profiling enhancements (#10374, #10570, #10699, #11074, #11076, #10382, #10665, #10564, #11125, #11185, #1
vLLM v0.6.4.post1 — Release Notes & Engineering Impact
vLLM v0.6.4.post1. Source highlight: This patch release covers bug fixes (#10347, #10349, #10348, #10352, #10363), keep compatibility for `vLLMConfig` usage in out of tree models (#10356)
vLLM v0.6.4 — Release Notes & Engineering Impact
vLLM v0.6.4. Source highlight: Significant progress in V1 engine core refactor (#9826, #10135, #10288, #10211, #10225, #10228, #10268, #9954, #10272, #9971, #10224, #10166, #9289, #10058, #9888, #9972, #10059, #9945, #9679, #9871, #10227, #10245, #9629, #10097, #10203, #10148). You can checkout more details regarding the design a
vLLM v0.6.3.post1 — Release Notes & Engineering Impact
vLLM v0.6.3.post1. Source highlight: Support Ministral 3B and Ministral 8B via interleaved attention (#9414)
vLLM v0.6.3 — Release Notes & Engineering Impact
vLLM v0.6.3. Source highlight: Text: Granite MoE (#8206), Mamba (#6484, #8533)
vLLM v0.6.2 — Release Notes & Engineering Impact
vLLM v0.6.2. Source highlight: Support Llama 3.2 models (#8811, #8822)
vLLM v0.6.1.post2 — Release Notes & Engineering Impact
vLLM v0.6.1.post2. Source highlight: This release contains an important bugfix related to token streaming combined with stop string (#8468)
vLLM v0.6.1.post1 — Release Notes & Engineering Impact
vLLM v0.6.1.post1. Source highlight: This release features important bug fixes and enhancements for
vLLM v0.6.1 — Release Notes & Engineering Impact
vLLM v0.6.1. Source highlight: Added support for Pixtral (`mistralai/Pixtral-12B-2409`). (#8377, #8168)
vLLM v0.6.0 — Release Notes & Engineering Impact
vLLM v0.6.0. Source highlight: We are excited to announce a faster vLLM delivering 2x more throughput compared to v0.5.3. The default parameters should achieve great speed up, but we recommend also try out turning on multi step scheduling. You can do so by setting `--num-scheduler-steps 8` in the engine arguments. Please note tha
vLLM v0.5.5 — Release Notes & Engineering Impact
vLLM v0.5.5. Source highlight: We introduced a new mode that schedule multiple GPU steps in advance, reducing CPU overhead (#7000, #7387, #7452, #7703). Initial result shows 20% improvements in QPS for a single GPU running 8B and 30B models. You can set `--num-scheduler-steps 8` as a parameter to the API server (via `vllm serve`)
vLLM v0.5.4 — Release Notes & Engineering Impact
vLLM v0.5.4. Source highlight: Enhanced pipeline parallelism support for DeepSeek v2 (#6519), Qwen (#6974), Qwen2 (#6924), and Nemotron (#6863)
vLLM v0.5.3.post1 — Release Notes & Engineering Impact
vLLM v0.5.3.post1. Source highlight: We fixed an configuration incompatibility between vLLM (which tested against pre-released version) and the published Meta Llama 3.1 weights (#6693)
vLLM v0.5.3 — Release Notes & Engineering Impact
vLLM v0.5.3. Source highlight: vLLM now supports Meta Llama 3.1! Please checkout our blog here for initial details on running the model.
vLLM v0.5.2 — Release Notes & Engineering Impact
vLLM v0.5.2. Source highlight: ❗Planned breaking change ❗: we plan to remove beam search (see more in #6226) in the next few releases. This release come with a warning when beam search is enabled for the request. Please voice your concern in the RFC if you do have a valid use case for beam search in vLLM
vLLM v0.5.1 — Release Notes & Engineering Impact
vLLM v0.5.1. Source highlight: vLLM now has pipeline parallelism! (#4412, #5408, #6115, #6120). You can now run the API server with `--pipeline-parallel-size`. This feature is in early stage, please let us know your feedback.
vLLM v0.5.0.post1 — Release Notes & Engineering Impact
vLLM v0.5.0.post1. Source highlight: Add initial TPU integration (#5292)
vLLM v0.5.0 — Release Notes & Engineering Impact
vLLM v0.5.0. Source highlight: FP8 support is ready for testing. By quantizing the portion model weights to 8 bit precision float point, the inference speed gets 1.5x boost. Please try it out and let us know your thoughts! (#5352, #5388, #5159, #5238, #5294, #5183, #5144, #5231)
vLLM v0.4.3 — Release Notes & Engineering Impact
vLLM v0.4.3. Source highlight: Added support for Falcon (#5069)
vLLM v0.4.2 — Release Notes & Engineering Impact
vLLM v0.4.2. Source highlight: Chunked prefill is ready for testing! It improves inter-token latency in high load scenario by chunking the prompt processing and priortizes decode (#4580)
vLLM v0.4.1 — Release Notes & Engineering Impact
vLLM v0.4.1. Source highlight: Support and enhance CommandR+ (#3829), minicpm (#3893), Meta Llama 3 (#4175, #4182), Mixtral 8x22b (#4073, #4002)
vLLM: v0.4.0.post1, restore sm70/75 support
vLLM v0.4.0.post1. Source highlight: v0.4.0 lacks support for sm70/75 support. We did a hotfix for it.
vLLM v0.4.0 — Release Notes & Engineering Impact
vLLM v0.4.0. Source highlight: New models: Command+R(#3433), Qwen2 MoE(#3346), DBRX(#3660), XVerse (#3610), Jais (#3183).
vLLM v0.3.3 — Release Notes & Engineering Impact
vLLM v0.3.3. Source highlight: Performance optimization and LoRA support for Gemma
vLLM v0.3.2 — Release Notes & Engineering Impact
vLLM v0.3.2. Source highlight: This version adds support for the OLMo and Gemma Model, as well as `seed` parameter.
vLLM v0.3.1 — Release Notes & Engineering Impact
vLLM v0.3.1. Source highlight: This version fixes the following major bugs:
vLLM v0.3.0 — Release Notes & Engineering Impact
vLLM v0.3.0. Source highlight: Experimental multi-lora support
vLLM v0.2.7 — Release Notes & Engineering Impact
vLLM v0.2.7. Source highlight: Up to 70% throughput improvement for distributed inference by removing serialization/deserialization overheads
vLLM v0.2.6 — Release Notes & Engineering Impact
vLLM v0.2.6. Source highlight: Fast model execution with CUDA/HIP graph
vLLM v0.2.5 — Release Notes & Engineering Impact
vLLM v0.2.5. Source highlight: Optimize Mixtral performance with expert parallelism (thanks to @Yard1)
vLLM v0.2.4 — Release Notes & Engineering Impact
vLLM v0.2.4. Source highlight: Mixtral model support (officially from @mistralai)
vLLM v0.2.3 — Release Notes & Engineering Impact
vLLM v0.2.3. Source highlight: Refactoring on Worker, InputMetadata, and Attention
vLLM v0.2.2 — Release Notes & Engineering Impact
vLLM v0.2.2. Source highlight: Bump up to PyTorch v2.1 + CUDA 12.1 (vLLM+CUDA 11.8 is also provided)
vLLM v0.2.1.post1 — Release Notes & Engineering Impact
vLLM v0.2.1.post1 is published, but the available notes are too limited to assess upgrade impact. Review the official source before upgrading.
vLLM v0.2.1 — Release Notes & Engineering Impact
vLLM v0.2.1. Source highlight: PagedAttention V2 kernel: Up to 20% end-to-end latency reduction
vLLM v0.2.0 — Release Notes & Engineering Impact
vLLM v0.2.0. Source highlight: Up to 60% performance improvement by optimizing de-tokenization and sampler