vLLM v0.4.1 — Release Notes & Engineering Impact

## Highlights Features * Support and enhance CommandR+ (#3829), minicpm (#3893), Meta Llama 3 (#4175, #4182), Mixtral 8x22b (#4073, #4002) * Support private model registration, and updating our support policy (#3871, 3948) * Support PyTorch 2.2.1 and Triton 2.2.0 (#4061, #4079, #3805, #3904, #4271) * Add option for using LM Format Enforcer for guided decoding (#3868) * Add option for optionally initialize tokenizer and detokenizer (#3748) * Add option for load model using `tensorizer` (#3476) Enhancements * vLLM is now mostly type checked by `mypy` (#3816, #4006, #4161, #4043) * Progress towards chunked prefill scheduler (#3550, #3853, #4280, #3884) * Progress towards speculative decoding (#3250, #3706, #3894) * Initial support with dynamic per-tensor scaling via FP8 (#4118) Hardwares * Intel CPU inference backend is added (#3993, #3634) * AMD backend is enhanced with Triton kernel and e4m3fn KV cache (#3643, #3290) ## What's Changed * [Kernel] Layernorm performance optimization by @mawong-amd in https://github.com/vllm-project/vllm/pull/3662 * [Doc] Update installation doc for build from source and explain the dependency on torch/cuda version by @youkaichao in https://github.com/vllm-project/vllm/pull/3746 * [CI/Build] Make Marlin Tests Green by @robertgshaw2-neuralmagic in https://github.com/vllm-project/vllm/pull/3753 * [Misc] Minor fixes in requirements.txt by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3769 * [Misc] Some minor simplifications to detokenization logic by @njhill in https://github.com/vllm-project/vllm/pull/3670 * [Misc] Fix Benchmark TTFT Calculation for Chat Completions by @ywang96 in https://github.com/vllm-project/vllm/pull/3768 * [Speculative decoding 4/9] Lookahead scheduling for speculative decoding by @cadedaniel in https://github.com/vllm-project/vllm/pull/3250 * [Misc] Add support for new autogptq checkpoint_format by @Qubitium in https://github.com/vllm-project/vllm/pull/3689 * [Misc] [CI/Build] Speed up block manager CPU-only unit tests ~10x by opting-out of GPU cleanup by @cadedaniel in https://github.com/vllm-project/vllm/pull/3783 * [Hardware][Intel] Add CPU inference backend by @bigPYJ1151 in https://github.com/vllm-project/vllm/pull/3634 * [HotFix] [CI/Build] Minor fix for CPU backend CI by @bigPYJ1151 in https://github.com/vllm-project/vllm/pull/3787 * [Frontend][Bugfix] allow using the default middleware with a root path by @A-Mahla in https://github.com/vllm-project/vllm/pull/3788 * [Doc] Fix vLLMEngine Doc Page by @ywang96 in https://github.com/vllm-project/vllm/pull/3791 * [CI/Build] fix TORCH_CUDA_ARCH_LIST in wheel build by @youkaichao in https://github.com/vllm-project/vllm/pull/3801 * Fix crash when try torch.cuda.set_device in worker by @leiwen83 in https://github.com/vllm-project/vllm/pull/3770 * [Bugfix] Add `__init__.py` files for `vllm/core/block/` and `vllm/spec_decode/` by @mgoin in https://github.com/vllm-project/vllm/pull/3798 * [CI/Build] 0.4.0.post1, fix sm 7.0/7.5 binary by @youkaichao in https://github.com/vllm-project/vllm/pull/3803 * [Speculative decoding] Adding configuration object for speculative decoding by @cadedaniel in https://github.com/vllm-project/vllm/pull/3706 * [BugFix] Use different mechanism to get vllm version in `is_cpu()` by @njhill in https://github.com/vllm-project/vllm/pull/3804 * [Doc] Update README.md by @robertgshaw2-neuralmagic in https://github.com/vllm-project/vllm/pull/3806 * [Doc] Update contribution guidelines for better onboarding by @michaelfeil in https://github.com/vllm-project/vllm/pull/3819 * [3/N] Refactor scheduler for chunked prefill scheduling by @rkooo567 in https://github.com/vllm-project/vllm/pull/3550 * Enable scaled FP8 (e4m3fn) KV cache on ROCm (AMD GPU) by @AdrianAbeyta in https://github.com/vllm-project/vllm/pull/3290 * [Misc] Publish 3rd meetup slides by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3835 * Fixes the argument for local_tokenizer_group by @sighingnow in https://github.com/vllm-project/vllm/pull/3754 * [Core] Enable hf_transfer by default if available by @michaelfeil in https://github.com/vllm-project/vllm/pull/3817 * [Bugfix] Add kv_scale input parameter to CPU backend by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3840 * [Core] [Frontend] Make detokenization optional by @mgerstgrasser in https://github.com/vllm-project/vllm/pull/3749 * [Bugfix] Fix args in benchmark_serving by @CatherineSue in https://github.com/vllm-project/vllm/pull/3836 * [Benchmark] Refactor sample_requests in benchmark_throughput by @gty111 in https://github.com/vllm-project/vllm/pull/3613 * [Core] manage nccl via a pypi package & upgrade to pt 2.2.1 by @youkaichao in https://github.com/vllm-project/vllm/pull/3805 * [Hardware][CPU] Update cpu torch to match default of 2.2.1 by @mgoin in https://github.com/vllm-project/vllm/pull/3854 * [Model] Cohere CommandR+ by @saurabhdash2512 in https://github.com/vllm-project/vllm/pull/3829 * [Core] improve robustness of pynccl by @youkaichao in https://github.com/vllm-project/vllm/pull/3860 * [Doc]Add asynchronous engine arguments to documentation. by @SeanGallen in https://github.com/vllm-project/vllm/pull/3810 * [CI/Build] fix pip cache with vllm_nccl & refactor dockerfile to build wheels by @youkaichao in https://github.com/vllm-project/vllm/pull/3859 * [Misc] Add pytest marker to opt-out of global test cleanup by @cadedaniel in https://github.com/vllm-project/vllm/pull/3863 * [Misc] Fix linter issues in examples/fp8/quantizer/quantize.py by @cadedaniel in https://github.com/vllm-project/vllm/pull/3864 * [Bugfix] Fixing requirements.txt by @noamgat in https://github.com/vllm-project/vllm/pull/3865 * [Misc] Define common requirements by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3841 * Add option to completion API to truncate prompt tokens by @tdoublep in https://github.com/vllm-project/vllm/pull/3144 * [Chunked Prefill][4/n] Chunked prefill scheduler. by @rkooo567 in https://github.com/vllm-project/vllm/pull/3853 * [Bugfix] Fix incorrect output on OLMo models in Tensor Parallelism by @Isotr0py in https://github.com/vllm-project/vllm/pull/3869 * [CI/Benchmark] add more iteration and use multiple percentiles for robust latency benchmark by @youkaichao in https://github.com/vllm-project/vllm/pull/3889 * [Core] enable out-of-tree model register by @youkaichao in https://github.com/vllm-project/vllm/pull/3871 * [WIP][Core] latency optimization by @youkaichao in https://github.com/vllm-project/vllm/pull/3890 * [Bugfix] Fix Llava inference with Tensor Parallelism. by @Isotr0py in https://github.com/vllm-project/vllm/pull/3883 * [Model] add minicpm by @SUDA-HLT-ywfang in https://github.com/vllm-project/vllm/pull/3893 * [Bugfix] Added Command-R GPTQ support by @egortolmachev in https://github.com/vllm-project/vllm/pull/3849 * [Bugfix] Enable Proper `attention_bias` Usage in Llama Model Configuration by @Ki6an in https://github.com/vllm-project/vllm/pull/3767 * [Hotfix][CI/Build][Kernel] CUDA 11.8 does not support layernorm optimizations by @mawong-amd in https://github.com/vllm-project/vllm/pull/3782 * [BugFix][Model] Fix commandr RoPE max_position_embeddings by @esmeetu in https://github.com/vllm-project/vllm/pull/3919 * [Core] separate distributed_init from worker by @youkaichao in https://github.com/vllm-project/vllm/pull/3904 * [Misc] [Core] Implement RFC "Augment BaseExecutor interfaces to enable hardware-agnostic speculative decoding" by @cadedaniel in https://github.com/vllm-project/vllm/pull/3837 * [Bugfix] Fix KeyError on loading GPT-NeoX by @jsato8094 in https://github.com/vllm-project/vllm/pull/3925 * [ROCm][Hardware][AMD] Use Triton Kernel for default FA on ROCm by @jpvillam-amd in https://github.com/vllm-project/vllm/pull/3643 * [Misc] Avoid loading incorrect LoRA config by @jeejeelee in https://github.com/vllm-project/vllm/pull/3777 * [Benchmark] Add cpu options to bench scripts by @PZD-CHINA in https://github.com/vllm-project/vllm/pull/3915 * [Bugfix] fix utils.py/merge_dict func TypeError: 'type' object is not subscriptable by @zhaotyer in https://github.com/vllm-project/vllm/pull/3955 * [Bugfix] Fix logits processor when prompt_logprobs is not None by @huyiwen in https://github.com/vllm-project/vllm/pull/3899 * [Bugfix] handle prompt_logprobs in _apply_min_tokens_penalty by @tjohnson31415 in https://github.com/vllm-project/vllm/pull/3876 * [Bugfix][ROCm] Add numba to Dockerfile.rocm by @WoosukKwon in https://github.com/vllm-project/vllm/pull/3962 * [Model][AMD] ROCm support for 256 head dims for Gemma by @jamestwhedbee in https://github.com/vllm-project/vllm/pull/3972 * [Doc] Add doc to state our model support policy by @youkaichao in https://github.com/vllm-project/vllm/pull/3948 * [Bugfix] Remove key sorting for `guided_json` parameter in OpenAi compatible Server by @dmarasco in https://github.com/vllm-project/vllm/pull/3945 * [Doc] Fix getting stared to use publicly available model by @fpaupier in https://github.com/vllm-project/vllm/pull/3963 * [Bugfix] handle hf_config with architectures == None by @tjohnson31415 in https://github.com/vllm-project/vllm/pull/3982 * [WIP][Core][Refactor] move vllm/model_executor/parallel_utils into vllm/distributed and vllm/device_communicators by @youkaichao in https://github.com/vllm-project/vllm/pull/3950 * [Core][5/N] Fully working chunked prefill e2e by @rkooo567 in https://github.com/vllm-project/vllm/pull/3884 * [Core][Model] Use torch.compile to accelerate layernorm in commandr by @youkaichao in https://github.com/vllm-project/vllm/pull/3985 * [Test] Add xformer and flash attn tests by @rkooo567 in https://github.com/vllm-project/vllm/pull/3961 * [Misc] refactor ops and cache_ops layer by @jikunshang in https://github.com/vllm-project/vllm/pull/3913 * [Doc][Installation] delete python setup.py develop by @youkaichao in https://github.com/vllm-project/vllm/pull/3989 * [Kernel] Fused MoE Config for Mixtral 8x22 by @ywang96 in https://github.com/vllm-project/vllm/pull/4002 * fix-bgmv-kernel-640 by @kingljl in https://github.com/vllm-project/vllm/pull/4007 * [Hardware][Intel] Isolate CPUModelRunner and ModelRunner for better maintenance by @bigPYJ1151 in https://github.com/vllm-project/vllm/pull/3824 * [Core] Set `linear_weights` directly on the layer by @Yard1 in https://github.com/vllm-project/vllm/pull/3977 * [Core][Distributed] make init_distributed_environment compatible with init_process_group by @youkaichao in https://github.com/vllm-project/vllm/pull/4014 * Fix echo/logprob OpenAI completion bug by @dylanwhawk in https://github.com/vllm-project/vllm/pull/3441 * [Kernel] Add extra punica sizes to support bigger vocabs by @Yard1 in https://github.com/vllm-project/vllm/pull/4015 * [BugFix] Fix handling of stop strings and stop token ids by @njhill in https://github.com/vllm-project/vllm/pull/3672 * [Doc] Add typing hints / mypy types cleanup by @michaelfeil in https://github.com/vllm-project/vllm/pull/3816 * [Core] Support LoRA on quantized models by @jeejeelee in https://github.com/vllm-project/vllm/pull/4012 * [Frontend][Core] Move `merge_async_iterators` to utils by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/4026 * [Test] Test multiple attn backend for chunked prefill. by @rkooo567 in https://github.com/vllm-project/vllm/pull/4023 * [Bugfix] fix type hint for py 3.8 by @youkaichao in https://github.com/vllm-project/vllm/pull/4036 * [Misc] Fix typo in scheduler.py by @zhuohan123 in https://github.com/vllm-project/vllm/pull/4022 * [mypy] Add mypy type annotation part 1 by @rkooo567 in https://github.com/vllm-project/vllm/pull/4006 * [Core] fix custom allreduce default value by @youkaichao in https://github.com/vllm-project/vllm/pull/4040 * Fix triton compilation issue by @Bellk17 in https://github.com/vllm-project/vllm/pull/3984 * [Bugfix] Fix LoRA bug by @jeejeelee in https://github.com/vllm-project/vllm/pull/4032 * [CI/Test] expand ruff and yapf for all supported python version by @youkaichao in https://github.com/vllm-project/vllm/pull/4037 * [Bugfix] More type hint fixes for py 3.8 by @dylanwhawk in https://github.com/vllm-project/vllm/pull/4039 * [Core][Distributed] improve logging for init dist by @youkaichao in https://github.com/vllm-project/vllm/pull/4042 * [Bugfix] fix_log_time_in_metrics by @zspo in https://github.com/vllm-project/vllm/pull/4050 * [Bugfix] fix_small_bug_in_neuron_executor by @zspo in https://github.com/vllm-project/vllm/pull/4051 * [Kernel] Add punica dimension for Baichuan-13B by @jeejeelee in https://github.com/vllm-project/vllm/pull/4053 * [Frontend] [Core] feat: Add model loading using `tensorizer` by @sangstar in https://github.com/vllm-project/vllm/pull/3476 * [Core] avoid too many cuda context by caching p2p test by @youkaichao in https://github.com/vllm-project/vllm/pull/4021 * [BugFix] Fix tensorizer extra in setup.py by @njhill in https://github.com/vllm-project/vllm/pull/4072 * [Docs] document that mixtral 8x22b is supported by @simon-mo in https://github.com/vllm-project/vllm/pull/4073 * [Misc] Upgrade triton to 2.2.0 by @esmeetu in https://github.com/vllm-project/vllm/pull/4061 * [Bugfix] Fix filelock version requirement by @zhuohan123 in https://github.com/vllm-project/vllm/pull/4075 * [Misc][Minor] Fix CPU block num log in CPUExecutor. by @bigPYJ1151 in https://github.com/vllm-project/vllm/pull/4088 * [Core] Simplifications to executor classes by @njhill in https://github.com/vllm-project/vllm/pull/4071 * [Doc] Add better clarity for tensorizer usage by @sangstar in https://github.com/vllm-project/vllm/pull/4090 * [Bugfix] Fix ray workers profiling with nsight by @rickyyx in https://github.com/vllm-project/vllm/pull/4095 * [Typing] Fix Sequence type GenericAlias only available after Python 3.9. by @rkooo567 in https://github.com/vllm-project/vllm/pull/4092 * [Core] Fix engine-use-ray broken by @rkooo567 in https://github.com/vllm-project/vllm/pull/4105 * LM Format Enforcer Guided Decoding Support by @noamgat in https://github.com/vllm-project/vllm/pull/3868 * [Core] Refactor model loading code by @Yard1 in https://github.com/vllm-project/vllm/pull/4097 * [Speculative decoding 6/9] Integrate speculative decoding with LLMEngine by @cadedaniel in https://github.com/vllm-project/vllm/pull/3894 * [Misc] [CI] Fix CI failure caught after merge by @cadedaniel in https://github.com/vllm-project/vllm/pull/4126 * [CI] Move CPU/AMD tests to after wait by @cadedaniel in https://github.com/vllm-project/vllm/pull/4123 * [Core] replace narrow-usage RayWorkerVllm to general WorkerWrapper to reduce code duplication by @youkaichao in https://github.com/vllm-project/vllm/pull/4024 * [Bugfix] fix output parsing error for trtllm backend by @elinx in https://github.com/vllm-project/vllm/pull/4137 * [Kernel] Add punica dimension for Swallow-MS-7B LoRA by @ucciicci in https://github.com/vllm-project/vllm/pull/4134 * [Typing] Mypy typing part 2 by @rkooo567 in https://github.com/vllm-project/vllm/pull/4043 * [Core] Add integrity check during initialization; add test for it by @youkaichao in https://github.com/vllm-project/vllm/pull/4155 * Allow model to be served under multiple names by @hmellor in https://github.com/vllm-project/vllm/pull/2894 * [Bugfix] Get available quantization methods from quantization registry by @mgoin in https://github.com/vllm-project/vllm/pull/4098 * [Bugfix][Kernel] allow non-power-of-two head sizes in prefix prefill by @mmoskal in https://github.com/vllm-project/vllm/pull/4128 * [Docs] document that Meta Llama 3 is supported by @simon-mo in https://github.com/vllm-project/vllm/pull/4175 * [Bugfix] Support logprobs when using guided_json and other constrained decoding fields by @jamestwhedbee in https://github.com/vllm-project/vllm/pull/4149 * [Misc] Bump transformers to latest version by @njhill in https://github.com/vllm-project/vllm/pull/4176 * [CI/CD] add neuron docker and ci test scripts by @liangfu in https://github.com/vllm-project/vllm/pull/3571 * [Bugfix] Fix CustomAllreduce pcie nvlink topology detection (#3974) by @agt in https://github.com/vllm-project/vllm/pull/4159 * [Core] add an option to log every function call to for debugging hang/crash in distributed inference by @youkaichao in https://github.com/vllm-project/vllm/pull/4079 * Support eos_token_id from generation_config.json by @simon-mo in https://github.com/vllm-project/vllm/pull/4182 * [Bugfix] Fix LoRA loading check by @jeejeelee in https://github.com/vllm-project/vllm/pull/4138 * Bump version of 0.4.1 by @simon-mo in https://github.com/vllm-project/vllm/pull/4177 * [Misc] fix docstrings by @UranusSeven in https://github.com/vllm-project/vllm/pull/4191 * [Bugfix][Core] Restore logging of stats in the async engine by @ronensc in https://github.com/vllm-project/vllm/pull/4150 * [Misc] add nccl in collect env by @youkaichao in https://github.com/vllm-project/vllm/pull/4211 * Pass `tokenizer_revision` when getting tokenizer in openai serving by @chiragjn in https://github.com/vllm-project/vllm/pull/4214 * [Bugfix] Add fix for JSON whitespace by @ayusher in https://github.com/vllm-project/vllm/pull/4189 * Fix missing docs and out of sync `EngineArgs` by @hmellor in https://github.com/vllm-project/vllm/pull/4219 * [Kernel][FP8] Initial support with dynamic per-tensor scaling by @comaniac in https://github.com/vllm-project/vllm/pull/4118 * [Frontend] multiple sampling params support by @nunjunj in https://github.com/vllm-project/vllm/pull/3570 * Updating lm-format-enforcer version and adding links to decoding libraries in docs by @noamgat in https://github.com/vllm-project/vllm/pull/4222 * Don't show default value for flags in `EngineArgs` by @hmellor in https://github.com/vllm-project/vllm/pull/4223 * [Doc]: Update the page of adding new models by @YeFD in https://github.com/vllm-project/vllm/pull/4236 * Make initialization of tokenizer and detokenizer optional by @GeauxEric in https://github.com/vllm-project/vllm/pull/3748 * [AMD][Hardware][Misc][Bugfix] xformer cleanup and light navi logic and CI fixes and refactoring by @hongxiayang in https://github.com/vllm-project/vllm/pull/4129 * [Core][Distributed] fix _is_full_nvlink detection by @youkaichao in https://github.com/vllm-project/vllm/pull/4233 * [Misc] Add vision language model support to CPU backend by @Isotr0py in https://github.com/vllm-project/vllm/pull/3968 * [Bugfix] Fix type annotations in CPU model runner by @WoosukKwon in https://github.com/vllm-project/vllm/pull/4256 * [Frontend] Enable support for CPU backend in AsyncLLMEngine. by @sighingnow in https://github.com/vllm-project/vllm/pull/3993 * [Bugfix] Ensure download_weights_from_hf(..) inside loader is using the revision parameter by @alexm-nm in https://github.com/vllm-project/vllm/pull/4217 * Add example scripts to documentation by @hmellor in https://github.com/vllm-project/vllm/pull/4225 * [Core] Scheduler perf fix by @rkooo567 in https://github.com/vllm-project/vllm/pull/4270 * [Doc] Update the SkyPilot doc with serving and Llama-3 by @Michaelvll in https://github.com/vllm-project/vllm/pull/4276 * [Core][Distributed] use absolute path for library file by @youkaichao in https://github.com/vllm-project/vllm/pull/4271 * Fix `autodoc` directives by @hmellor in https://github.com/vllm-project/vllm/pull/4272 * [Mypy] Part 3 fix typing for nested directories for most of directory by @rkooo567 in https://github.com/vllm-project/vllm/pull/4161 * [Core] Some simplification of WorkerWrapper changes by @njhill in https://github.com/vllm-project/vllm/pull/4183 * [Core] Scheduling optimization 2 by @rkooo567 in https://github.com/vllm-project/vllm/pull/4280 * [Speculative decoding 7/9] Speculative decoding end-to-end correctness tests. by @cadedaniel in https://github.com/vllm-project/vllm/pull/3951 * [Bugfix] Fixing max token error message for openai compatible server by @jgordley in https://github.com/vllm-project/vllm/pull/4016 * [Bugfix] Add init_cached_hf_modules to RayWorkerWrapper by @DefTruth in https://github.com/vllm-project/vllm/pull/4286 * [Core][Logging] Add last frame information for better debugging by @youkaichao in https://github.com/vllm-project/vllm/pull/4278 * [CI] Add ccache for wheel builds job by @simon-mo in https://github.com/vllm-project/vllm/pull/4281 * AQLM CUDA support by @jaemzfleming in https://github.com/vllm-project/vllm/pull/3287 * [Bugfix][Frontend] Raise exception when file-like chat template fails to be opened by @DarkLight1337 in https://github.com/vllm-project/vllm/pull/4292 * [Kernel] FP8 support for MoE kernel / Mixtral by @pcmoritz in https://github.com/vllm-project/vllm/pull/4244 * [Bugfix] fixed fp8 conflict with aqlm by @robertgshaw2-neuralmagic in https://github.com/vllm-project/vllm/pull/4307 * [Core][Distributed] use cpu/gloo to initialize pynccl by @youkaichao in https://github.com/vllm-project/vllm/pull/4248 * [CI][Build] change pynvml to nvidia-ml-py by @youkaichao in https://github.com/vllm-project/vllm/pull/4302 * [Misc] Reduce supported Punica dtypes by @WoosukKwon in https://github.com/vllm-project/vllm/pull/4304 ## New Contributors * @mawong-amd made their first contribution in https://github.com/vllm-project/vllm/pull/3662 * @Qubitium made their first contribution in https://github.com/vllm-project/vllm/pull/3689 * @bigPYJ1151 made their first contribution in https://github.com/vllm-project/vllm/pull/3634 * @A-Mahla made their first contribution in https://github.com/vllm-project/vllm/pull/3788 * @AdrianAbeyta made their first contribution in https://github.com/vllm-project/vllm/pull/3290 * @mgerstgrasser made their first contribution in https://github.com/vllm-project/vllm/pull/3749 * @CatherineSue made their first contribution in https://github.com/vllm-project/vllm/pull/3836 * @saurabhdash2512 made their first contribution in https://github.com/vllm-project/vllm/pull/3829 * @SeanGallen made their first contribution in https://github.com/vllm-project/vllm/pull/3810 * @SUDA-HLT-ywfang made their first contribution in https://github.com/vllm-project/vllm/pull/3893 * @egortolmachev made their first contribution in https://github.com/vllm-project/vllm/pull/3849 * @Ki6an made their first contribution in https://github.com/vllm-project/vllm/pull/3767 * @jsato8094 made their first contribution in https://github.com/vllm-project/vllm/pull/3925 * @jpvillam-amd made their first contribution in https://github.com/vllm-project/vllm/pull/3643 * @PZD-CHINA made their first contribution in https://github.com/vllm-project/vllm/pull/3915 * @zhaotyer made their first contribution in https://github.com/vllm-project/vllm/pull/3955 * @huyiwen made their first contribution in https://github.com/vllm-project/vllm/pull/3899 * @dmarasco made their first contribution in https://github.com/vllm-project/vllm/pull/3945 * @fpaupier made their first contribution in https://github.com/vllm-project/vllm/pull/3963 * @kingljl made their first contribution in https://github.com/vllm-project/vllm/pull/4007 * @DarkLight1337 made their first contribution in https://github.com/vllm-project/vllm/pull/4026 * @Bellk17 made their first contribution in https://github.com/vllm-project/vllm/pull/3984 * @sangstar made their first contribution in https://github.com/vllm-project/vllm/pull/3476 * @rickyyx made their first contribution in https://github.com/vllm-project/vllm/pull/4095 * @elinx made their first contribution in https://github.com/vllm-project/vllm/pull/4137 * @ucciicci made their first contribution in https://github.com/vllm-project/vllm/pull/4134 * @mmoskal made their first contribution in https://github.com/vllm-project/vllm/pull/4128 * @agt made their first contribution in https://github.com/vllm-project/vllm/pull/4159 * @ayusher made their first contribution in https://github.com/vllm-project/vllm/pull/4189 * @nunjunj made their first contribution in https://github.com/vllm-project/vllm/pull/3570 * @YeFD made their first contribution in https://github.com/vllm-project/vllm/pull/4236 * @GeauxEric made their first contribution in https://github.com/vllm-project/vllm/pull/3748 * @alexm-nm made their first contribution in https://github.com/vllm-project/vllm/pull/4217 * @jgordley made their first contribution in https://github.com/vllm-project/vllm/pull/4016 * @DefTruth made their first contribution in https://github.com/vllm-project/vllm/pull/4286 * @jaemzfleming made their first contribution in https://github.com/vllm-project/vllm/pull/3287 **Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.4.0...v0.4.1