vLLM v0.2.0 — Release Notes & Engineering Impact

## Major changes * Up to 60% performance improvement by optimizing de-tokenization and sampler * Initial support for AWQ (performance not optimized) * Support for RoPE scaling and LongChat * Support for Mistral-7B * Many bug fixes ## What's Changed * add option to shorten prompt print in log by @leiwen83 in https://github.com/vllm-project/vllm/pull/991 * Make `max_model_len` configurable by @Yard1 in https://github.com/vllm-project/vllm/pull/972 * Fix typo in README.md by @eltociear in https://github.com/vllm-project/vllm/pull/1033 * Use TGI-like incremental detokenization by @Yard1 in https://github.com/vllm-project/vllm/pull/984 * Add Model Revision Support in https://github.com/vllm-project/vllm/pull/1014 * [FIX] Minor bug fixes by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1035 * Announce paper release by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1036 * Fix detokenization leaving special tokens by @Yard1 in https://github.com/vllm-project/vllm/pull/1044 * Add pandas to requirements.txt by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1047 * OpenAI-Server: Only fail if logit_bias has actual values by @LLukas22 in https://github.com/vllm-project/vllm/pull/1045 * Fix warning message on LLaMA FastTokenizer by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1037 * Abort when coroutine is cancelled by @rucyang in https://github.com/vllm-project/vllm/pull/1020 * Implement AWQ quantization support for LLaMA by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1032 * Remove AsyncLLMEngine busy loop, shield background task by @Yard1 in https://github.com/vllm-project/vllm/pull/1059 * Fix hanging when prompt exceeds limit by @chenxu2048 in https://github.com/vllm-project/vllm/pull/1029 * [FIX] Don't initialize parameter by default by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1067 * added support for quantize on LLM module by @orellavie1212 in https://github.com/vllm-project/vllm/pull/1080 * align llm_engine and async_engine step method. by @esmeetu in https://github.com/vllm-project/vllm/pull/1081 * Fix get_max_num_running_seqs for waiting and swapped seq groups by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1068 * Add safetensors support for quantized models by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1073 * Add minimum capability requirement for AWQ by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1064 * [Community] Add vLLM Discord server by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1086 * Add pyarrow to dependencies & Print warning on Ray import error by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1094 * Add gpu_memory_utilization and swap_space to LLM by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1090 * Add documentation to Triton server tutorial by @tanmayv25 in https://github.com/vllm-project/vllm/pull/983 * rope_theta and max_position_embeddings from config by @Yard1 in https://github.com/vllm-project/vllm/pull/1096 * Replace torch.cuda.DtypeTensor with torch.tensor by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1123 * Add float16 and float32 to dtype choices by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1115 * clean api code, remove redundant background task. by @esmeetu in https://github.com/vllm-project/vllm/pull/1102 * feat: support stop_token_ids parameter. by @gesanqiu in https://github.com/vllm-project/vllm/pull/1097 * Use `--ipc=host` in `docker run` for distributed inference by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1125 * Docs: Fix broken link to openai example by @nkpz in https://github.com/vllm-project/vllm/pull/1145 * Announce the First vLLM Meetup by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1148 * [Sampler] Vectorized sampling (simplified) by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1048 * [FIX] Simplify sampler logic by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1156 * Fix config for Falcon by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1164 * Align `max_tokens` behavior with openai by @HermitSun in https://github.com/vllm-project/vllm/pull/852 * [Setup] Enable `TORCH_CUDA_ARCH_LIST` for selecting target GPUs by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1074 * Add comments on RoPE initialization by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1176 * Allocate more shared memory to attention kernel by @Yard1 in https://github.com/vllm-project/vllm/pull/1154 * Support Longchat by @LiuXiaoxuanPKU in https://github.com/vllm-project/vllm/pull/555 * fix typo (?) by @WrRan in https://github.com/vllm-project/vllm/pull/1184 * fix qwen-14b model by @Sanster in https://github.com/vllm-project/vllm/pull/1173 * Automatically set `max_num_batched_tokens` by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1198 * Use standard extras for `uvicorn` by @danilopeixoto in https://github.com/vllm-project/vllm/pull/1166 * Keep special sampling params by @blahblahasdf in https://github.com/vllm-project/vllm/pull/1186 * qwen add rope_scaling by @Sanster in https://github.com/vllm-project/vllm/pull/1210 * [Mistral] Mistral-7B-v0.1 support by @Bam4d in https://github.com/vllm-project/vllm/pull/1196 * Fix Mistral model by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1220 * [Fix] Remove false assertion by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1222 * Add Mistral to supported model list by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1221 * Fix OOM in attention kernel test by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1223 * Provide default max model length by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1224 * Bump up the version to v0.2.0 by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1212 ## New Contributors * @leiwen83 made their first contribution in https://github.com/vllm-project/vllm/pull/991 * @LLukas22 made their first contribution in https://github.com/vllm-project/vllm/pull/1045 * @rucyang made their first contribution in https://github.com/vllm-project/vllm/pull/1020 * @chenxu2048 made their first contribution in https://github.com/vllm-project/vllm/pull/1029 * @orellavie1212 made their first contribution in https://github.com/vllm-project/vllm/pull/1080 * @tanmayv25 made their first contribution in https://github.com/vllm-project/vllm/pull/983 * @nkpz made their first contribution in https://github.com/vllm-project/vllm/pull/1145 * @WrRan made their first contribution in https://github.com/vllm-project/vllm/pull/1184 * @danilopeixoto made their first contribution in https://github.com/vllm-project/vllm/pull/1166 * @blahblahasdf made their first contribution in https://github.com/vllm-project/vllm/pull/1186 * @Bam4d made their first contribution in https://github.com/vllm-project/vllm/pull/1196 **Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.1.7...v0.2.0