vLLM v0.2.1 — Release Notes & Engineering Impact

## Major Changes * PagedAttention V2 kernel: Up to 20% end-to-end latency reduction * Support log probabilities for prompt tokens * AWQ support for Mistral 7B ## What's Changed * fixing typo in `tiiuae/falcon-rw-7b` model name by @0ssamaak0 in https://github.com/vllm-project/vllm/pull/1226 * Added `dtype` arg to benchmarks by @kg6-sleipnir in https://github.com/vllm-project/vllm/pull/1228 * fix vulnerable memory modification to gpu shared memory by @soundOfDestiny in https://github.com/vllm-project/vllm/pull/1241 * support sharding llama2-70b on more than 8 GPUs by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1209 * [Minor] Fix type annotations by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1238 * TP/quantization/weight loading refactor part 1 - Simplify parallel linear logic by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1181 * add support for tokenizer revision by @cassanof in https://github.com/vllm-project/vllm/pull/1163 * Use monotonic time where appropriate by @Yard1 in https://github.com/vllm-project/vllm/pull/1249 * API server support ipv4 / ipv6 dualstack by @yunfeng-scale in https://github.com/vllm-project/vllm/pull/1288 * Move bfloat16 check to worker by @Yard1 in https://github.com/vllm-project/vllm/pull/1259 * [FIX] Explain why the finished_reason of ignored sequences are length by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1289 * Update README.md by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1292 * [Minor] Fix comment in mistral.py by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1303 * lock torch version to 2.0.1 when build for #1283 by @yanxiyue in https://github.com/vllm-project/vllm/pull/1290 * minor update by @WrRan in https://github.com/vllm-project/vllm/pull/1311 * change the timing of sorting logits by @yhlskt23 in https://github.com/vllm-project/vllm/pull/1309 * workaround of AWQ for Turing GPUs by @twaka in https://github.com/vllm-project/vllm/pull/1252 * Fix overflow in awq kernel by @chu-tianxiang in https://github.com/vllm-project/vllm/pull/1295 * Update model_loader.py by @AmaleshV in https://github.com/vllm-project/vllm/pull/1278 * Add blacklist for model checkpoint by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1325 * Update README.md Aquila2. by @ftgreat in https://github.com/vllm-project/vllm/pull/1331 * Improve detokenization performance by @Yard1 in https://github.com/vllm-project/vllm/pull/1338 * Bump up transformers version & Remove MistralConfig by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1254 * Fix the issue for AquilaChat2-* models by @lu-wang-dl in https://github.com/vllm-project/vllm/pull/1339 * Fix error message on `TORCH_CUDA_ARCH_LIST` by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1239 * Minor fix on AWQ kernel launch by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1356 * Implement PagedAttention V2 by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1348 * Implement prompt logprobs & Batched topk for computing logprobs by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1328 * Fix PyTorch version to 2.0.1 in workflow by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1377 * Fix PyTorch index URL in workflow by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1378 * Fix sampler test by @WoosukKwon in https://github.com/vllm-project/vllm/pull/1379 * Bump up the version to v0.2.1 by @zhuohan123 in https://github.com/vllm-project/vllm/pull/1355 ## New Contributors * @0ssamaak0 made their first contribution in https://github.com/vllm-project/vllm/pull/1226 * @kg6-sleipnir made their first contribution in https://github.com/vllm-project/vllm/pull/1228 * @soundOfDestiny made their first contribution in https://github.com/vllm-project/vllm/pull/1241 * @cassanof made their first contribution in https://github.com/vllm-project/vllm/pull/1163 * @yunfeng-scale made their first contribution in https://github.com/vllm-project/vllm/pull/1288 * @yanxiyue made their first contribution in https://github.com/vllm-project/vllm/pull/1290 * @yhlskt23 made their first contribution in https://github.com/vllm-project/vllm/pull/1309 * @chu-tianxiang made their first contribution in https://github.com/vllm-project/vllm/pull/1295 * @AmaleshV made their first contribution in https://github.com/vllm-project/vllm/pull/1278 * @lu-wang-dl made their first contribution in https://github.com/vllm-project/vllm/pull/1339 **Full Changelog**: https://github.com/vllm-project/vllm/compare/v0.2.0...v0.2.1