Impact Score: 5/10
Hugging Face Transformers v3.3.0 — Release Notes & Engineering Impact
Source available — automated, not human-reviewed
Component: transformers · Channel: stable. Automated analysis, not a compatibility certification. How this works.
Analysis updated: 2026-11-01.
⚡ Executive Summary (TL;DR)
Hugging Face Transformers v3.3.0. Source highlight: The RAG model is a retrieval-augmented generation model that can be leveraged for question-answering tasks using `RagTokenForGeneration` or `RagSequenceForGeneration` as proposed in Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks by Patrick Lewis, Ethan Perez, Aleksandara Piktus, Fa
Who Does This Impact? (Role-Based Analysis)
💻 Software Engineers & Devs
Check the source notes against the package and APIs you use before upgrading.
📊 Product & Engineering Leads
Use the documented changes to assess relevance to your product; release frequency alone is not a quality measure.
⚙️ DevOps & Security
Verify the target version, runtime requirements and rollback procedure in the official documentation.
✨ New Features & Capabilities (2)
- [RAG] Add `attention_mask` to RAG generate #7373 (@patrickvonplaten)
- [RAG] Add missing doc and attention_mask to rag #7382 (@patrickvonplaten)
🔧 Bug Fixes & Patches (6)
- [Rag] Fix wrong usage of `num_beams` and `bos_token_id` in Rag Sequence generation #7386 (@patrickvonplaten)
- [RAG] Fix retrieval offset in RAG's HfIndex and better integration tests #7372 (@lhoestq)
- [Rag] fix rag retriever save_pretrained method #7399 (@patrickvonplaten)
- Mark big downloads slow #7325 (@sgugger)
- [Bug Fix] The actual batch_size is inconsistent with the settings. #7235 (@HuangLianzhe)
- Fixed results of SQuAD-FR evaluation #7313 (@psorianom)
📄 View Raw Release Notes (Original Developer Log)
# RAG
## RAG Model
The RAG model is a retrieval-augmented generation model that can be leveraged for question-answering tasks using `RagTokenForGeneration` or `RagSequenceForGeneration` as proposed in [Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks](https://arxiv.org/abs/2005.11401) by Patrick Lewis, Ethan Perez, Aleksandara Piktus, Fabio Petroni, Vladimir Karpukhin, Naman Goyal, Heinrich Küttler, Mike Lewis, Wen-tau Yih, Tim Rocktäschel, Sebastian Riedel, Douwe Kiela.
It was added to the library in PyTorch with the following checkpoints:
- `facebook/rag-token-nq`
- `facebook/rag-sequence-nq`
- `facebook/rag-token-base`
- `facebook/rag-sequence-base`
### Contributions:
- **RAG** #6813 (@ola13)
- [RAG] Add `attention_mask` to RAG generate #7373 (@patrickvonplaten)
- [RAG] Add missing doc and attention_mask to rag #7382 (@patrickvonplaten)
- [Rag] Fix wrong usage of `num_beams` and `bos_token_id` in Rag Sequence generation #7386 (@patrickvonplaten)
- [RAG] Fix retrieval offset in RAG's HfIndex and better integration tests #7372 (@lhoestq)
- [RAG] Remove dependency on `examples/seq2seq` from rag #7395 (@ola13)
- [Rag] fix rag retriever save_pretrained method #7399 (@patrickvonplaten)
- [RAG] Clean Rag readme in examples #7413 (@ola13)
- [RAG] Model cards - clean cards #7420 (@patrickvonplaten)
- Document RAG again #7377 (@sgugger)
## Bug fixes and improvements
- Mark big downloads slow #7325 (@sgugger)
- [Bug Fix] The actual batch_size is inconsistent with the settings. #7235 (@HuangLianzhe)
- Fixed results of SQuAD-FR evaluation #7313 (@psorianom)
- [s2s] add supported architecures to MD #7252 (@sshleifer)
- Add num workers cli arg #7322 (@chadykamar)
- [s2s] add src_lang kwarg for distributed eval #7300 (@sshleifer)
- [s2s] only save metrics.json from rank zero #7331 (@sshleifer)
- [code quality] fix confused flake8 #7309 (@stas00)
- [testing] skip decorators: docs, tests, bugs #7334 (@stas00)
- Fixed evaluation_strategy on epoch end bug #7340 (@WissamAntoun)
- Models doc #7345 (@sgugger)
- Ensure that integrations are imported before transformers or ml libs #7330 (@dsblank)
- [Benchmarks] Change all args to from `no_...` to their positive form #7075 (@fmcurti)
- Remove reference to args in XLA check #7344 (@ZeroCool2u)
- wip: Code to add lang tags to marian model cards #6586 (@sshleifer)
- Expand a bit the documentation doc #7350 (@sgugger)
- Check decorator order #7326 (@sgugger)
- Update modeling_tf_longformer.py #7359 (@Line290)
- Updata tokenization_auto.py #6870 (@hjptriplebee)
- Update the TF models to remove their interdependencies #7238 (@jplu)
- Make PyTorch model files independent from each other #7352 (@sgugger)
- Clean RAG docs and template docs #7348 (@sgugger)
- Fixing case in which `Trainer` hung while saving model in distributed training #7365 (@TevenLeScao)
- Formatter #7368 (@LysandreJik)
- [seq2seq] make it easier to run the scripts #7274 (@stas00)
- Remove mentions of RAG from the docs #7376 (@sgugger)
- [fsmt] build/test scripts #7257 (@stas00)
- [s2s] distributed eval allows num_return_sequences > 1 #7254 (@sshleifer)
- Seq2SeqTrainer #6769 (@patil-suraj)
- modeling_bart: 3 small cleanups that dont change outputs #7381 (@sshleifer)
- Check config type using `type` instead of `isinstance` #7363 (@LysandreJik)
- [s2s, examples] minor doc changes #7385 (@patil-suraj)
- Remove unhelpful bart warning #7391 (@sshleifer)
- [code quality] new make target that combines style and quality targets #7310 (@stas00)
- Speedup check_copies script #7394 (@sgugger)
- Fix BartModel output documentation #7390 (@sgugger)
- Fix FP16 and attention masks in FunnelTransformer #7374 (@sgugger)
- [Longformer, Bert, Roberta, ...] Fix multi gpu training #7272 (@patrickvonplaten)
- [s2s] add create student script #7290 (@patil-suraj)
- [s2s] rougeLSum expects \n between sentences #7410 (@sshleifer)
- [T5] allow config.decoder_layers to control decoer size #7409 (@sshleifer)
- Flos fix #7384 (@marrrcin)
- Catch PyTorch warning when saving/loading scheduler #7401 (@sgugger)
- Pull request template #7392 (@LysandreJik)
- Reorganize documentation navbar #7423 (@sgugger)