Hugging Face Transformers: ELECTRA, Bad word filters, bugfixes & improvements

# ELECTRA Model (@LysandreJik) ELECTRA is a new method for self-supervised language representation learning. It can be used to pre-train transformer networks using relatively little compute. ELECTRA models are trained to distinguish "real" input tokens vs "fake" input tokens generated by another neural network, similar to the discriminator of a GAN. At small scale, ELECTRA achieves strong results even when trained on a single GPU. At large scale, ELECTRA achieves state-of-the-art results on the SQuAD 2.0 dataset. This release comes with 6 ELECTRA checkpoints: - `google/electra-small-discriminator` - `google/electra-small-generator` - `google/electra-base-discriminator` - `google/electra-base-generator` - `google/electra-large-discriminator` - `google/electra-large-generator` Related: - [Paper](https://openreview.net/pdf?id=r1xMH1BtvB) - [Original implementation in TF 1.x](https://github.com/google-research/electra) - 📊 [**Model cards** on huggingface.co](https://huggingface.co/models?search=electra) - [Docs](https://huggingface.co/transformers/model_doc/electra.html) Thanks to the author @clarkkev for his help during the implementation. Thanks to community members @hfl-rc @stefan-it @shoarora for already sharing more fine-tuned Electra variants! # Bad word filters in `generate` (@patrickvonplaten) The `generate` method now has a bad word filter. # Fixes and improvements - Decoder input ids are not necessary for T5 training anymore (@patrickvonplaten) - Update encoder and decoder on set_input_embedding for BART (@sshleifer) - Using loaded checkpoint with --do_predict (instead of random init) for Pytorch-lightning scripts (@ethanjperez) - Clean summarization and translation example testing files for T5 and Bart (@patrickvonplaten) - Cleaner examples (@julien-c) - Extensive testing for T5 model (@patrickvonplaten) - Force models outputs to always have batch_size as their first dim (@patrickvonplaten) - Fix for continuing training in some scripts (@xeb) - Resizing embedding matrix before sending it to the optimizer (@ngarneau) - BertJapaneseTokenizer accept options for mecab (@tamuhey) - Speed up GELU computation with torch.jit (@mryab) - fix argument order of update_mems fn in TF version (@patrickvonplaten, @dmytyar) - Split generate test function into beam search, no beam search (@patrickvonplaten)