Papers of the day   All papers

FlauBERT: Unsupervised Language Model Pre-training for French


Jan 27 2020 Sebastian Ruder

Transfer learning is increasingly going multilingual with language-specific BERT models: - 🇩🇪 German BERT - 🇫🇷 CamemBERT, FlauBERT - 🇮🇹 AlBERTo - 🇳🇱 RobBERT
18 replies, 615 likes

Jan 08 2020 Jeremy Howard

TIL Google invented universal language model pre-training with BERT.
18 replies, 578 likes

Jan 30 2020 laurent besacier

Our FlauBERT (French BERT) models have now been integrated into official @huggingface library with 4 below configurations !
3 replies, 105 likes

Jan 30 2020 Hang Le

Our FlauBERT is now natively supported by @huggingface's transformers library. Many thanks to @julien_c, @LysandreJik and the Hugging Face team for the active technical support! Paper (new version will be available soon): Code:
0 replies, 94 likes

Jan 27 2020 Hugging Face

You can now find most of them here:
0 replies, 53 likes

Jan 08 2020 D. Khuê Lê-Huu

This reminds me of this Flaubert paper: that gave proper credits to ULMFiT (by @jeremyphoward and @seb_ruder).
2 replies, 49 likes

Mar 16 2020 laurent besacier

The (LREC) camera-ready paper on FlauBERT is now online: . Includes new results with FlauBERT_large. All models available on @huggingface transformers library. Benchmark NLP tasks (FLUE) provided on
0 replies, 47 likes

Dec 12 2019 laurent besacier

Here is FlauBERT: a French LM learnt (with #CNRS J-Zay supercomputer) on a large and heterogeneous corpus. Along with it comes FLUE (evaluation setup for French NLP). FlauBERT was successfully applied to complex tasks (NLI, WSD, Parsing). More on
0 replies, 42 likes

Jan 27 2020 Jay Alammar جهاد العمار

Somebody please make AraBERT happen!
3 replies, 13 likes

Dec 12 2019 Hang Le

Our work on FlauBERT and FLUE (language models and evaluation benchmark for French) have been released today (198th birthday of Gustave Flaubert). #Flaubert Paper: Code and models:
1 replies, 11 likes

Dec 15 2019 Dr Jochen L Leidner

French #NLP with the Transformer: From #BERT over CamemBERT to FlauBERT
0 replies, 8 likes

Dec 12 2019 Machine Learning

FlauBERT: Unsupervised Language Model Pre-training for French.
0 replies, 7 likes

Dec 19 2019 Julien Velcin

Between CamemBERT ( and FlauBERT (, which one will win the race? Anyway thank you for working on French-oriented NLP resources, and well done for finding such interesting names! #nlp #deeplearning #bert
0 replies, 6 likes

Jan 31 2020 Dominique Mariko

Comes with FLUE benchmark ! A GLUE for French!!
0 replies, 5 likes

Dec 13 2019 Christopher

FlauBERT - Unsupervised Language Model Pre-training for French. The repo contains pre-trained large & small models, all the data used plus code for training & inference. It also contains FLUE, a GLUE like benchmark for French NLProc
0 replies, 2 likes


PDF content of a computer science paper: FlauBERT: Unsupervised Language Model Pre-training for French"