Papers of the day   All papers

FlauBERT: Unsupervised Language Model Pre-training for French


Sebastian Ruder: Transfer learning is increasingly going multilingual with language-specific BERT models: - 🇩🇪 German BERT - 🇫🇷 CamemBERT, FlauBERT - 🇮🇹 AlBERTo - 🇳🇱 RobBERT

19 replies, 613 likes

Jeremy Howard: TIL Google invented universal language model pre-training with BERT.

18 replies, 578 likes

laurent besacier: Our FlauBERT (French BERT) models have now been integrated into official @huggingface library with 4 below configurations !

3 replies, 105 likes

Hang Le: Our FlauBERT is now natively supported by @huggingface's transformers library. Many thanks to @julien_c, @LysandreJik and the Hugging Face team for the active technical support! Paper (new version will be available soon): Code:

0 replies, 94 likes

Hugging Face: You can now find most of them here:

0 replies, 53 likes

D. Khuê Lê-Huu: This reminds me of this Flaubert paper: that gave proper credits to ULMFiT (by @jeremyphoward and @seb_ruder).

2 replies, 49 likes

laurent besacier: The (LREC) camera-ready paper on FlauBERT is now online: . Includes new results with FlauBERT_large. All models available on @huggingface transformers library. Benchmark NLP tasks (FLUE) provided on

0 replies, 47 likes

laurent besacier: Here is FlauBERT: a French LM learnt (with #CNRS J-Zay supercomputer) on a large and heterogeneous corpus. Along with it comes FLUE (evaluation setup for French NLP). FlauBERT was successfully applied to complex tasks (NLI, WSD, Parsing). More on

0 replies, 42 likes

Jay Alammar جهاد العمار: Somebody please make AraBERT happen!

3 replies, 13 likes

Hang Le: Our work on FlauBERT and FLUE (language models and evaluation benchmark for French) have been released today (198th birthday of Gustave Flaubert). #Flaubert Paper: Code and models:

1 replies, 11 likes

Dr Jochen L Leidner: French #NLP with the Transformer: From #BERT over CamemBERT to FlauBERT

0 replies, 8 likes

Machine Learning: FlauBERT: Unsupervised Language Model Pre-training for French.

0 replies, 7 likes

Julien Velcin: Between CamemBERT ( and FlauBERT (, which one will win the race? Anyway thank you for working on French-oriented NLP resources, and well done for finding such interesting names! #nlp #deeplearning #bert

0 replies, 6 likes

Dominique Mariko: Comes with FLUE benchmark ! A GLUE for French!!

0 replies, 5 likes

Christopher: FlauBERT - Unsupervised Language Model Pre-training for French. The repo contains pre-trained large & small models, all the data used plus code for training & inference. It also contains FLUE, a GLUE like benchmark for French NLProc

0 replies, 2 likes


Found on Jan 27 2020 at

PDF content of a computer science paper: FlauBERT: Unsupervised Language Model Pre-training for French