Papers of the day   All papers

Language Models as Knowledge Bases?


Sep 26 2019 Graham Neubig

"Language models as knowledge bases?" they asked: "A cat has four kidneys", replied GPT-2.
23 replies, 590 likes

Sep 04 2019 Fabio Petroni

Can language models replace knowledge bases? In our recent @EMNLP2019 paper we present the LAMA probe for analyzing the knowledge contained in pretrained language models. w/ @_rockt @PSH_Lewis @anton_bakhtin @mindjimmy @riedelcastro from @FacebookAI
4 replies, 467 likes

Sep 05 2019 (((ل()(ل() 'yoav))))

no, they really CANNOT replace knowledge bases yet. they recover correctly only a fraction of the facts in the KB, are restricted to single tokens, and when they don't know the answer they just make something up.
10 replies, 305 likes

Sep 04 2019 Andrew Beam

Question: How do we unify language models and knowledge bases? Answer: Oops, I think we already did... by accident. Language models implicitly contain information that can be queried like a knowledge base. Cool new work from FAIR and UCL: Paper:
2 replies, 215 likes

Oct 22 2019 hardmaru 😷

Language Models as Knowledge Bases? They present an in-depth analysis of the relational knowledge already present (without fine-tuning) in a wide range of state-of-the-art pretrained language models. h/t @Synced_Global
6 replies, 166 likes

Oct 22 2019 Synced

.@facebookai and @ucl have introduced their LAMA (LAnguage Model Analysis) probe to explore the feasibility of using language models as knowledge bases. #LANGUAGE #ArtificialIntelligence #MachineLearning
0 replies, 143 likes

Sep 04 2019 Tim Rocktäschel

Excited to share our work on assessing factual and commonsense knowledge in pretrained language models. Turns out they are surprisingly competitive to knowledge bases that are constructed from recent relation extraction pipelines. Great work by @Fabio_Petroni and the team!
0 replies, 122 likes

Sep 10 2019 WikiResearch

"Language Models as Knowledge Bases?" - an analysis of the relational knowledge present in pretrained language models shows a strong ability of these models to recall factual knowledge - tested on #Wikidata! (Petroni et al., 2019)
1 replies, 70 likes

Sep 04 2019 Patrick Lewis

How much knowledge do language models have? How does that knowledge compare traditional KB? @Fabio_Petroni ‘s LAMA probe suggests that some language models are actually pretty good KBs! Read our @emnlp2019 paper, with @riedelcastro @_rockt @anton_bakhtin @mindjimmy
0 replies, 43 likes

Sep 26 2019 Alex Polozov

I love everything about this. Not only GPT-2 doesn't understand anatomy (of course), but it also starts the claim with four kidneys and proceeds to list a grand total of five!
1 replies, 41 likes

Oct 22 2019 Daisuke Okanohara

Language models (e.g. BERT) are competitive with structured knowledge bases for several NLP tasks. LM requires no schema engineering, easy to exploit more data. How to make an effective question to extract knowledge may be an interesting direction.
0 replies, 21 likes

Sep 11 2019 Jonathan Fly 👾

BERT versus ConceptNET: Time is _______. RIGHT ANSWER: Finite. BERT's GUESSES: short, passing, precious, irrelevant, gone 😔 Bert is a poet. "Language Models as Knowledge Bases?"
1 replies, 19 likes

Sep 05 2019 Ben Lorica 罗瑞卡

"analysis of the factual & commonsense knowledge in publicly available pretrained language models & found that BERT-large is able to recall such knowledge ... at a level remarkably competitive with non-neural and supervised alternatives" 🆕 #Nlproc
1 replies, 15 likes

Sep 27 2019 Andy Way

Anyone who thinks #AI can simply replace humans should have a look at this ... 😂 Jump on anyone who says #NLProc is a ‘solved problem’ just because deep learning is involved!
0 replies, 14 likes

Oct 20 2019 Montreal.AI

Language Models as Knowledge Bases? Petroni et al.: #Transformers #NaturalLanguageProcessing #MachineLearning
0 replies, 9 likes

Sep 26 2019 Andromeda Yelton

So not ONLY is this hilarious but it is an excellent demonstration of why we maybe shouldn't trust AI *too* much because here's what GPT-2 has to say on library policy.
0 replies, 6 likes

Sep 26 2019 Jeffrey Snover

AI for natural language processing NLP is advancing rapidly but obviously - miles to go before we sleep. Given this description of cats, I won’t be sleeping with both eyes closed for a while. :-)
1 replies, 4 likes

Sep 12 2019 The Institute for Ethical AI & Machine Learning

Language models as knowledgebases. Interesting perspective on how the potential of languge models can be generalised. In this paper, the authors argue that language models may also be storing relational knowledge present in the training data.
0 replies, 4 likes

Sep 26 2019 amandalynne (she)

0 replies, 4 likes

Sep 27 2019 nutanc

GPT2 like many other deep learning models is just good for fun(very costly fun though). Cannot be used in a business environment.
0 replies, 2 likes

Dec 06 2019 Matt Krause

@jgordonfisher @GaryMarcus @huggingface But people *are* making the argument that LMs "know" things:
1 replies, 2 likes

Oct 22 2019 4InData

#4InData #Transformers #NaturalLanguageProcessing #MachineLearning Language Models as Knowledge Bases? Petroni et al.:
0 replies, 1 likes

Sep 05 2019 arXiv CS-CL

Language Models as Knowledge Bases?
0 replies, 1 likes

Oct 21 2019 Hicham EL BOUKKOURI

Very interesting work by @facebookai & UCL for analysing the factual and common-sense knowledge that is available in large language models such as BERT, ELMo or GPT-2. Paper: GitHub:
0 replies, 1 likes

Sep 04 2019 Andrey Kurenkov 🤖

Super cool! As with the recent paper analyzing memorization in language models, another good question addressed here (how much knowledge about the world they end up containing). Good to see more of this type of research.
1 replies, 1 likes

Sep 27 2019 Katie Anderson

I'm glad to finally know the scientific reasoning behind my black and white cat's coloring.
0 replies, 1 likes


PDF content of a computer science paper: Language Models as Knowledge Bases?"