Papers of the day   All papers

Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims


OpenAI: We've contributed to a multi-stakeholder report by 58 co-authors at 30 organizations that describes 10 mechanisms to improve the verifiability of claims made about AI systems.

5 replies, 306 likes

Miles Brundage: Super excited to share a report I’ve been working on with a bunch of colleagues from different orgs since last year. Building on a workshop last April, we analyze various ways to improve the verifiability of claims about AI systems. You can read it here:

6 replies, 153 likes

Dr Heidy Khlaaf (هايدي خلاف): After a year of work, it's finally here! Our report is a collaboration with @OpenAI @PartnershipAI @LeverhulmeCFI @Google and many others! It was a privilege to work with such diverse and expert authors, towards a common goal of trustworthy AI development.

3 replies, 109 likes

Helen Toner: Honored to be involved in creating this report, and excited to share it today! If you've ever been all 🤨 about AI ethics principles, this is for you - it's all about ways we could make verifiable claims about the trustworthiness of AI systems. 1/

2 replies, 75 likes

Emma Bluemke: Existing regulations & norms in industry/academia are insufficient to ensure responsible AI development What hardware, software, and institutional mechanisms can help? Could privacy preserving tech help? See section 3.3

0 replies, 44 likes

Carina Prunkl: Excited to have been part of a multi-stakeholder effort promoting more accountability in AI development. Co-authors include (amog many others) @Miles_Brundage from @OpenAI and @FHIOxford's Allan Dafoe. Preprint version: Key takeaways follow

2 replies, 41 likes

Shagun Sodhani: With co-authors @Miles_Brundage, Shahar Avin, @HaydnBelfield, @j_asminewang, @tegan_maharaj, @GretchenMarina & many others, I am happy to share a multi-stakeholder report “Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims”:

2 replies, 40 likes

Tegan Maharaj: Saying nice things about responsible AI development is great, but actions speak louder than words. Concrete recommendations for making verifiable claims, distilled by a wonderful multidiscplinary group I'm proud to be a part of!

1 replies, 37 likes

Théo Ryffel: I'm *very honored* to have been involved in this report! If you want to move beyond principles in #AIEthics and read about verifiable mechanisms for #privacy #fairness #interpretability, have a look! 👉 👉

4 replies, 28 likes

Gillian Hadfield: How can AI developers credibly raise confidence that they are committed to responsible AI practices? Ideas here:

0 replies, 27 likes

jonathanstray: One big AI problem is how do you prove that a system does what you say it does? Researchers from 26 organizations, including @PartnershipAI, wrote a majestic survey on how to approach this.

2 replies, 24 likes

Ed Teather: This paper is excellent on how to move AI ethics from principles to practice. Particularly enjoy the idea of putting a bounty on bias & developing 3rd party audits. Congrats to all those involved @adrian_weller @jackclarkSF @Miles_Brundage @Lisa_LynnD

0 replies, 23 likes

Paresh Kathrani: New 80 page paper, ‘Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims’ by @Miles_Brundage and others via @arxiv looking at trust etc through applied ethics in ai @JohnAFlood @David_Gunkel @EmergTechEthics @PabloRedux @sd_marlow

2 replies, 16 likes

Lavanya 🧜🏻‍♀️: So proud of my friend @Miles_Brundage for co-authoring this report on the need for verification mechanisms to demonstrate responsible behavior by AI systems. They lay out concrete recs on making verifiable claims to which engineers can be held accountable

0 replies, 16 likes

Ben Laurie: "Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims"

1 replies, 13 likes

Center for Human-Compatible AI: Alongside co-authors from more than 26 institutions, CHAI researcher Tom Gilbert (@sociotiose) contributed to the multistakeholder report, “Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims”

0 replies, 8 likes

JB: New paper I was a part of, "Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims”, is out today. Major props to @Miles_Brundage, @GretchenMarina and anyone I missed who helped facilitate this cross-institution collaboration

0 replies, 8 likes

esteve almirall: A group of AI scientists around Open AI suggest that a council of citizens is a good mechanism to regulate AI @kpeiro @ulisescortes @KarinagibertK @ramonsang

0 replies, 6 likes

Giorgio Patrini 🛡️👾: AI deployed systems do not live the ideal and isolated world defined in (most of) the papers-- it's time for AI incidents reporting, safety bounties and red teaming, as they are done in more mature tech areas. Cool work:

1 replies, 5 likes

Tom Gilbert: Parallel to my recent work with @roeldobbe , excited to share a piece I co-wrote on the need for verification mechanisms to address large-scale AI impacts:

1 replies, 4 likes

roadrunner01: Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims pdf: abs: project page:

0 replies, 4 likes

Scott H. Hawley: Ideas for incentivizing compliance with AI Ethics principles. Paper:

0 replies, 3 likes

Faizy: Toward Trustworthy AI: [abs] [pdf] #MachineLearning #DeepLearning #AI #ML #Bigdata #Analytics #ArtificialIntelligence #DataScience #NeuralNetworks

0 replies, 3 likes

Noa Zilberman: How often do you wonder if you can trust #AI? In this report, we make concrete recommendations toward #trustworthyAI

1 replies, 3 likes

Larissa Schiavo: Ensuring that AI systems can be trusted - by the public, by other institutions, by AI developers - is a huge struggle. This report adds pragmatic and nuanced advice to the discussion. I’m proud to have provided input on early versions.

0 replies, 3 likes

Cambridge Trust & Technology Initiative: @TowardstrustworthyAI: where good intentions are not quite good enough. 58 researchers co-author groundbreaking report with 10 step-plan for AI development that points the way to @responsibleAI development in a transparent and verifiable manner.

0 replies, 2 likes

Prashant Kumar Rai: An awesome report on fair and responsible #AI. #AIEthics

0 replies, 2 likes

AI for People: Some amazing work towards trustworthy AI Development! Finally a report which comprehensively focuses on the level of institutions, software and hardware. #trustworthy #ArtificialIntelligence

0 replies, 2 likes

Eline Chivot: AI researchers from Google Brain, Intel & others propose ‘bias bounties’ to put ethics principles into practice Embedding #ethics by #design - 10 recommended mechanisms - Paper here:

0 replies, 2 likes

Juan Mateos Garcia: Many AI developers and adopters are talking the ethics talk - but how can we know if they are walking the walk? The recent collaborative report about Trustworthy AI proposes institutional, software and hardware mechanisms to verify their claims.

1 replies, 0 likes


Found on Apr 16 2020 at

PDF content of a computer science paper: Toward Trustworthy AI Development: Mechanisms for Supporting Verifiable Claims