Papers of the day   All papers

Model Cards for Model Reporting

Comments

Jeff Dean: Great to see more transparent and interpretable ML models & tools being made available via @GCPcloud. Particularly excited that Model Cards, which started as a @GoogleAI research project by @mmitchell_ai et al. are part of this (https://arxiv.org/abs/1810.03993) https://cloud.google.com/blog/products/ai-machine-learning/google-cloud-ai-explanations-to-increase-fairness-responsibility-and-trust

4 replies, 512 likes


hardmaru: ☝️ This is how models trained on Celeb-A might perceive the world. If you’re looking to learn more about tackling ML bias, I recommend checking out the work on “Model Cards” by @mmitchell_ai et al. https://arxiv.org/abs/1810.03993 https://web.stanford.edu/class/archive/cs/cs224n/cs224n.1194/slides/cs224n-2019-lecture19-bias.pdf https://t.co/rCmezP8lMn

9 replies, 412 likes


Thomas Wolf: Current status: adding model cards for the 63 (!) pretrained models in 🤗 Transformers library 👉 more transparent reporting on models' capabilities, limitations, uses... Read https://arxiv.org/abs/1810.03993 (@mmitchell_ai @timnitGebru et al.) for details. Example (from this paper)👇 https://t.co/qnRkb3WGa7

4 replies, 290 likes


hardmaru: I’m by no means an expert on ML bias+fairness and I think it’s one of the most challenging research areas. But whenever I can, I try to listen to what people working in this area are saying, and try to understand some of the nuances to expand my worldview.https://twitter.com/hardmaru/status/1274522862120919040?s=21

3 replies, 77 likes


Grady Booch: @AnimaAnandkumar @reddit @alexisohanian @OpenAI @timnitGebru @mmitchell_ai Fantastic. Thank you for the rapid response. A link for those who want to learn more about model cards: https://arxiv.org/pdf/1810.03993.pdf

1 replies, 51 likes


DPManchee: @jovialjoy @StanfordHAI How to identify bias in models / datasets ? Adopt ideas from Mitchell et al (2018), "Model Cards For Model Reporting", https://arxiv.org/abs/1810.03993 ? Discussed more here at a @stanfordnlp lecture, https://www.youtube.com/watch?v=XR8YSRcuVLE https://t.co/5Mftvxrd3o

0 replies, 35 likes


Rachel Thomas: Some practical tools you can use towards tech ethics: https://www.scu.edu/ethics-in-technology-practice/ethical-toolkit/ @ShannonVallor https://arxiv.org/abs/1810.03993 @mmitchell_ai https://www.ics.uci.edu/~djp3/classes/2011_01_INF134/papers/impl9-rev.pdf @EricPSB https://techpolicylab.uw.edu/wp-content/uploads/2019/03/TowardInclusiveTechPolicyDesign.pdf @megyoung0

1 replies, 30 likes


HotComputerScience: Most popular computer science paper of the day: "Model Cards for Model Reporting" https://hotcomputerscience.com/paper/model-cards-for-model-reporting https://twitter.com/JeffDean/status/1197645913709256704

1 replies, 28 likes


Arvind Narayanan: Oops, wrong link to "Model Cards for Model Reporting" earlier in the thread (link goes to "Datasheets for Datasets" instead). Correct link: https://arxiv.org/pdf/1810.03993.pdf I meant to say that Datasheets for Datasets is *also* a great paper on how to do ML in a more responsible way :)

1 replies, 25 likes


Dr. Angelica Lim: Took some time in my Machine Learning lecture today to talk about "Datasheets for Datasets" and "Model Cards for Model Reporting" by @timnitGebru Are you teaching your students this? https://arxiv.org/abs/1810.03993 https://arxiv.org/abs/1803.09010 @officialAAAC https://t.co/LXfeFood1R

1 replies, 18 likes


Kai-Wei Chang: @AnimaAnandkumar @VioletNPeng @OpenAI This reminds me of "Model Cards for Model Reporting" https://arxiv.org/abs/1810.03993 by @timnitGebru @mmitchell_ai etal -- "released models should be accompanied by documentation detailing their performance characteristics"

1 replies, 17 likes


Tracy Pizzo Frey: *Tracy stay professional* *Tracy you are an adult* *Tracy take a deep bre--* OMG I AM SO EXCITED I CAN'T HOLD IT IN THIS JUST HAPPENED

1 replies, 14 likes


Vincent D. Warmerdam: Never heard of this paper before but it's got a cute idea. Just like you've got an ISO standard spec sheet when you buy a television screen, maybe you can also make those for machine learning models. Model Cards for Machine Learning. https://arxiv.org/pdf/1810.03993.pdf

1 replies, 9 likes


Marcus East: One of the barriers to the adoption of AI has been the opaqueness of its inner workings — this announcement will help to improve transparency and interpretability, critical in decisions connected with financial services, law enforcement and medicine. #google #googlecloud #ai #ml

0 replies, 7 likes


Hugo Bowne-Anderson: I’m pretty excited about this, particularly the incorporation of Model Cards for Model Reporting, which is one of my fave papers of the ~past year (thanks @mmitchell_ai @timnitGebru et al; & thanks to @mikeloukides for introducing me to the paper).

0 replies, 7 likes


Larissa Schiavo: Idea (for the graphic design gifted): model cards for trained machine learning models (see paper)... ... made into physical cards, designed like baseball cards. AI people give them to other AI people and trade them. https://arxiv.org/pdf/1810.03993.pdf

2 replies, 7 likes


Joseph Futoma: Great to see Model Cards (https://arxiv.org/abs/1810.03993) picking up steam at Google! https://modelcards.withgoogle.com/about h/t @jackclarkSF from his always-informative ImportAI newsletter! Looking fwd to when this is standard & reqd across field, like @TRIPODStatement (@GSCollins have u seen MCs?)

0 replies, 6 likes


Bugra Akyildiz: Very nice paper regarding responsible usage of machine learning models: https://arxiv.org/pdf/1810.03993.pdf I like the structure of model cards and how it is divided into different parts through concerns/use cases.

0 replies, 6 likes


Amy Tabb @ 🏠: @colinraffel @timnitGebru And "Model Cards for Model Reporting" paper afterwards ...! Margaret Mitchell, Simone Wu, Andrew Zaldivar, Parker Barnes, Lucy Vasserman, BenHutchinson, Elena Spitzer, Inioluwa Deborah Raji, Timnit Gebru abs:https://arxiv.org/abs/1810.03993 pdf: https://arxiv.org/pdf/1810.03993.pdf

1 replies, 5 likes


Data Nutrition Project: .@mmitchell_ai @DrLucyWasserman @timnitGebru et al. advocated for a framework of transparent model reporting to accompany trained ML models as a step towards responsible AI works in their paper https://arxiv.org/pdf/1810.03993.pdf. We're glad to be referenced in your work!💫

0 replies, 4 likes


andrea panizza: Learnt today about Datasheets for Datasets by @timnitGebru et al. https://arxiv.org/abs/1803.09010 and Model Cards for Model Performance by @mmitchell_ai et al. https://arxiv.org/abs/1810.03993 both sound like very good ideas. Are they used in practice?

0 replies, 4 likes


Carol Smith: Also use the "Datasheets for Datasets" markdown template to support your #DataScience work and the people who use the data - thanks @timnitGebru and @_josh_meyer_ https://github.com/JRMeyer/markdown-datasheet-for-datasets/blob/master/DATASHEET.md #ethics #AIEthics #AI #ML h/t @lh3com

0 replies, 4 likes


mauro_iker: Model Cards: Explainable model reporting to document any trained #MachineLearning model #XAI #ExplainableAI #ArtificialIntelligence #AI #Standards #DeepLearning #DL #NeuralNetworks #NN #ComputerVision #NLP

0 replies, 1 likes


Mia Dand: @lizjosullivan A1. Here is a great example of Model Cards for Model Reporting by @timnitgebru Margaret Mitchell and others https://arxiv.org/abs/1810.03993 #AIEthics #TwitterChat #Transparency #Explainability

0 replies, 1 likes


Marcel Fröhlich: Besides data transparency, #GDPR also requires algorithmic transparency for automated decisions. Most of that is a metadata exercise. The missing piece is usually the actual trained model, in case of #ML #algorithms. Here is a way to tackle that, too: https://arxiv.org/abs/1810.03993 https://t.co/JqQiq18lb2

0 replies, 1 likes


Max 🤪: @justinnhli If haven't seen it already, Model Cards (https://arxiv.org/abs/1810.03993) cover many fundamental questions you should be asking in regards to releasing your algorithm

1 replies, 0 likes


Content

Found on Nov 21 2019 at https://arxiv.org/pdf/1810.03993.pdf

PDF content of a computer science paper: Model Cards for Model Reporting