Papers of the day   All papers

Energy and Policy Considerations for Deep Learning in NLP

Comments

May 17 2019 Emma Strubell

Are you interested in deep learning for NLP but also concerned about the CO2 footprint of training? You should be! Excited to share our work "Energy and Policy Considerations for Deep Learning in NLP" at @ACL2019_Italy! With @ananya__g and @andrewmccallum. Preprint coming soon. https://t.co/kIgZWcptRR
77 replies, 2551 likes


Oct 03 2019 Safiya Umoja Noble PhD

Training a single AI model can emit as much carbon as five cars in their lifetimes - via @techreview https://www.technologyreview.com/s/613630/training-a-single-ai-model-can-emit-as-much-carbon-as-five-cars-in-their-lifetimes/?utm_campaign=site_visitor.unpaid.engagement&utm_source=twitter&utm_medium=social_share&utm_content=2019-10-03
19 replies, 697 likes


Jun 07 2019 MIT Technology Review

Training a single AI model can emit as much carbon as five cars in their lifetimes https://trib.al/LmEr6F7
1 replies, 96 likes


Jun 25 2019 Arthur Charpentier 🌻

"Training a single AI model can emit as much carbon as five cars in their lifetimes" https://www.technologyreview.com/s/613630/training-a-single-ai-model-can-emit-as-much-carbon-as-five-cars-in-their-lifetimes/?utm_source=twitter&utm_medium=tr_social&utm_campaign=site_visitor.unpaid.engagement (I knew it was bad.... but that bad !?!) "Deep learning has a terrible carbon footprint" see https://arxiv.org/abs/1906.02243
2 replies, 38 likes


Jun 25 2019 MIT Technology Review

Deep learning has a terrible carbon footprint. https://trib.al/7XL71bW
0 replies, 32 likes


Aug 07 2019 Shreya Shankar

https://arxiv.org/pdf/1906.02243.pdf https://t.co/sEyuYr55v2
2 replies, 29 likes


Jun 12 2019 Jose Javier Garde

Energy and Policy Considerations for Deep Learning in #NLP by Emma Strubell, Ananya Ganesh, Andrew McCallum https://arxiv.org/abs/1906.02243 #deeplearning #ArtificialIntelligence #machinelearning #energy #climatechange #climate #policy #NeuralNetworks #algorithms https://t.co/j3KNLSOr20
0 replies, 25 likes


Jun 07 2019 Emma Strubell

@ACL2019_Italy @ananya__g @andrewmccallum preprint now available! https://arxiv.org/abs/1906.02243
3 replies, 23 likes


Jul 31 2019 Alice Coucke

Yes! This should be mandatory. Very interesting talk by @strubell at #acl2019nlp 🌱 👉https://arxiv.org/abs/1906.02243 https://t.co/OYpuluuglu
1 replies, 15 likes


Jun 13 2019 Xander Steenbrugge

"Energy and Policy Considerations for Deep Learning in NLP" They provide some very interesting statistics on the environmental impact of training large Deep Learning models with today's various Cloud Providers! Paper: https://arxiv.org/abs/1906.02243 Article: https://www.technologyreview.com/s/613630/training-a-single-ai-model-can-emit-as-much-carbon-as-five-cars-in-their-lifetimes/ https://t.co/ZsY1ClMa4W
0 replies, 14 likes


Sep 18 2019 tante

(btw. here are CO2 estimates for training neural nets https://arxiv.org/pdf/1906.02243.pdf )
1 replies, 8 likes


Jun 09 2019 Julie Grollier

"It is estimated that we must cut carbon emissions by half over the next decade, and based on the estimated CO2 emissions listed in Table1, model training and development likely make up a substantial portion of the greenhouse gas emissions" https://arxiv.org/abs/1906.02243 https://t.co/NpNqadgPL6
0 replies, 8 likes


Jul 30 2019 Denis Newman-Griffis

Fantastic turnout for @strubell's paper on energy consumption in NLP research @ACL2019_Italy (this is only half the room) Paper at https://arxiv.org/abs/1906.02243 #ACL2019 https://t.co/oUNsvl0wio
1 replies, 8 likes


Jun 07 2019 arXiv CS-CL

Energy and Policy Considerations for Deep Learning in NLP http://arxiv.org/abs/1906.02243
0 replies, 7 likes


Jun 09 2019 Anish Mohammed

Energy and Policy Considerations for Deep Learning in NLP < wondering if this was by accident or design, yet another moat for incumbents against challengers #DeepLearning https://arxiv.org/abs/1906.02243
0 replies, 5 likes


Oct 01 2019 Jean Senellart

@lorenlugosch @SanhEstPasMoi Yes it does. I used the CO₂e calculation model from https://arxiv.org/pdf/1906.02243.pdf. This estimate is based on energy production in USA with 36% non-carbon energy. For China, where the training has maybe (?) been run by #ShannonAI, the figure would be a bit higher.
0 replies, 4 likes


Jul 19 2019 Paul Bradshaw

If you're a data journalist exploring #AI/#ML/#NLP, prepare to feel guilty... https://arxiv.org/abs/1906.02243
0 replies, 4 likes


Jun 07 2019 Russell Neches

626,155 pounds of CO2 to train one model? Ouch. Well, I guess now you know why huge tech companies are building custom chips for machine learning. https://arxiv.org/abs/1906.02243
0 replies, 3 likes


Jun 08 2019 arXiv CS-CL

Energy and Policy Considerations for Deep Learning in NLP http://arxiv.org/abs/1906.02243
0 replies, 3 likes


Aug 15 2019 Beril Sirmacek 🦋

From now on, the #artificialintelligence frameworks will not be judged by their #performance but by their #energy labels. #carbonfootprint #climatechange #co2 https://arxiv.org/abs/1906.02243 https://t.co/NYN5xqJnMq
1 replies, 2 likes


Jun 18 2019 Dave Costenaro

Interesting paper: "Energy and Policy Considerations for Deep Learning in NLP." Training 1 big NN model has the carbon footprint of 5 cars over their lifetimes. (https://arxiv.org/abs/1906.02243). Compute is cheap...but not free, so please give efficient code some thought!
0 replies, 2 likes


Jun 14 2019 Manyvip

A Deep Learning Process Can Emit 284.000 kilograms of Carbon Dioxide (CO2). Download PDF. https://arxiv.org/abs/1906.02243 https://t.co/OGOT6UXFB4
0 replies, 2 likes


Oct 06 2019 Erik Hamburger

Can #MachineLearning and #Sustainability go together? This study shows how energy intensive all this #ML and #AI is. https://arxiv.org/pdf/1906.02243.pdf
0 replies, 2 likes


Jun 14 2019 Dr William Marshall

Paper: Energy and Policy Considerations for Deep Learning in NLP (Natural Language Processing). Training an #AI model uses a lot of electrical power and leaves a vast carbon footprint. #climatechange https://arxiv.org/pdf/1906.02243.pdf
0 replies, 2 likes


Jul 24 2019 GetzlerChem

@Chemjobber See also the surprising and staggering cost of deep learning. (preprint, so caveat emptor, etc) https://arxiv.org/abs/1906.02243 https://t.co/ZxC7snogK9
0 replies, 1 likes


Jun 08 2019 Jordan Foley

Really interesting work on the potential environmental implications of machine learning and AI. Ive seen lots of important convos about the ethical dimensions of these technologies but few that center questions like these. https://arxiv.org/abs/1906.02243
0 replies, 1 likes


Aug 15 2019 Tim Heiler

The average deep learning model using fossil fuel releases around 78,000 pounds of carbon. That’s more than half of a car's output from assembly line to landfill. According to this paper: https://arxiv.org/abs/1906.02243 #AI #machinelearning #ClimateChange #ClimateEmergency https://t.co/27GQVlH1wK
0 replies, 1 likes


Jun 11 2019 DOSE Engineering

Data centers and cloud computing providers need to up their use of renewable energy in order to meet the high energy demands of CPU/GPU/TPU by artificial intelligence/deep learning. #greenenergy #cloudcomputing #datacenters #AI https://arxiv.org/pdf/1906.02243.pdf https://t.co/8SssmMf7Hl
0 replies, 1 likes


Jul 18 2019 Alasdair Allan

@swardley I do have some issues with the broader applicability of their analysis, but here's the link to the Strubell, Ganesh & McCallum (2019) paper with the life-cycle analysis of #MachineLearning training I talked about at the start of my #OSCON talk, https://arxiv.org/abs/1906.02243.
0 replies, 1 likes


Jun 07 2019 Ken Figueredo

#Green credentials and #MachineLearning https://arxiv.org/pdf/1906.02243.pdf https://t.co/3zJ3FxxZxi
0 replies, 1 likes


Jun 07 2019 Nate Jue

Colleague sent this article to me and it's making me think more and more about the environmental impacts of my computational work and associated ethical decisions. How many of us computational biologists even have this on our radar? I sure haven't.😑 https://arxiv.org/abs/1906.02243
0 replies, 1 likes


Jun 10 2019 Philippe Durance

Recent progress in training #neuralnetworks depends on the availability of exceptionally large computational resources that necessitate similarly substantial #energy consumption https://arxiv.org/abs/1906.02243?utm_campaign=the_algorithm.unpaid.engagement&utm_source=hs_email&utm_medium=email&utm_content=73464008&_hsenc=p2ANqtz-_Fe6QnHVjx60Rmn8sSlVsG30Q0TJFvIXv2ykzz8aVKdt_RV6sUronq35AtaM5iZ1YOF8qIQdICVUflzM_vHIREtVblgQ&_hsmi=73464008 @Cornell
0 replies, 1 likes


Jun 07 2019 Andrés Murcia

La huella de carbono del Deep Learning - "Runs on energy-intensive computer chips, can emit more than 626,000 pounds of carbon dioxide equivalent, nearly five times the lifetime emissions of the average American car." - https://arxiv.org/abs/1906.02243?utm_campaign=the_algorithm.unpaid.engagement&utm_source=hs_email&utm_medium=email&utm_content=73464008&_hsenc=p2ANqtz-_46saoiHzXONwwvcO8_1mRilORNzze1VMZK13OjGfGio6b6T1fa4hK60qYibywgomX5-w8tRl0vrOP0HnIcSWyXcS9wQ&_hsmi=73464009 https://t.co/mR3lVt6tNp
1 replies, 1 likes


Jun 16 2019 World Ethical Data Forum

Here's the UMass paper, if the numbers interest you: https://arxiv.org/pdf/1906.02243.pdf Thanks to @InfoMgmtExec for pointing out the original post wasn't clear enough.
0 replies, 1 likes


Jun 10 2019 Charles Starrett

We know computation like this, not to mention #blockchain (*shudder*), is worsening our #climate crisis. Why aren't we pushing harder for data centers to have their own solar/wind farms? — Energy and Policy Considerations for Deep Learning in NLP https://arxiv.org/abs/1906.02243
0 replies, 1 likes


Aug 22 2019 Libby Hemphill, PhD

@danieljkelley Just starting the read, but the environmental impact of our models is definitely something we talk about on my team. @strubell had a great paper at #ACL2019 about neural models and their costs: https://arxiv.org/abs/1906.02243
0 replies, 1 likes


Jul 30 2019 Swadhin | স্বাধীন

@krismicinski A recent related paper focusing on NLP model and energy : https://arxiv.org/abs/1906.02243 . Emma discusses about this in detail in TWIMLAI podcast this week.
0 replies, 1 likes


Jun 12 2019 Nathaniel Bullard

First: an AI model doesn't run on coal or gas; it runs on electricity, and its carbon will be a direct result of the power mix used to energize the data centers it runs on. The paper https://arxiv.org/pdf/1906.02243.pdf gets that right...
1 replies, 0 likes


Jul 05 2019 Zachary Lipton

@VishnuBoddeti So far, I have yet to be convinced of neural architecture search as a research direction but my degree of certainty is not high. To date, NAS requires 1000s× more resources w/o qualitatively stronger results. See paper by @strubell on environmental impact—https://arxiv.org/abs/1906.02243
1 replies, 0 likes


Jun 28 2019 Lancelot PECQUET

#environment - Training a single #AI model can emit as much #carbon as five cars in their lifetimes https://arxiv.org/pdf/1906.02243.pdf https://t.co/64HGDETeML
1 replies, 0 likes


Nov 04 2019 Emily Hopkins

@emiliagogu @IEEEorg 5. Diversity, non-discrimination, fairness accessibility, bias, competing interests & objectives #ismir2019 6. Societal and environmental well-being sustainability and benefit to future generations energy use of deep learning: https://arxiv.org/abs/1906.02243
1 replies, 0 likes


Content