Publications
On the Societal Impact of Open Foundation Models (manuscript with Sayash Kapoor, Rishi Bommasani, Kevin Klyman, Shayne Longpre, Ashwin Ramaswami, Peter Cihon, Aspen Hopkins, Kevin Bankston, Stella Biderman, Rumman Chowdhury, Alex Engler, Peter Henderson, Yacine Jernite, Seth Lazar, Stefano Maffulli, Alondra Nelson, Joelle Pineau, Aviya Skowron, Dawn Song, Victor Storchan, Daniel Zhang, Daniel E. Ho, Percy Liang, and Arvind Narayanan)
Towards Fairness in Personalized Ads Using Impression Variance Aware Reinforcement Learning, KDD 2023 (with Aditya Srinivas Timmaraju, Mehdi Mashayekhi, Mingliang Chen, Qi Zeng, Quintin Fettes, Wesley Cheung, Yihan Xiao, Manojkumar Rangasamy Kannadasan, Pushkar Tripathi, Sean Gahagan, and Rob Roudani)
Toward Fairness in Personalized Ads, Meta (with Pushkar Tripathi, Aditya Srinivas Timmaraju, Mehdi Mashayekhi, Qi Zeng, Rabyd (Rob) Roudani, Sean Gahagan, Andrew Howard, and Isabella Leone)
Casual Conversations v2: Designing a large consent-driven dataset to measure algorithmic bias and robustness (manuscript with Caner Hazirbas, Yejin Bang, Tiezheng Yu, Parisa Assar, Bilal Porgali, Vítor Albiero, Stefan Hermanek, Jacqueline Pan, Emily McReynolds, Pascale Fung, and Cristian Canton Ferrer)
Adaptive Sampling Strategies to Construct Equitable Training Datasets, Proceedings of the 2022 ACM Conference on Fairness, Accountability, and Transparency '22 (with William Cai, Ro Encarnacion, Bobbie Chern, Sam Corbett-Davies, Stevie Bergman, and Sharad Goel)
How Meta is working to assess fairness in relation to race in the U.S. across its products and systems, Meta (with Rachad Alao, Jingang Miao, Ilya Mironov, and Jonathan Tannen)
Fairness on the ground: Applying algorithmic fairness approaches to complex production systems (manuscript with Renata Barreto, Bobbie Chern, Sam Corbett-Davies, Melissa Hall, Michelle Lam, Joaquin Quinonero Candela, Manish Ragavan, Joshua Simons, Jonathan Tannen, Edmund Tong, Kate Vredenburgh, and Jiejing Zhao)
Awareness in practice: tensions in access to sensitive attribute data for antidiscrimination, Proceedings of the 2020 ACM Conference on Fairness, Accountability, and Transparency (with Aaron Rieke and Shazeda Ahmed)
Discrimination through optimization: How Facebook's ad delivery can lead to skewed outcomes, Proceedings of the 2019 ACM Conference on Human-Computer Interaction; Diversity & Inclusion Award, Honorable Mention Award (with Muhammad Ali, Piotr Sapiezynski, Aleksandra Korolova, Alan Mislove, and Aaron Rieke)
Help Wanted: Hiring Algorithms, Equity and Bias, Upturn (with Aaron Rieke)
Leveling the Platform: Real Transparency for Paid Messages on Facebook, Upturn (with Aaron Rieke)
Public Scrutiny of Automated Decisions: Early Lessons and Emerging Methods, An Upturn and Omidyar Network Report (with Aaron Rieke and David G. Robinson)
The Illusion of Accuracy: How Body-Worn Camera Footage Can Distort Evidence, Upturn and The Leadership Conference for Civil and Human Rights
Police Body Worn Cameras: A Policy Scorecard, Upturn and The Leadership Conference for Civil and Human Rights
Automation and the Quantified Society, Upturn and Netgain Partnership (with David G. Robinson)
Data Ethics: Investing Wisely in Data at Scale, Upturn (with David G. Robinson)
Articles & Media
Ensuring NIST’s AI Safety Institute Consortium Lives Up to its Potential, CDT Blog, March 6, 2024
Trustworthy AI needs trustworthy measurement, CDT Blog, March 6, 2024
「」Miranda Bogen is creating solutions to help govern AI, TechCrunch, February 24, 2024
「」Artificial Intelligence, Last Week Tonight with John Oliver (HBO), February 27, 2023 (at 20:28)
「」Meta Agrees to Alter Ad Technology in Settlement With U.S. New York Times, January 21, 2023
「」Facebook Starts Effort to Boost Equity in Housing Ads, Wall Street Journal, January 9, 2023
A new system to help ensure ads are delivered fairly to different demographic groups, Meta AI, January 9, 2023
「」Facebook’s Ad System Might Be Hard-Coded for Discrimination, Wired, April 6, 2020
「」Facebook’s ad system seems to discriminate by race and gender, The Economist, April 4, 2020
「」Facebook's ads system leans on stereotypes for housing, job ads: study, Reuters, April 3, 2020
All the Ways Hiring Algorithms Can Introduce Bias, Harvard Business Review, May 6, 2019
Did Instagram Really Revoke the Verification of Venezuelan Dictator Nicolás Maduro?, Slate, January 25, 2019
「」Artificial Intelligence Was Supposed to Reduce Hiring Discrimination. It’s Already Backfiring, Mother Jones, January 3, 2019
Transformers: Artificial Intelligence, The Ethical Pitfalls in AI, Washington Post Live, January 2019
「」Will Facebook's New Ad-Transparency Tools Protect Democracy?, The Atlantic, May 10, 2018
“The Past is Never Dead: Why Google Shows Different Maps to Different Countries”, On the Media, WNYC (NPR), March 2018
What happens to body cam footage after fatal police shootings?, Equal Future, March 1, 2018
Body cameras are only as effective as the policies that govern them, St. Louis Post Dispatch, October 12, 2017
When drawing a line is hard: Connecting the dots between math, technology, and law in the challenges of gerrymandering, Equal Future, September 2017
Good cop cameras, bad rules, NY Daily News, April 2017
Artificial intelligence will force us to confront our values, Equal Future, September 2016
How Google Has Managed the Tricky Balancing Act of Geopolitics, Newsweek, April 2016
Drones Without Borders, This Week in Law, January 8, 2016
Algorithms of War, Slate, December 8, 2015
Can Digital Privacy Be Protected By Borders?, Fletcher Forum of World Affairs, October 2015
「」: work featured in
Selected Speaking Engagements
The AI Governance Puzzle: Assembling the Pieces of Policy and Trust
State of the Net | February 2024
This panel delved into the current state of AI governance, assessing the practicality of various governance approaches that can be implemented in the near term versus those contingent on the maturation of AI standards. With Oma Seddiq, Travis Hall. Adam Thierer, and Evangelos Razis
Closing the Gap: Navigating Complexities and Contradictions In the Mitigation of Algorithmic Bias
CITP | November 2023
This talk explored key tensions and open questions that emerge when attempting to mitigate algorithmic bias, exploring the complex interplay between legal, technical, and policy realities that will need to be resolved in order to effectively tackle algorithmic bias.
US AI Safety Institute Workshop: Collaboration to Enable Safe and Trustworthy AI
NIST | November 2023
NIST hosted a workshop on November 17, 2023, to engage in a conversation about artificial intelligence (AI) safety. With Elham Tabassi, Ali Farhadi, and Ramayya Krishnan.
AI safety beyond the UK Summit
Brookings Institution | November 2023
The Brookings Institution hosted Secretary Michelle Donelan, the U.K.’s secretary of state for science, innovation and technology, for a discussion on the U.K.’s efforts on AI and online safety, followed by an expert panel who assessed progress made at the AI Safety Summit and how it fits in with broader trends toward cooperation in the global policy landscape. With Chinasa T. Okolo and Valerie Wirtschafter.
AI Safety Policy: Advancing and Operationalizing Solutions
Partnership on AI Policy Forum | November 2023
From the G7 to the White House to 10 Downing Street, leaders from around the world are making AI safety a priority–but do all parties have the same understanding of the term? This session dove into what policymakers mean when they discuss “AI safety” and whether their definition and priorities align with industry, civil society, and academia. With Lisa Einstein, Sophia Ignatidou, Lucy Poole, and Stephanie Ifayemi.
Steering AI for the Public Good: A Dialogue for the Future
Institute for Advanced Study | June 2023
A panel of computer scientists, social scientists, technology developers, and policymakers discussing how the research community, industry, government, civil society, and the public can work collaboratively on strategies for the development of AI systems that embed a diverse and representative range of humanity’s priorities and advance democratic values. With Alondra Nelson, Brian Christian, Sorelle Friedler, and William Isaac.
Centering Civil Rights in the Privacy Debate
New America Foundation | May 2019
New America’s Open Technology Institute and Color of Change held a discussion about the importance of centering the privacy debate around harms marginalized communities experience, and how to address them. With Gaurav Laroia, Erin Shields, Alisa Valentin, and Brandi Collins-Dexter.
What Sci-Fi Futures Can (and Can't) Teach Us About AI Policy
New America Foundation | May 2019
Future Tense and New America’s Open Technology Institute held an afternoon of discussion on sci-fi and AI with policy and tech experts, futurists, and science fiction authors. With Kevin Bankston, Rumman Chowdhury, Elana Zeide, and Lindsey Sheppard.
International Perspectives On How Technology Impacts Democracy
Canadian International Council | April 2019
A discussion on how digital technologies have transformed the politics of every nation in the world and how the future of democracy will depend on those who can grasp the opportunities presented by technology while protecting citizens from new threats. With Minister Karina Gould, Nobel Peace Prize nominee Wael Ghonim, and Anatoliy Gruzd.
The Ethical Pitfalls in AI: Bias and the Rise of Machines
Washington Post Live | January 2019
The Washington Post brought together researchers, innovators and experts for a discussion of the dangers of creating intelligent systems and machines fueled by data that reflects bias, from hiring practices to criminal sentencing. With NAACP President and Director-Counsel Sherrilyn Ifill and Head of Diversity Analytics at Pymetrics, Kelly Trindel.
Why Google Shows Different Maps to Different Countries
On the Media, WNYC (NPR) | March 2018
How Google has come to both threaten and facilitate diplomatic relations, while fraying our sense of a shared geographical reality.