Publications

Find at Harvard

2019
Article

An Empirical Study of Rich Subgroup Fairness for Machine Learning

By: Michael J Kearns, Seth Neel, Aaron Leon Roth and Zhiwei Steven Wu

Kearns et al. [2018] recently proposed a notion of rich subgroup fairness intended to bridge the gap between statistical and individual notions of fairness. Rich subgroup fairness picks a statistical fairness constraint (say, equalizing false positive rates across... View Details

Keywords: Machine Learning; Fairness; AI and Machine Learning

At the heart of any innovation process lies a fundamental practice: the way people create ideas and solve problems. This “decision making” side of innovation is what scholars and practitioners refer to as “design.” Decisions in innovation processes have so far been... View Details

2023
Article

On the Impact of Actionable Explanations on Social Segregation

By: Ruijiang Gao and Himabindu Lakkaraju

As predictive models seep into several real-world applications, it has become critical to ensure that individuals who are negatively impacted by the outcomes of these models are provided with a means for recourse. To this end, there has been a growing body of research... View Details

Keywords: Forecasting and Prediction; AI and Machine Learning; Outcome or Result

2023
Article

Towards Bridging the Gaps between the Right to Explanation and the Right to Be Forgotten

By: Himabindu Lakkaraju, Satyapriya Krishna and Jiaqi Ma

The Right to Explanation and the Right to be Forgotten are two important principles outlined to regulate algorithmic decision making and data usage in real-world applications. While the right to explanation allows individuals to request an actionable explanation for an... View Details

Keywords: Analytics and Data Science; AI and Machine Learning; Decision Making; Governing Rules, Regulations, and Reforms

2025
Article

Statistical Inference for Heterogeneous Treatment Effects Discovered by Generic Machine Learning in Randomized Experiments

By: Kosuke Imai and Michael Lingzhi Li

Researchers are increasingly turning to machine learning (ML) algorithms to investigate causal heterogeneity in randomized experiments. Despite their promise, ML algorithms may fail to accurately ascertain heterogeneous treatment effects under practical settings with... View Details

Keywords: AI and Machine Learning; Mathematical Methods; Analytics and Data Science

Find at Harvard

Forthcoming
Article

Human-Algorithm Collaboration with Private Information: Naïve Advice Weighting Behavior and Mitigation

By: Maya Balakrishnan, Kris Ferreira and Jordan Tong

Even if algorithms make better predictions than humans on average, humans may sometimes have private information which an algorithm does not have access to that can improve performance. How can we help humans effectively use and adjust recommendations made by... View Details

Keywords: AI and Machine Learning; Analytics and Data Science; Forecasting and Prediction; Digital Marketing

Article

Adaptive Machine Unlearning

By: Varun Gupta, Christopher Jung, Seth Neel, Aaron Roth, Saeed Sharifi-Malvajerdi and Chris Waites

Data deletion algorithms aim to remove the influence of deleted data points from trained models at a cheaper computational cost than fully retraining those models. However, for sequences of deletions, most prior work in the non-convex setting gives valid guarantees... View Details

Keywords: Machine Learning; AI and Machine Learning

May 2022 (Revised June 2024)
Case

LOOP: Driving Change in Auto Insurance Pricing

By: Elie Ofek and Alicia Dadlani

John Henry and Carey Anne Nadeau, co-founders and co-CEOs of LOOP, an insurtech startup based in Austin, Texas, were on a mission to modernize the archaic $250 billion automobile insurance market. They sought to create equitably priced insurance by eliminating pricing... View Details

Keywords: AI and Machine Learning; Technological Innovation; Equality and Inequality; Prejudice and Bias; Growth and Development Strategy; Customer Relationship Management; Price; Insurance Industry; Financial Services Industry

Educators

03 Apr 2019
Book

Fintech's Game-Changing Opportunities for Small Business

services—lending in particular—in full force. This trend will be transformative. With the ability to aggregate and organize data and analyze it rigorously, lenders can have more predictive algorithms about who is creditworthy, and small... View Details

Keywords: by Martha Lagace; Banking; Financial Services

January 2021
Article

Machine Learning for Pattern Discovery in Management Research

By: Prithwiraj Choudhury, Ryan Allen and Michael G. Endres

Supervised machine learning (ML) methods are a powerful toolkit for discovering robust patterns in quantitative data. The patterns identified by ML could be used for exploratory inductive or abductive research, or for post-hoc analysis of regression results to detect... View Details

Keywords: Machine Learning; Supervised Machine Learning; Induction; Abduction; Exploratory Data Analysis; Pattern Discovery; Decision Trees; Random Forests; Neural Networks; ROC Curve; Confusion Matrix; Partial Dependence Plots; AI and Machine Learning

Find at Harvard

February 26, 2024
Article

Making Workplaces Safer Through Machine Learning

By: Matthew S. Johnson, David I. Levine and Michael W. Toffel

Machine learning algorithms can dramatically improve regulatory effectiveness. This short article describes the authors' scholarly work that shows how the U.S. Occupational Safety and Health Administration (OSHA) could have reduced nearly twice as many occupational... View Details

Keywords: Government Experimentation; Auditing; Inspection; Evaluation; Process Improvement; Government Administration; AI and Machine Learning; Safety; Governing Rules, Regulations, and Reforms

2023
Working Paper

In-Context Unlearning: Language Models as Few Shot Unlearners

By: Martin Pawelczyk, Seth Neel and Himabindu Lakkaraju

Machine unlearning, the study of efficiently removing the impact of specific training points on the trained model, has garnered increased attention of late, driven by the need to comply with privacy regulations like the Right to be Forgotten. Although unlearning is... View Details

Keywords: AI and Machine Learning; Copyright; Information

2024
Working Paper

The Cram Method for Efficient Simultaneous Learning and Evaluation

By: Zeyang Jia, Kosuke Imai and Michael Lingzhi Li

We introduce the "cram" method, a general and efficient approach to simultaneous learning and evaluation using a generic machine learning (ML) algorithm. In a single pass of batched data, the proposed method repeatedly trains an ML algorithm and tests its empirical... View Details

Keywords: AI and Machine Learning

February 2024
Teaching Note

Data-Driven Denim: Financial Forecasting at Levi Strauss

By: Mark Egan

Teaching Note for HBS Case No. 224-029. Levi Strauss & Co. (“Levi Strauss”) partnered with the IT services company Wipro to incorporate more sophisticated methods, such as machine learning, into their financial forecasting process starting in 2018. The decision to... View Details

Keywords: Forecasting; Regression; Machine Learning; Artificial Intelligence; Apparel; Corporate Finance; Forecasting and Prediction; AI and Machine Learning; Apparel and Accessories Industry; United States

Research Summary

Overview

By: Ayelet Israeli

Professor Israeli utilizes econometric methods and field experiments to study data driven decision making in marketing context. Her research focuses on data-driven marketing, with an emphasis on how businesses can leverage their own data, customer data, and market data... View Details

Keywords: Channel Management; Pricing; Pricing Policies; Online Marketing; E-commerce; Analytics; Econometrics; Field Experiments; Data Analytics; Artificial Intelligence; Value Of Data

July 2023
Case

DayTwo: Going to Market with Gut Microbiome (Abridged)

By: Ayelet Israeli

DayTwo is a young Israeli startup that applies research on the gut microbiome and machine learning algorithms to deliver personalized nutritional recommendations to its users in order to minimize blood sugar spikes after meals. After a first year of trial rollout in... View Details

Keywords: Business Startups; AI and Machine Learning; Nutrition; Market Entry and Exit; Product Marketing; Distribution Channels

Educators

2023
Working Paper

Insufficiently Justified Disparate Impact: A New Criterion for Subgroup Fairness

By: Neil Menghani, Edward McFowland III and Daniel B. Neill

In this paper, we develop a new criterion, "insufficiently justified disparate impact" (IJDI), for assessing whether recommendations (binarized predictions) made by an algorithmic decision support tool are fair. Our novel, utility-based IJDI criterion evaluates false... View Details

Keywords: AI and Machine Learning; Forecasting and Prediction; Prejudice and Bias

July 2025
Article

Digital Lending and Financial Well-Being: Through the Lens of Mobile Phone Data

By: AJ Chen, Omri Even-Tov, Jung Koo Kang and Regina Wittenberg-Moerman

To mitigate information asymmetry about borrowers in developing economies, digital lenders use machine-learning algorithms and nontraditional data from borrowers’ mobile devices. Consequently, digital lenders have managed to expand access to credit for millions of... View Details

Keywords: Informal Economy; Digital Banking; Mobile Phones; Developing Countries and Economies; Mobile and Wireless Technology; AI and Machine Learning; Analytics and Data Science; Credit; Borrowing and Debt; Well-being; Banking Industry; Kenya