Publications

Mar 2021
Conference Presentation

Descent-to-Delete: Gradient-Based Methods for Machine Unlearning

By: Seth Neel, Aaron Leon Roth and Saeed Sharifi-Malvajerdi

We study the data deletion problem for convex models. By leveraging techniques from convex optimization and reservoir sampling, we give the first data deletion algorithms that are able to handle an arbitrarily long sequence of adversarial updates while promising both... View Details

Keywords: Machine Learning; Unlearning Algorithm; Mathematical Methods

08 Oct 2018
Working Paper Summaries

Developing Theory Using Machine Learning Methods

Keywords: by Prithwiraj Choudhury, Ryan Allen, and Michael G. Endres

October 2021
Article

Overcoming the Cold Start Problem of CRM Using a Probabilistic Machine Learning Approach

By: Nicolas Padilla and Eva Ascarza

The success of Customer Relationship Management (CRM) programs ultimately depends on the firm's ability to understand consumers' preferences and precisely capture how these preferences may differ across customers. Only by understanding customer heterogeneity, firms can... View Details

Keywords: Customer Management; Targeting; Deep Exponential Families; Probabilistic Machine Learning; Cold Start Problem; Customer Relationship Management; Programs; Consumer Behavior; Analysis

Find at Harvard

Article

Learning Models for Actionable Recourse

By: Alexis Ross, Himabindu Lakkaraju and Osbert Bastani

As machine learning models are increasingly deployed in high-stakes domains such as legal and financial decision-making, there has been growing interest in post-hoc methods for generating counterfactual explanations. Such explanations provide individuals adversely... View Details

Keywords: Machine Learning Models; Recourse; Algorithm; Mathematical Methods

February 26, 2024
Article

Making Workplaces Safer Through Machine Learning

By: Matthew S. Johnson, David I. Levine and Michael W. Toffel

Machine learning algorithms can dramatically improve regulatory effectiveness. This short article describes the authors' scholarly work that shows how the U.S. Occupational Safety and Health Administration (OSHA) could have reduced nearly twice as many occupational... View Details

Keywords: Government Experimentation; Auditing; Inspection; Evaluation; Process Improvement; Government Administration; AI and Machine Learning; Safety; Governing Rules, Regulations, and Reforms

02 Aug 2017
Working Paper Summaries

Machine Learning Methods for Strategy Research

Keywords: by Mike Horia Teodorescu

February 2021
Tutorial

Assessing Prediction Accuracy of Machine Learning Models

By: Michael Toffel and Natalie Epstein

This video describes how to assess the accuracy of machine learning prediction models, primarily in the context of machine learning models that predict binary outcomes, such as logistic regression, random forest, or nearest neighbor models. After introducing and... View Details

Keywords: Statistics; Experiments; Forecasting and Prediction; Performance Evaluation; AI and Machine Learning

2020
Working Paper

Overcoming the Cold Start Problem of CRM Using a Probabilistic Machine Learning Approach

By: Eva Ascarza

The success of Customer Relationship Management (CRM) programs ultimately depends on the firm's ability to understand consumers' preferences and precisely capture how these preferences may differ across customers. Only by understanding customer heterogeneity, firms can... View Details

Keywords: Customer Management; Targeting; Deep Exponential Families; Probabilistic Machine Learning; Cold Start Problem; Customer Relationship Management; Customer Value and Value Chain; Consumer Behavior; Analytics and Data Science; Mathematical Methods; Retail Industry

SSRN

Research Summary

Making Machine Learning Robust to Adversarial Attacks

By: Himabindu Lakkaraju

The goal of this research is to ensure that machine learning models that we build and deploy are not easily susceptible to attacks by adversarial or malicious entities. View Details

December 2023
Article

Self-Orienting in Human and Machine Learning

By: Julian De Freitas, Ahmet Uğuralp, Zeliha Uğuralp, Laurie Paul, Joshua B. Tenenbaum and T. Ullman

A current proposal for a computational notion of self is a representation of one’s body in a specific time and place, which includes the recognition of that representation as the agent. This turns self-representation into a process of self-orientation, a challenging... View Details

Keywords: AI and Machine Learning; Behavior; Learning

Article

Preventing Fairness Gerrymandering: Auditing and Learning for Subgroup Fairness

By: Michael J Kearns, Seth Neel, Aaron Leon Roth and Zhiwei Steven Wu

The most prevalent notions of fairness in machine learning are statistical definitions: they fix a small collection of pre-defined groups, and then ask for parity of some statistic of the classifier (like classification rate or false positive rate) across these groups.... View Details

Keywords: Machine Learning; Algorithms; Fairness; Mathematical Methods

Teaching Interest

Interpretability and Explainability in Machine Learning

By: Himabindu Lakkaraju

As machine learning models are increasingly being employed to aid decision makers in high-stakes settings such as healthcare and criminal justice, it is important to ensure that the decision makers correctly understand and consequent trust the functionality of these... View Details

25 Oct 2017
Research & Ideas

Will Machine Learning Make You a Better Manager?

Credit: PhonlamaiPhoto Thirty years ago, the idea of a machine learning on its own would have stoked the worst kind of sci-fi nightmares about robots taking over the planet. These days, View Details

Keywords: by Michael Blanding; Information Technology

November 2023
Case

Open Source Machine Learning at Google

By: Shane Greenstein, Martin Wattenberg, Fernanda B. Viégas, Daniel Yue and James Barnett

Set in early 2023, the case exposes students to the challenges of managing open source software at Google. The case focuses on the challenges for Alex Spinelli, Vice President of Product Management for Core Machine Learning. He must set priorities for Google’s efforts... View Details

Keywords: Decision Choices and Conditions; Technological Innovation; Open Source Distribution; Strategy; AI and Machine Learning; Applications and Software; Technology Industry; United States

October 2022 (Revised December 2022)
Case

SMART: AI and Machine Learning for Wildlife Conservation

By: Brian Trelstad and Bonnie Yining Cao

Spatial Monitoring and Reporting Tool (SMART), a set of software and analytical tools designed for the purpose of wildlife conservation, had demonstrated significant improvements in patrol coverage, with some observed reductions in poaching and contributing to wildlife... View Details

Keywords: Business and Government Relations; Emerging Markets; Technology Adoption; Strategy; Management; Ethics; Social Enterprise; AI and Machine Learning; Analytics and Data Science; Natural Environment; Technology Industry; Cambodia; United States; Africa

November 2021 (Revised December 2021)
Supplement

PittaRosso (B): Human and Machine Learning

By: Ayelet Israeli

This case supplements the "PittaRosso: Artificial Intelligence-Driven Pricing and Promotion" case, and provides major highlights on what happened at the company since the first case. View Details

Research Summary

Adoption of Machine Learning Models in Real World Decision Making

By: Himabindu Lakkaraju

The goal of this research is to assess the impact of deploying machine learning models in real world decision making in domains such as health care. View Details

14 Mar 2023
Cold Call Podcast

Can AI and Machine Learning Help Park Rangers Prevent Poaching?

Keywords: Re: Brian L. Trelstad; Computer; Information Technology; Technology

April 2025
Background Note

Climate Change Adaptation with Artificial Intelligence and Machine Learning

By: Michael W. Toffel and Nabig Chaudhry

Artificial Intelligence (AI) and machine learning (ML) have emerged as powerful tools to address climate change. This note summarizes a wide range of the uses of AI/ML to drive climate change adaptation and resilience, the measures organizations and governments are... View Details

Keywords: Climate Change; Adaptation