Publications
Publications
Show Results For
- All HBS Web
(668)
- Faculty Publications (208)
- 2023
- Article
Benchmarking Large Language Models on CMExam—A Comprehensive Chinese Medical Exam Dataset
By: Junling Liu, Peilin Zhou, Yining Hua, Dading Chong, Zhongyu Tian, Andrew Liu, Helin Wang, Chenyu You, Zhenhua Guo, Lei Zhu and Michael Lingzhi Li
Recent advancements in large language models (LLMs) have transformed the field of question answering (QA). However, evaluating LLMs in the medical field is challenging due to the lack of standardized and comprehensive datasets. To address this gap, we introduce CMExam,... View Details
- October 2023 (Revised June 2024)
- Case
ReUp Education: Can AI Help Learners Return to College?
By: Kris Ferreira, Christopher Thomas Ryan and Sarah Mehta
Founded in 2015, ReUp Education helps “stopped out students”—learners who have stopped making progress towards graduation—achieve their college completion goals. The company relies on a team of success coaches to engage with learners and help them reenroll. In 2019,... View Details
- 2023
- Working Paper
In-Context Unlearning: Language Models as Few Shot Unlearners
By: Martin Pawelczyk, Seth Neel and Himabindu Lakkaraju
Machine unlearning, the study of efficiently removing the impact of specific training points on the
trained model, has garnered increased attention of late, driven by the need to comply with privacy
regulations like the Right to be Forgotten. Although unlearning is... View Details
- October 2023
- Article
Matching Mechanisms for Refugee Resettlement
By: David Delacrétaz, Scott Duke Kominers and Alexander Teytelboym
Current refugee resettlement processes account for neither the preferences of refugees nor the priorities of hosting communities. We introduce a new framework for matching with multidimensional knapsack constraints that captures the (possibly multidimensional) sizes of... View Details
- 2023
- Working Paper
The Impact of Input Inaccuracy on Leveraging AI Tools: Evidence from Algorithmic Labor Scheduling
By: Caleb Kwon, Antonio Moreno and Ananth Raman
Are the inputs used by your AI tool correct and up to date? In this paper, we show that the answer to this question: (i) is frequently a “no” in real business contexts, and (ii) has significant implications on the performance of AI tools. In the context of algorithmic... View Details
- September 29, 2023
- Article
Eliminating Algorithmic Bias Is Just the Beginning of Equitable AI
By: Simon Friis and James Riley
When it comes to artificial intelligence and inequality, algorithmic bias rightly receives a lot of attention. But it’s just one way that AI can lead to inequitable outcomes. To truly create equitable AI, we need to consider three forces through which it might make... View Details
- September–October 2023
- Article
Interpretable Matrix Completion: A Discrete Optimization Approach
By: Dimitris Bertsimas and Michael Lingzhi Li
We consider the problem of matrix completion on an n × m matrix. We introduce the problem of interpretable matrix completion that aims to provide meaningful insights for the low-rank matrix using side information. We show that the problem can be... View Details
Keywords: Mathematical Methods
- 2023
- Article
On Minimizing the Impact of Dataset Shifts on Actionable Explanations
By: Anna P. Meyer, Dan Ley, Suraj Srinivas and Himabindu Lakkaraju
The Right to Explanation is an important regulatory principle that allows individuals to request actionable explanations for algorithmic decisions. However, several technical challenges arise when providing such actionable explanations in practice. For instance, models... View Details
- 2023
- Article
On the Impact of Actionable Explanations on Social Segregation
By: Ruijiang Gao and Himabindu Lakkaraju
As predictive models seep into several real-world applications, it has become critical to ensure that individuals who are negatively impacted by the outcomes of these models are provided with a means for recourse. To this end, there has been a growing body of research... View Details
- July–August 2023
- Article
Demand Learning and Pricing for Varying Assortments
By: Kris Ferreira and Emily Mower
Problem Definition: We consider the problem of demand learning and pricing for retailers who offer assortments of substitutable products that change frequently, e.g., due to limited inventory, perishable or time-sensitive products, or the retailer’s desire to... View Details
- 2023
- Article
Towards Bridging the Gaps between the Right to Explanation and the Right to Be Forgotten
By: Himabindu Lakkaraju, Satyapriya Krishna and Jiaqi Ma
The Right to Explanation and the Right to be Forgotten are two important principles outlined to regulate algorithmic decision making and data usage in real-world applications. While the right to explanation allows individuals to request an actionable explanation for an... View Details
- July 2023
- Case
DayTwo: Going to Market with Gut Microbiome (Abridged)
By: Ayelet Israeli
DayTwo is a young Israeli startup that applies research on the gut microbiome and machine learning algorithms to deliver personalized nutritional recommendations to its users in order to minimize blood sugar spikes after meals. After a first year of trial rollout in... View Details
- 2023
- Working Paper
Algorithm Failures and Consumers' Response: Evidence from Zillow
By: Isamar Troncoso, Runshan Fu, Nikhil Malik and Davide Proserpio
In November 2021, Zillow announced the closure of its iBuyer business. Popular media largely attributed this to a failure of its proprietary forecasting algorithm. We study the response of consumers to Zillow’s iBuyer business closure. We show that after the iBuyer... View Details
- June 2023
- Simulation
Artea Dashboard and Targeting Policy Evaluation
By: Ayelet Israeli and Eva Ascarza
Companies deploy A/B experiments to gain valuable insights about their customers in order to answer strategic business problems. In marketing, A/B tests are often used to evaluate marketing interventions intended to generate incremental outcomes for the firm. The Artea... View Details
Keywords: Algorithm Bias; Algorithmic Data; Race And Ethnicity; Experimentation; Promotion; Marketing And Society; Big Data; Privacy; Data-driven Management; Data Analysis; Data Analytics; E-Commerce Strategy; Discrimination; Targeted Advertising; Targeted Policies; Pricing Algorithms; A/B Testing; Ethical Decision Making; Customer Base Analysis; Customer Heterogeneity; Coupons; Marketing; Race; Gender; Diversity; Customer Relationship Management; Marketing Communications; Advertising; Decision Making; Ethics; E-commerce; Analytics and Data Science; Retail Industry; Apparel and Accessories Industry; United States
- 2023
- Working Paper
Insufficiently Justified Disparate Impact: A New Criterion for Subgroup Fairness
By: Neil Menghani, Edward McFowland III and Daniel B. Neill
In this paper, we develop a new criterion, "insufficiently justified disparate impact" (IJDI), for assessing whether recommendations (binarized predictions) made by an algorithmic decision support tool are fair. Our novel, utility-based IJDI criterion evaluates false... View Details
- June 2023
- Exercise
Experimenting with Algorithm Resume Screening
By: Michael Luca, Jesse M. Shapiro, Adrian Obleton, Evelyn Ramirez and Nathan Sun
- May–June 2023
- Article
Analytics for Marketers: When to Rely on Algorithms and When to Trust Your Gut
By: Fabrizio Fantini and Das Narayandas
Advanced analytics can help companies solve a host of management problems, including those related to marketing, sales, and supply-chain operations, which can lead to a sustainable competitive advantage. But as more data becomes available and advanced analytics are... View Details
- 2023
- Working Paper
Digital Lending and Financial Well-Being: Through the Lens of Mobile Phone Data
By: AJ Chen, Omri Even-Tov, Jung Koo Kang and Regina Wittenberg-Moerman
To mitigate information asymmetry about borrowers in developing economies, digital lenders utilize machine-learning algorithms and nontraditional data from borrowers’ mobile devices. Consequently, digital lenders have managed to expand access to credit for millions of... View Details
- May 2023
- Technical Note
Dynamic Pricing: Timing is Everything
By: Elie Ofek
This note provides a comprehensive exposition to the topic of dynamic pricing (whereby the fee customers are charged is time-dependent). It covers the motivation for firms to engage in dynamic pricing, provides a typology of the main formats dynamic pricing can take,... View Details
- 2023
- Article
Probabilistically Robust Recourse: Navigating the Trade-offs between Costs and Robustness in Algorithmic Recourse
By: Martin Pawelczyk, Teresa Datta, Johannes van-den-Heuvel, Gjergji Kasneci and Himabindu Lakkaraju
As machine learning models are increasingly being employed to make consequential decisions in real-world settings, it becomes critical to ensure that individuals who are adversely impacted (e.g., loan denied) by the predictions of these models are provided with a means... View Details