Publications

September–October 2024
Article

How AI Can Power Brand Management

By: Julian De Freitas and Elie Ofek

Marketers have begun experimenting with AI to improve their brand-management efforts. But unlike other marketing tasks, brand management involves more than just repeatedly executing one specialized function. Long considered the exclusive domain of creative talent, it... View Details

Keywords: Creativity; AI and Machine Learning; Brands and Branding; Product Positioning; Customer Focus and Relationships

August 2024 (Revised September 2024)
Case

Dishoom: From Bombay with Love

By: Anjali M. Bhatt and Thomas J. DeLong

Shamil and Kavi Thakrar, co-founders of Dishoom, faced critical decisions as they looked to expand the UK-based restaurant group. Shamil, the CEO, was confident in Dishoom's potential for growth but he was concerned about preserving the culture and values centered... View Details

Keywords: Human Resource Management; Growth And Scaling; Organizational Culture; Values and Beliefs; Growth Management; Expansion; United Kingdom

Educators

July 2024
Article

Chatbots and Mental Health: Insights into the Safety of Generative AI

By: Julian De Freitas, Ahmet Kaan Uğuralp, Zeliha Uğuralp and Stefano Puntoni

Chatbots are now able to engage in sophisticated conversations with consumers. Due to the ‘black box’ nature of the algorithms, it is impossible to predict in advance how these conversations will unfold. Behavioral research provides little insight into potential safety... View Details

Keywords: Autonomy; Chatbots; New Technology; Brand Crises; Mental Health; Large Language Model; AI and Machine Learning; Behavior; Well-being; Technological Innovation; Ethics

July–August 2024
Article

Doing More with Less: Overcoming Ineffective Long-Term Targeting Using Short-Term Signals

By: Ta-Wei Huang and Eva Ascarza

Firms are increasingly interested in developing targeted interventions for customers with the best response, which requires identifying differences in customer sensitivity, typically through the conditional average treatment effect (CATE) estimation. In theory, to... View Details

Keywords: Long-run Targeting; Heterogeneous Treatment Effect; Statistical Surrogacy; Customer Churn; Field Experiments; Consumer Behavior; Customer Focus and Relationships; AI and Machine Learning; Marketing Strategy

July 2024
Article

How Artificial Intelligence Constrains Human Experience

By: A. Valenzuela, S. Puntoni, D. Hoffman, N. Castelo, J. De Freitas, B. Dietvorst, C. Hildebrand, Y.E. Huh, R. Meyer, M. Sweeney, S. Talaifar, G. Tomaino and K. Wertenbroch

Many consumption decisions and experiences are digitally mediated. As a consequence, consumer behavior is increasingly the joint product of human psychology and ubiquitous algorithms (Braun et al. 2024; cf. Melumad et al. 2020). The coming of age of Large Language... View Details

Keywords: Large Language Model; User Experience; AI and Machine Learning; Consumer Behavior; Technology Adoption; Risk and Uncertainty; Cost vs Benefits

2024
Working Paper

Navigating Software Vulnerabilities: Eighteen Years of Evidence from Medium and Large U.S. Organizations

By: Raviv Murciano-Goroff, Ran Zhuo and Shane Greenstein

How prevalent are severe software vulnerabilities, how fast do software users respond to the availability of secure versions, and what determines the variance in the installation distribution? Using the largest dataset ever assembled on user updates, tracking server... View Details

Keywords: Cybersecurity; Applications and Software; Technology Adoption; Consumer Behavior

Find at Harvard

2024
Working Paper

Webmunk: A New Tool for Studying Online Behavior and Digital Platforms

By: Chiara Farronato, Audrey Fradkin and Chris Karr

Understanding the behavior of users online is important for researchers, policymakers, and private companies alike. But observing online behavior and conducting experiments is difficult without direct access to the user base and software of technology companies. We... View Details

Keywords: Policy; Technology Adoption; Behavior; Research; Consumer Behavior; Internet and the Web

Find at Harvard

June 2024
Teaching Note

Skills-First Hiring at IBM

By: Boris Groysberg and Sarah Mehta

Teaching note for “Skills-First Hiring at IBM,” case no. 422-013. View Details

Keywords: Competency and Skills; Experience and Expertise; Talent and Talent Management; Human Resources; Human Capital; Employees; Recruitment; Retention; Selection and Staffing; Jobs and Positions; Job Design and Levels; Job Interviews; Society; Societal Protocols; Technology Industry; United States; New York (state, US)

2024
Working Paper

AI Companions Reduce Loneliness

By: Julian De Freitas, Ahmet K Uguralp, Zeliha O Uguralp and Puntoni Stefano

Chatbots are now able to engage in sophisticated conversations with consumers in the domain of relationships, providing a potential coping solution to widescale societal loneliness. Behavioral research provides little insight into whether these applications are... View Details

Keywords: AI and Machine Learning; Technological Innovation; Behavior; Well-being

2024
Working Paper

Incrementality Representation Learning: Synergizing Past Experiments for Intervention Personalization

By: Ta-Wei Huang, Eva Ascarza and Ayelet Israeli

This paper introduces Incrementality Representation Learning (IRL), a novel multitask representation learning framework that predicts heterogeneous causal effects of marketing interventions. By leveraging past experiments, IRL efficiently designs and targets... View Details

Keywords: Heterogeneous Treatment Effect; Multi-task Learning; Representation Learning; Personalization; Promotion; Deep Learning; Field Experiments; Customer Focus and Relationships; Customization and Personalization

2024
Working Paper

Winner Take All: Exploiting Asymmetry in Factorial Designs

By: Matthew DosSantos DiSorbo, Iavor I. Bojinov and Fiammetta Menchetti

Researchers and practitioners have embraced factorial experiments to simultaneously test multiple treatments, each with different levels. With the rise of technologies like Generative AI, factorial experimentation has become even more accessible: it is easier than ever... View Details

Keywords: Factorial Designs; Fisher Randomizations; Rank Estimators; Employer Interventions; Causal Inference; Mathematical Methods; Performance Improvement

2024
Working Paper

Personalization and Targeting: How to Experiment, Learn & Optimize

By: Aurelie Lemmens, Jason M.T. Roos, Sebastian Gabel, Eva Ascarza, Hernan Bruno, Elea McDonnell Feit, Brett Gordon, Ayelet Israeli, Carl F. Mela and Oded Netzer

Personalization has become the heartbeat of modern marketing. Advances in causal inference and machine learning enable companies to understand how the same marketing action can impact the choices of individual customers differently. This article provides an academic... View Details

Keywords: Personalization; Targeting; Experiments; Observational Studies; Policy Implementation; Policy Evaluation; Customization and Personalization; Marketing Strategy; AI and Machine Learning

2024
Working Paper

Business Experiments as Persuasion

By: Orie Shelef, Rebecca Karp and Robert Wuebker

Much of the prior work on experimentation rests upon the assumption that entrepreneurs and managers use—or should optimally adopt—a "scientific approach" to test possible decisions before making them. This paper offers an alternative view of experimental strategy,... View Details

Keywords: Experimentation; Entrepreneurship; Strategy; Decision Making; Innovation Strategy

2023
Working Paper

An Experimental Design for Anytime-Valid Causal Inference on Multi-Armed Bandits

By: Biyonka Liang and Iavor I. Bojinov

Typically, multi-armed bandit (MAB) experiments are analyzed at the end of the study and thus require the analyst to specify a fixed sample size in advance. However, in many online learning applications, it is advantageous to continuously produce inference on the... View Details

Keywords: Analytics and Data Science; AI and Machine Learning; Mathematical Methods

2024
Working Paper

Anytime-Valid Inference in Linear Models and Regression-Adjusted Causal Inference

By: Michael Lindon, Dae Woong Ham, Martin Tingley and Iavor I. Bojinov

Linear regression adjustment is commonly used to analyze randomized controlled experiments due to its efficiency and robustness against model misspecification. Current testing and interval estimation procedures leverage the asymptotic distribution of such estimators to... View Details

Keywords: Mathematical Methods; Analytics and Data Science

2024
Working Paper

Design of Panel Experiments with Spatial and Temporal Interference

By: Tu Ni, Iavor Bojinov and Jinglong Zhao

One of the main practical challenges companies face when running experiments (or A/B tests) over a panel is interference, the setting where one experimental unit's treatment assignment at one time period impacts another's outcomes, possibly at the following time... View Details

Keywords: Research

2021
Working Paper

Quantifying the Value of Iterative Experimentation

By: Iavor I Bojinov and Jialiang Mao

Over the past decade, most technology companies and a growing number of conventional firms have adopted online experimentation (or A/B testing) into their product development process. Initially, A/B testing was deployed as a static procedure in which an experiment was... View Details

Keywords: Product Development; Value Creation; Research

2024
Book

Deals: The Economic Structure of Business Transactions

By: Guhan Subramanian and Michael Klausner

Drawing on real-life cases from a wide range of industries, two acclaimed experts offer a sophisticated but accessible guide to business deals, designed to maximize value for your side.

Business transactions take widely varying forms—from multibillion-dollar... View Details

Keywords: Negotiation; Contracts

Find at Harvard

January 11, 2024
Article

Understanding the Tradeoffs of the Amazon Antitrust Case

By: Chiara Farronato, Andrey Fradkin, Andrei Hagiu and Dionne Lomax

Regulators in the United States and Europe have been taking on Big Tech, challenging what they say are the companies’ anti-competitive and predatory strategies that harm consumers and third-party users of their platforms. This article examines the FTC’s case against... View Details

Keywords: Monopoly; Governing Rules, Regulations, and Reforms; Market Design; Lawsuits and Litigation