A/B Testing - Publications - Faculty & Research - Harvard Business School

Filter Results: (44)

Filter Results: (44)

Show Results For

Page 1 of 44 Results →

Sort by

Are you looking for?

→Search All HBS Web

March–April 2020
Article

Avoid the Pitfalls of A/B Testing

By: Iavor I. Bojinov, Guillaume Sait-Jacques and Martin Tingley

Online experiments measuring whether “A,” usually the current approach, is inferior to “B,” a proposed improvement, have become integral to the product-development cycle, especially at digital enterprises. But often firms make serious mistakes in conducting these... View Details

Keywords: A/B Testing; Experiment Design; Social Networks; Product Development; Performance Improvement; Measurement and Metrics; Social Media

Find at Harvard

Register to Read

2020
Working Paper

Digital Experimentation and Startup Performance: Evidence from A/B Testing

By: Rembrand Koning, Sharique Hasan and Aaron Chatterji

Recent work argues that experimentation is the appropriate framework for entrepreneurial strategy. We investigate this proposition by exploiting the time-varying adoption of A/B testing technology, which has drastically reduced the cost of experimentally testing... View Details

Keywords: Experimentation; A/B Testing; Data-driven Decision-making; Entrepreneurship; Strategy; Business Startups; Information Technology; Performance

September 2022
Article

Experimentation and Start-up Performance: Evidence from A/B Testing

By: Rembrand Koning, Sharique Hasan and Aaron Chatterji

Recent scholarship has argued that experimentation should be the organizing principle for entrepreneurial strategy. Experimentation leads to organizational learning, which drives improvements in firm performance. We investigate this proposition by exploiting the... View Details

Keywords: Experimentation; A/B Testing; Data-driven Decision-making; Organizational Learning; Entrepreneurship; Strategy; Business Startups; Learning; Performance; Decision Making

Find at Harvard

September–October 2017
Article

The Surprising Power of Online Experiments: Getting the Most Out of A/B and Other Controlled Tests

By: Ron Kohavi and Stefan Thomke

In the fast-moving digital world, even experts have a hard time assessing new ideas. Case in point: At Bing, a small headline change an employee proposed was deemed a low priority and shelved for months until one engineer decided to do a quick online controlled... View Details

Keywords: Experiments; A/B Testing; Research; Consumer Behavior

Find at Harvard

Register to Read

11 Jan 2021
Research & Ideas

Is A/B Testing Effective? Evidence from 35,000 Startups

Startups that use A/B testing scale more quickly, launch more products, and attract more venture capital, according to recent research. But not all startups enjoy these benefits. Early-stage startups using... View Details

Keywords: by Kristen Senz

06 Sep 2019
Working Paper Summaries

Experimentation and Startup Performance: Evidence from A/B Testing

Keywords: by Rembrand Koning, Sharique Hasan, and Aaron Chatterji

June 23, 2021
Article

Research: When A/B Testing Doesn't Tell You the Whole Story

By: Eva Ascarza

When it comes to churn prevention, marketers traditionally start by identifying which customers are most likely to churn, and then running A/B tests to determine whether a proposed retention intervention will be effective at retaining those high-risk customers. While... View Details

Keywords: Customer Retention; Churn; Targeting; Market Research; Marketing; Investment Return; Customers; Retention; Research

Find at Harvard

Register to Read

Article

Online Experimentation: Benefits, Operational and Methodological Challenges, and Scaling Guide

By: Iavor Bojinov and Somit Gupta

In the past decade, online controlled experimentation, or A/B testing, at scale has proved to be a significant driver of business innovation. The practice was first pioneered by the technology sector and, more recently, has been adopted by traditional companies... View Details

Keywords: A/B Testing; Experimentation; Data-driven Culture; Product Development; Innovation and Invention; Digital Transformation

March 2021
Supplement

Artea (A), (B), (C), and (D): Designing Targeting Strategies

By: Eva Ascarza and Ayelet Israeli

Power Point Supplement to Teaching Note for HBS No. 521-021,521-022,521-037,521-043. This collection of exercises aims to teach students about 1)Targeting Policies; and 2)Algorithmic bias in marketing—implications, causes, and possible solutions. Part (A) focuses on... View Details

Keywords: Targeted Advertising; Targeting; Algorithmic Data; Bias; A/B Testing; Experiment; Advertising; Gender; Race; Diversity; Marketing; Customer Relationship Management; Prejudice and Bias; Analytics and Data Science; Retail Industry; Apparel and Accessories Industry; Technology Industry; United States

October 2018
Case

Booking.com

By: Stefan Thomke and Daniela Beyersdorfer

The case reveals how Booking.com has become the world's leading travel accommodation platform. The company has put online experimentation at the heart of how it designs digital experiences for its customers and partners. To unlock the potential of large-scale testing,... View Details

Keywords: Travel; Product Innovation; Experimentation; A/B Testing; User Experience Design; Product Development; Product Design; Innovation and Management; Transformation; Information Technology; Digital Transformation; Travel Industry

September 2015 (Revised March 2025)
Technical Note

FIELD Global Capstone: Developing Customer Empathy

The Design Thinking process begins with empathizing with potential customers. Empathizing, being aware of, interpreting, and understanding the thoughts of others, as well as being able to vicariously experience them oneself, requires the careful and deliberate study of... View Details

Keywords: Market Research; Design Thinking; Customer Behavior; Ethnography; Interviews; Surveys; A/B Testing; Experimentation; Marketing; Customer Focus and Relationships; Consumer Behavior; Demand and Consumers

June 2023
Simulation

Artea Dashboard and Targeting Policy Evaluation

By: Ayelet Israeli and Eva Ascarza

Companies deploy A/B experiments to gain valuable insights about their customers in order to answer strategic business problems. In marketing, A/B tests are often used to evaluate marketing interventions intended to generate incremental outcomes for the firm. The Artea... View Details

2013
Book

Managing Startups: Best Blog Posts

By: Tom Eisenmann

Harvard Business School Professor Tom Eisenmann annually compiles the best posts from many blogs on technology startup management, primarily for the benefit of his students. This book makes his latest collection available to the broader entrepreneur community. Divided... View Details

Keywords: Lean Startup; Startup; Prototyping; MVP; Minimum Viable Product; Freemium; SaaS; A/B Testing; Business Model

September 2020 (Revised June 2023)
Exercise

Artea: Designing Targeting Strategies

By: Eva Ascarza and Ayelet Israeli

This collection of exercises aims to teach students about 1)Targeting Policies; and 2)Algorithmic bias in marketing—implications, causes, and possible solutions. Part (A) focuses on A/B testing analysis and targeting. Parts (B),(C),(D) Introduce algorithmic bias. The... View Details

August 2018 (Revised February 2023)
Case

Hubble Contact Lenses: Data Driven Direct-to-Consumer Marketing

By: Jill Avery and Ayelet Israeli

As its Series A extension round approaches, the founders of Hubble, a subscription-based, social-media fueled, direct-to-consumer (DTC) brand of contact lenses, are reflecting on the marketing strategies that have taken them to a valuation of $200 million and debating... View Details

January 2019 (Revised February 2024)
Teaching Note

Hubble Contact Lenses: Data Driven Direct-to-Consumer Marketing

By: Ayelet Israeli

Teaching Note for HBS No. 519-011. As its Series A extension round approaches, the founders of Hubble, a subscription-based, social-media fueled, direct-to-consumer (DTC) brand of contact lenses, are reflecting on the marketing strategies that have taken them to a... View Details

February 2024
Module Note

Data-Driven Marketing in Retail Markets

By: Ayelet Israeli

This note describes an eight-class sessions module on data-driven marketing in retail markets. The module aims to familiarize students with core concepts of data-driven marketing in retail, including exploring the opportunities and challenges, adopting best practices,... View Details

September 2023
Supplement

Design and Evaluation of Targeted Interventions

By: Eva Ascarza

Targeted interventions serve as a pivotal tool in business strategy, streamlining decisions for enhanced efficiency and effectiveness. This note delves into two central facets of such interventions: first, the design of potent decision guidelines, or targeting... View Details

Keywords: Marketing; Design; Business Strategy; Policy; Retail Industry; Apparel and Accessories Industry; Technology Industry; Financial Services Industry; Telecommunications Industry

October 2023 (Revised February 2024)
Technical Note

Design and Evaluation of Targeted Interventions

By: Eva Ascarza and Ta-Wei (David) Huang

Targeted interventions serve as a pivotal tool in business strategy, streamlining decisions for enhanced efficiency and effectiveness. This note delves into two central facets of such interventions: first, the design of potent decision guidelines, or targeting... View Details

Keywords: Marketing; Customer Relationship Management; Analysis; Design; Business Strategy; Retail Industry; Apparel and Accessories Industry; Technology Industry; Financial Services Industry; Telecommunications Industry

February 2021
Tutorial

T-tests: Theory and Practice

By: Michael Parzen, Natalie Epstein, Chiara Farronato and Michael Toffel

This video provides an introduction to hypothesis testing, sampling, t-tests, and p-values. It provides examples of A/B testing and t-testing to assess whether difference between two groups are statistically significant. This video can be assigned in conjunction with... View Details

Keywords: Data Analysis; Data Analytics; Experiment Design; Experimentation; Analytics and Data Science; Analysis

1
2
3
→