Publications
Publications
Show Results For
- All HBS Web
(31)
- Faculty Publications (14)
Page 1 of 14
Results
- 2024
- Working Paper
Winner Take All: Exploiting Asymmetry in Factorial Designs
By: Matthew DosSantos DiSorbo, Iavor I. Bojinov and Fiammetta Menchetti
Researchers and practitioners have embraced factorial experiments to simultaneously test multiple treatments, each with different levels. With the rise of technologies like Generative AI, factorial experimentation has become even more accessible: it is easier than ever... View Details
- 2024
- Working Paper
Design of Panel Experiments with Spatial and Temporal Interference
By: Tu Ni, Iavor Bojinov and Jinglong Zhao
One of the main practical challenges companies face when running experiments (or A/B tests) over a panel is interference, the setting where one experimental unit's treatment assignment at one time period impacts another's outcomes, possibly at the following time... View Details
Keywords: Research
- 2021
- Working Paper
Quantifying the Value of Iterative Experimentation
By: Iavor I Bojinov and Jialiang Mao
Over the past decade, most technology companies and a growing number of conventional firms have adopted online experimentation (or A/B testing) into their product development process. Initially, A/B testing was deployed as a static procedure in which an experiment was... View Details
- January 2024
- Article
Population Interference in Panel Experiments
By: Kevin Wu Han, Guillaume Basse and Iavor Bojinov
The phenomenon of population interference, where a treatment assigned to one experimental unit affects another experimental unit’s outcome, has received considerable attention in standard randomized experiments. The complications produced by population interference in... View Details
- 2023
- Article
Balancing Risk and Reward: An Automated Phased Release Strategy
By: Yufan Li, Jialiang Mao and Iavor Bojinov
Phased releases are a common strategy in the technology industry for gradually releasing new products or updates through a sequence of A/B tests in which the number of treated units gradually grows until full deployment or deprecation. Performing phased releases in a... View Details
- November–December 2023
- Article
Keep Your AI Projects on Track
By: Iavor Bojinov
AI—and especially its newest star, generative AI—is today a central theme in corporate boardrooms, leadership discussions, and casual exchanges among employees eager to supercharge their productivity. Sadly, beneath the aspirational headlines and tantalizing potential... View Details
- 2023
- Working Paper
Design-Based Confidence Sequences: A General Approach to Risk Mitigation in Online Experimentation
By: Dae Woong Ham, Michael Lindon, Martin Tingley and Iavor Bojinov
Randomized experiments have become the standard method for companies to evaluate the performance of new products or services. In addition to augmenting managers’ decision-making, experimentation mitigates risk by limiting the proportion of customers exposed to... View Details
- 2023
- Working Paper
Nailing Prediction: Experimental Evidence on the Value of Tools in Predictive Model Development
By: Daniel Yue, Paul Hamilton and Iavor Bojinov
Predictive model development is understudied despite its centrality in modern artificial
intelligence and machine learning business applications. Although prior discussions
highlight advances in methods (along the dimensions of data, computing power, and
algorithms)... View Details
Keywords: Analytics and Data Science
- Article
Online Experimentation: Benefits, Operational and Methodological Challenges, and Scaling Guide
By: Iavor Bojinov and Somit Gupta
In the past decade, online controlled experimentation, or A/B testing, at scale has proved to be a significant driver of business innovation. The practice was first pioneered by the technology sector and, more recently, has been adopted by traditional companies... View Details
- March 2022
- Article
Estimating the Effectiveness of Permanent Price Reductions for Competing Products Using Multivariate Bayesian Structural Time Series Models
By: Fiammetta Menchetti and Iavor Bojinov
Researchers regularly use synthetic control methods for estimating causal effects when a sub-set of units receive a single persistent treatment, and the rest are unaffected by the change. In many applications, however, units not assigned to treatment are nevertheless... View Details
- August 2021 (Revised February 2024)
- Case
Data Science at the Warriors
By: Iavor I. Bojinov and Michael Parzen
The case explores the development and early growth of a data science team at the Golden State Warriors, an NBA team based in San Francisco. The case begins by explaining the initial rationale for investing in data science, then covers a debate on the appropriate team... View Details
- 2023
- Working Paper
Virtual Water Coolers: A Field Experiment on the Role of Virtual Interactions on Organizational Newcomer Performance
Designing management practices to better onboard organizational newcomers working remotely is a key priority for firms. We report results from a randomized field experiment conducted at a large global firm that estimates the performance effects of different types of... View Details
- 2019
- Article
Time Series Experiments and Causal Estimands: Exact Randomization Tests and Trading
By: Iavor I Bojinov and Neil Shephard
We define causal estimands for experiments on single time series, extending the potential outcome framework to dealing with temporal data. Our approach allows the estimation of a broad class of these estimands and exact randomization based p-values for testing causal... View Details
- Research Summary
Overview
By: Iavor I. Bojinov
Over the last decade, technology companies like Amazon, Google, and Netflix have pioneered data-driven research and development processes centered on massive experimentation. However, as companies increase the breadth and scale of their experiments to millions of... View Details