Publications

November 2020
Case

Creating a Virtual Internship at Goldman Sachs

By: Prithwiraj Choudhury, Iavor I. Bojinov and Emma Salomon

Goldman Sachs runs an annual internship for over 3,000 participants, spread across dozens of the firm's global offices. In 2020, the team brought all its resources to bear to transform the internship program into a fully virtual format in just a few short weeks. The... View Details

Keywords: Remote Work; Remote Operations; Remote Internship; Internship; Virtual Socialization; Human Capital Management; Human Resources; Management; Health Pandemics; Adaptation

Educators

October 2020 (Revised March 2024)
Case

Experimentation at Yelp

By: Iavor Bojinov and Karim R. Lakhani

Over the last decade, experimentation has become integral to the research and development processes of technology companies—including Yelp—for understanding customer preferences and mitigating innovation risks. The case describes Yelp's journey with experimentation,... View Details

Keywords: Customer Relationship Management; Collaborative Innovation and Invention; Risk Management; Advertising; Research and Development; Technology Industry

Educators

2020
Working Paper

Design and Analysis of Switchback Experiments

By: Iavor I Bojinov, David Simchi-Levi and Jinglong Zhao

In switchback experiments, a firm sequentially exposes an experimental unit to a random treatment, measures its response, and repeats the procedure for several periods to determine which treatment leads to the best outcome. Although practitioners have widely adopted... View Details

Keywords: Switchback Experiments; Design; Analysis; Mathematical Methods

August 2020
Technical Note

Comparing Two Groups: Sampling and t-Testing

By: Iavor I Bojinov, Chiara Farronato, Yael Grushka-Cockayne, Willy C. Shih and Michael W. Toffel

This note describes sampling and t-tests, two fundamental statistical concepts. View Details

Keywords: Statistics; Econometric Analyses; Experimental Methods; Data Analysis; Data Analytics; Analytics and Data Science; Analysis; Surveys; Mathematical Methods

Educators

Article

The Importance of Being Causal

By: Iavor I Bojinov, Albert Chen and Min Liu

Causal inference is the study of how actions, interventions, or treatments affect outcomes of interest. The methods that have received the lion’s share of attention in the data science literature for establishing causation are variations of randomized experiments.... View Details

Keywords: Causal Inference; Observational Studies; Cross-sectional Studies; Panel Studies; Interrupted Time-series; Instrumental Variables

March–April 2020
Article

Avoid the Pitfalls of A/B Testing

By: Iavor I. Bojinov, Guillaume Sait-Jacques and Martin Tingley

Online experiments measuring whether “A,” usually the current approach, is inferior to “B,” a proposed improvement, have become integral to the product-development cycle, especially at digital enterprises. But often firms make serious mistakes in conducting these... View Details

Keywords: A/B Testing; Experiment Design; Social Networks; Product Development; Performance Improvement; Measurement and Metrics; Social Media

Find at Harvard

Register to Read

March 2020
Article

Diagnosing Missing Always at Random in Multivariate Data

By: Iavor I. Bojinov, Natesh S. Pillai and Donald B. Rubin

Models for analyzing multivariate data sets with missing values require strong, often assessable, assumptions. The most common of these is that the mechanism that created the missing data is ignorable—a twofold assumption dependent on the mode of inference. The first... View Details

Keywords: Missing Data; Diagnostic Tools; Sensitivity Analysis; Hypothesis Testing; Missing At Random; Row Exchangeability; Analytics and Data Science; Mathematical Methods

Find at Harvard

2020
Working Paper

A General Theory of Identification

By: Iavor Bojinov and Guillaume Basse

What does it mean to say that a quantity is identifiable from the data? Statisticians seem to agree on a definition in the context of parametric statistical models — roughly, a parameter θ in a model P = {Pθ : θ ∈ Θ} is identifiable if the mapping θ 7→ Pθ is injective.... View Details

Keywords: Identification; Econometric Models; Analytics and Data Science; Theory

2019
Article

Time Series Experiments and Causal Estimands: Exact Randomization Tests and Trading

By: Iavor I Bojinov and Neil Shephard

We define causal estimands for experiments on single time series, extending the potential outcome framework to dealing with temporal data. Our approach allows the estimation of a broad class of these estimands and exact randomization based p-values for testing causal... View Details

Keywords: Causality; Nonparametric; Potential Outcomes; Trading Costs; Mathematical Methods

Find at Harvard

Register to Read

2016
Conference Paper

The Pressing Game: Optimal Defensive Disruption in Soccer

By: Iavor I. Bojinov and Luke Bornn

Soccer, the most watched sport in the world, is a dynamic game where a team’s success relies on both team strategy and individual player contributions. Passing is a cardinal soccer skill and a key factor in strategy development; it helps the team to keep the ball... View Details

Keywords: Soccer; Strategy; Disruption