• 2025
• Working Paper

As firms continue to seek efficiency by deploying Generative AI (GenAI) tools across various organizational functions, a critical question emerges: When and why does GenAI enable (versus constrain) individuals from one occupational group (i.e., outsiders) to perform tasks assigned to another occupational group (i.e., insiders), with equivalent speed and quality? And does GenAI’s effect diminish as the “knowledge distance” between occupational groups increases? In an experiment conducted at a large UK firm, we examine these questions. Three groups—occupational insiders, adjacent outsiders, and distant outsiders—attempted to both conceptualize as well as execute tasks that are “core” to occupational insiders and were randomly assigned to receive support from a bespoke GenAI tool. We found that a “GenAI wall”—that is, the point at which GenAI can no longer meaningfully reduce the expertise gap between occupational insiders and outsiders—emerged for the joint effect of knowledge distance and task characteristics. Specifically, we found that GenAI is more effective at bridging expertise gaps between near (rather than distant) occupations, and more so for conceptualization (as opposed to execution) tasks. We discuss the implications of these findings for scholarship on occupations, learning, and division of labor in the wake of emerging technologies such as GenAI.

• 2025
• Working Paper

Auction environments play an increasingly central role across industries such as transportation, commodities, and media. Companies like Procter & Gamble (P&G) rely on advanced digital solutions to optimize auction performance and must rigorously validate algorithmic changes before wide-scale deployment. Yet standard A/B testing methods are frequently infeasible in these settings because user randomization is not under the control of the auction seller. Switchback experiments, which randomize treatment assignment over discrete time periods, offer a practical alternative but often suffer from low statistical power and are further complicated by potential carryover effects, where previous treatments can affect current outcomes. This paper introduces a new rerandomization-based design that addresses both the issue of achieving adequate covariate balance and the challenges posed by carryover effects. Building on assignment paths from a completely randomized design, we measure covariate imbalance and only accept assignment schedules below a specified threshold, thus enhancing precision. We then propose an analytic adjustment for the practical scenario when the experiment has to be paused, treating disruptions—such as business parameter constraints or technical failures—as fixed yet unknown events that can be incorporated into the variance calibration. Through extensive numerical simulations mimicking real data from the auction environment, we show that our approach substantially boosts efficiency, remains robust to different choices of covariate sets and imbalance thresholds, and adapts well under assignment non-compliance. Since Jan 2025, P&G has integrated the new rerandomization strategy into its experimentation workflow—applying it to 70 experiments across five auction hypotheses in eight global markets. The results indicate a potential 10–20% sales lift and 5–15% cost savings. Beyond its immediate application to switchback experiments at P&G, the methodology underscores a broader principle in data-driven decision-making: carefully engineered randomization, combined with robust analytic strategies that account for real-world implementation challenges, can yield reliable and actionable insights even in complex operational contexts.

• September–October 2025
• Article

For all the enthusiasm around generative AI, a hurdle is limiting its adoption: the technology’s tendency to make things up, leave things out, and create so many possibilities that it is hard to figure out which will be effective. That’s why the vast majority of companies employ human reviews and stand-alone testing tools, but these quality-control methods are expensive, and they can handle only a fraction of gen AI’s total output. Amazon has developed a better approach for its massive product ­catalog operation: a gen AI–based system named Catalog AI that can automatically detect and block unreliable data, produce ideas for new product pages and test their effectiveness, and improve itself with feedback from quality checks and experiments. In this article Harvard Business School’s Stefan Thomke and Amazon’s Philipp Eisenhauer and Puneet Sahni describe Amazon’s system for performing quality control on AI-generated content at scale. Although Amazon considers Catalog AI to be a work in progress, the authors believe that it is far enough along that managers at other organizations can benefit from learning about it now.