Filter Results:
(386)
Show Results For
- All HBS Web (386)
- Faculty Publications (149)
Show Results For
- All HBS Web (386)
- Faculty Publications (149)
- 25 Mar 2021
- Research & Ideas
Steer Clear of the Blind Spots That Derail Experiments
look at how companies could more effectively leverage structured, randomized experiments to inform decision-making. His goal: Enable C-suite leaders to avoid the blind spots and blunders that can occur when... View Details
Keywords: by Danielle Kost
- 2018
- Working Paper
Show or Tell? Improving Agent Decision Making in a Tanzanian Mobile Money Field Experiment
By: Jason Acimovic, Chris Parker, David F. Drake and Karthik Balasubramanian
When workers make operational decisions, the firm's global knowledge and the workers’ domain-specific knowledge complement each other. Oftentimes workers have the final decision-making power. Two key decisions a firm makes when designing systems to support these... View Details
Keywords: Employees; Decision Making; Training; Performance Improvement; Money; Mobile Technology; Developing Countries and Economies; Financial Services Industry
Acimovic, Jason, Chris Parker, David F. Drake, and Karthik Balasubramanian. "Show or Tell? Improving Agent Decision Making in a Tanzanian Mobile Money Field Experiment." Harvard Business School Working Paper, No. 18-106, May 2018.
- 2024
- Working Paper
Winner Take All: Exploiting Asymmetry in Factorial Designs
By: Matthew DosSantos DiSorbo, Iavor I. Bojinov and Fiammetta Menchetti
Researchers and practitioners have embraced factorial experiments to simultaneously test multiple treatments, each with different levels. With the rise of technologies like Generative AI, factorial experimentation has become even more accessible: it is easier than ever... View Details
Keywords: Factorial Designs; Fisher Randomizations; Rank Estimators; Employer Interventions; Causal Inference; Mathematical Methods; Performance Improvement
DosSantos DiSorbo, Matthew, Iavor I. Bojinov, and Fiammetta Menchetti. "Winner Take All: Exploiting Asymmetry in Factorial Designs." Harvard Business School Working Paper, No. 24-075, June 2024.
- April 2020
- Article
A Theory of Experimenters: Robustness, Randomization, and Balance
By: Abhijit Banerjee, Sylvain Chassang, Sergio Montero and Erik Snowberg
This paper studies the problem of experiment design by an ambiguity-averse decisionmaker who trades off subjective expected performance against robust performance guarantees. This framework accounts for real-world experimenters’ preference for randomization. It also... View Details
Banerjee, Abhijit, Sylvain Chassang, Sergio Montero, and Erik Snowberg. "A Theory of Experimenters: Robustness, Randomization, and Balance." American Economic Review 110, no. 4 (April 2020): 1206–1230.
- April 2023
- Article
Performance on Patient Experience Measures of Former Chief Medical Residents as Physician Exemplars Chosen by the Profession
By: Lucy Chen and J. Michael McWilliams
OBJECTIVE To compare care for patients of primary care physicians (PCPs) who were former chiefs with care for patients of nonchief PCPs.
DESIGN, SETTING, AND PARTICIPANTS Using 2010 to 2018 Medicare Fee-For-Service Consumer Assessment of Healthcare... View Details
DESIGN, SETTING, AND PARTICIPANTS Using 2010 to 2018 Medicare Fee-For-Service Consumer Assessment of Healthcare... View Details
Keywords: Performance Evaluation; Forecasting and Prediction; Knowledge Use and Leverage; Competency and Skills; Surveys; Health Industry
Chen, Lucy, and J. Michael McWilliams. "Performance on Patient Experience Measures of Former Chief Medical Residents as Physician Exemplars Chosen by the Profession." JAMA Internal Medicine 183, no. 4 (April 2023): 350–359.
- February 2021
- Tutorial
Assessing Prediction Accuracy of Machine Learning Models
By: Michael Toffel and Natalie Epstein
This video describes how to assess the accuracy of machine learning prediction models, primarily in the context of machine learning models that predict binary outcomes, such as logistic regression, random forest, or nearest neighbor models. After introducing and... View Details
- Article
Designing Social Networks: Joint Tasks and the Formation and Endurance of Network Ties
By: Sharique Hasan and Rembrand Koning
Can managers influence the formation of organizational networks? In this article, we evaluate the effect of joint tasks on the creation of network ties with data from a novel field experiment with 112 aspiring entrepreneurs. During the study, we randomized individuals... View Details
Keywords: Accelerators; Entrepreneur; Social Networks; Field Experiment; Entrepreneurship; Organizational Design; Networks; Social and Collaborative Networks; Social Media; Information Technology Industry; India
Hasan, Sharique, and Rembrand Koning. "Designing Social Networks: Joint Tasks and the Formation and Endurance of Network Ties." Art. 4. Journal of Organization Design 9 (2020).
- Article
Beyond the Target Customer: Social Effects in CRM Campaigns
By: Eva Ascarza, Peter Ebbes, Oded Netzer and Matthew Danielson
Customer relationship management (CRM) campaigns have traditionally focused on maximizing the profitability of the targeted customers. The authors demonstrate that in business settings characterized by network externalities, a CRM campaign that is aimed at changing the... View Details
Keywords: Social Effects; Field Experiment; Mobile; Customer Relationship Management; Network Effects; Consumer Behavior
Ascarza, Eva, Peter Ebbes, Oded Netzer, and Matthew Danielson. "Beyond the Target Customer: Social Effects in CRM Campaigns." Journal of Marketing Research (JMR) 54, no. 3 (June 2017): 347–363.
- November 2009
- Article
Finding Missing Markets (and a Disturbing Epilogue): Evidence from an Export Crop Adoption and Marketing Intervention in Kenya
By: Nava Ashraf, Xavier Gine and Dean Karlan
Farmers may grow crops for local consumption despite more profitable export options. DrumNet, a Kenyan NGO that helps small farmers adopt and market export crops, conducted a randomized trial to evaluate its impact. DrumNet services increased production of export crops... View Details
Keywords: Export Crop; Field Experiment; Food Safety Standards; Plant-Based Agribusiness; Trade; Profit; Marketing; Standards; Failure; Non-Governmental Organizations; Agriculture and Agribusiness Industry; Kenya; European Union
Ashraf, Nava, Xavier Gine, and Dean Karlan. "Finding Missing Markets (and a Disturbing Epilogue): Evidence from an Export Crop Adoption and Marketing Intervention in Kenya." American Journal of Agricultural Economics 91, no. 4 (November 2009): 973–990.
- 2023
- Working Paper
Life After Death: A Field Experiment with Small Businesses on Information Frictions, Stigma, and Bankruptcy
By: Shai Benjamin Bernstein, Emanuele Colonnelli, Mitchell Hoffman and Benjamin Iverson
In a randomized control trial (RCT) with U.S. small businesses, we document that a large share of firms are not well-informed about bankruptcy. Many assume that bankruptcy necessarily entails the death of a business and do not know about Chapter 11 bankruptcy, where... View Details
Bernstein, Shai Benjamin, Emanuele Colonnelli, Mitchell Hoffman, and Benjamin Iverson. "Life After Death: A Field Experiment with Small Businesses on Information Frictions, Stigma, and Bankruptcy." NBER Working Paper Series, No. 30933, February 2023.
- June 2022 (Revised January 2025)
- Technical Note
Causal Inference
This note provides an overview of causal inference for an introductory data science course. First, the note discusses observational studies and confounding variables. Next the note describes how randomized experiments can be used to account for the effect of... View Details
Keywords: Causal Inference; Causality; Experiment; Experimental Design; Data Science; Analytics and Data Science
Bojinov, Iavor I., Michael Parzen, and Paul Hamilton. "Causal Inference." Harvard Business School Technical Note 622-111, June 2022. (Revised January 2025.)
- 2024
- Working Paper
Do Information Frictions and Corruption Perceptions Kill Competition? A Field Experiment on Public Procurement in Uganda
By: Emanuele Colonnelli, Francesco Loiacono, Edwin Muhumuza and Edoardo Teso
We study whether information frictions and corruption perceptions deter firms from doing business with the government. We conduct two nationwide randomized controlled trials (RCTs) in collaboration with the national public procurement supervisory and anti-corruption... View Details
Keywords: Knowledge Use and Leverage; Government and Politics; Crime and Corruption; Trust; Perception; Business and Government Relations
Colonnelli, Emanuele, Francesco Loiacono, Edwin Muhumuza, and Edoardo Teso. "Do Information Frictions and Corruption Perceptions Kill Competition? A Field Experiment on Public Procurement in Uganda." NBER Working Paper Series, No. 32170, February 2024.
- March 2019
- Article
When Does Advice Impact Startup Performance?
By: Aaron Chatterji, Solène Delecourt, Sharique Hasan and Rembrand Koning
Why do some entrepreneurs thrive while others fail? We explore whether the advice entrepreneurs receive about managing their employees influences their startup's performance. We conducted a randomized field experiment in India with 100 high-growth technology firms... View Details
Keywords: Entrepreneurial Management; Field Experiment; Peer Effects; Entrepreneurial Ecosystems; Advice; Management Style; Management Practices and Processes; Knowledge Dissemination; Entrepreneurship; Performance; India
Chatterji, Aaron, Solène Delecourt, Sharique Hasan, and Rembrand Koning. "When Does Advice Impact Startup Performance?" Strategic Management Journal 40, no. 3 (March 2019): 331–356.
- 2023
- Working Paper
Personalized Game Design for Improved User Retention and Monetization in Freemium Games
By: Eva Ascarza, Oded Netzer and Julian Runge
One of the most crucial aspects and significant levers that gaming companies possess in designing
digital games is setting the level of difficulty, which essentially regulates the user’s ability to
progress within the game. This aspect is particularly significant in... View Details
Keywords: Freemium; Retention/churn; Field Experiment; Field Experiments; Gaming; Gaming Industry; Mobile App; Mobile App Industry; Monetization; Monetization Strategy; Games, Gaming, and Gambling; Mobile and Wireless Technology; Customers; Retention; Product Design; Strategy
Ascarza, Eva, Oded Netzer, and Julian Runge. "Personalized Game Design for Improved User Retention and Monetization in Freemium Games." Harvard Business School Working Paper, No. 21-062, November 2020. (Revised December 2023.)
- September 2018
- Article
Discretionary Task Ordering: Queue Management in Radiological Services
By: Maria Ibanez, Jonathan R. Clark, Robert S. Huckman and Bradley R. Staats
Work-scheduling research typically prescribes task sequences implemented by managers. Yet employees often have discretion to deviate from their prescribed sequence. Using data from 2.4 million radiological diagnoses, we find that doctors prioritize similar tasks... View Details
Keywords: Discretion; Scheduling; Queue; Healthcare; Learning; Experience; Decentralization; Operations; Service Operations; Service Delivery; Performance; Performance Effectiveness; Performance Efficiency; Performance Improvement; Performance Productivity; Decisions; Time Management; Cost vs Benefits; Health Industry
Ibanez, Maria, Jonathan R. Clark, Robert S. Huckman, and Bradley R. Staats. "Discretionary Task Ordering: Queue Management in Radiological Services." Management Science 64, no. 9 (September 2018): 4389–4407. (Working paper available here. Winner of the 2017 Best Paper Competition of the POMS College of Healthcare Operations Management. Featured in Forbes, Quartz, and Inc.)
- May–June 2019
- Article
U-Shaped Conformity in Online Social Networks
By: Monic Sun, Michael Zhang and Feng Zhu
We explore how people balance their needs to belong and to be different from their friends by studying their choices of a virtual-house wall color on a leading Chinese social-networking site. The setting enables us to randomize both the popular color and the adoption... View Details
Keywords: Conformity; Normative Social Influence; Social Networks; Field Experiment; Social and Collaborative Networks; Behavior; Attitudes; Social Media
Sun, Monic, Michael Zhang, and Feng Zhu. "U-Shaped Conformity in Online Social Networks." Marketing Science 38, no. 3 (May–June 2019): 461–480.
- February 2018
- Article
Retention Futility: Targeting High-Risk Customers Might Be Ineffective.
By: Eva Ascarza
Companies in a variety of sectors are increasingly managing customer churn proactively, generally by detecting customers at the highest risk of churning and targeting retention efforts towards them. While there is a vast literature on developing churn prediction models... View Details
Keywords: Retention/churn; Proactive Churn Management; Field Experiments; Heterogeneous Treatment Effect; Machine Learning; Customer Relationship Management; Risk Management
Ascarza, Eva. "Retention Futility: Targeting High-Risk Customers Might Be Ineffective." Journal of Marketing Research (JMR) 55, no. 1 (February 2018): 80–98.
- 2017
- Working Paper
Discretionary Task Ordering: Queue Management in Radiological Services
By: Maria Ibanez, Jonathan R. Clark, Robert S. Huckman and Bradley R. Staats
Work scheduling research typically prescribes task sequences implemented by managers. Yet employees often have discretion to deviate from their prescribed sequence. Using data from 2.4 million radiological diagnoses, we find that doctors prioritize similar tasks... View Details
Keywords: Discretion; Scheduling; Queue; Healthcare; Learning; Experience; Decentralization; Delegation; Behavioral Operations; Operations; Service Operations; Service Delivery; Performance; Performance Effectiveness; Performance Efficiency; Performance Improvement; Performance Productivity; Decisions; Time Management; Cost vs Benefits; Health Industry
Ibanez, Maria, Jonathan R. Clark, Robert S. Huckman, and Bradley R. Staats. "Discretionary Task Ordering: Queue Management in Radiological Services." Harvard Business School Working Paper, No. 16-051, October 2015. (Revised March 2017.)
- March 2022
- Article
Sensitivity Analysis of Agent-based Models: A New Protocol
By: Emanuele Borgonovo, Marco Pangallo, Jan Rivkin, Leonardo Rizzo and Nicolaj Siggelkow
Agent-based models (ABMs) are increasingly used in the management sciences. Though useful, ABMs are often critiqued: it is hard to discern why they produce the results they do and whether other assumptions would yield similar results. To help researchers address such... View Details
Keywords: Agent-based Modeling; Sensitivity Analysis; Design Of Experiments; Total Order Sensitivity Indices; Organizations; Behavior; Decision Making; Mathematical Methods
Borgonovo, Emanuele, Marco Pangallo, Jan Rivkin, Leonardo Rizzo, and Nicolaj Siggelkow. "Sensitivity Analysis of Agent-based Models: A New Protocol." Computational and Mathematical Organization Theory 28, no. 1 (March 2022): 52–94.
- 2024
- Working Paper
Anytime-Valid Inference in Linear Models and Regression-Adjusted Causal Inference
By: Michael Lindon, Dae Woong Ham, Martin Tingley and Iavor I. Bojinov
Linear regression adjustment is commonly used to analyze randomized controlled experiments due to its efficiency and robustness against model misspecification. Current testing and interval estimation procedures leverage the asymptotic distribution of such estimators to... View Details
Lindon, Michael, Dae Woong Ham, Martin Tingley, and Iavor I. Bojinov. "Anytime-Valid Inference in Linear Models and Regression-Adjusted Causal Inference." Harvard Business School Working Paper, No. 24-060, March 2024.