Publications

Article

Earnings Dynamics and Measurement Error in Matched Survey and Administrative Data

By: Dean Hyslop and Wilbur Townsend

This article analyzes earnings dynamics and measurement error using a matched longitudinal sample of individuals’ survey and administrative earnings. In line with previous literature, the reported differences are characterized by both persistent and transitory factors.... View Details

Keywords: Earnings Dynamics; Measurement Error; Panel Data; Validation Study; Business Earnings; Measurement and Metrics; Forecasting and Prediction

Register to Read

Research Summary

An Uncomfortable Predictability Paradox

In predictive regressions, we test the null hypothesis that a predictor has no information about expected returns, i.e. beta equals zero. However, the literature neglects to recognize that we are testing a joint hypothesis. The maintained... View Details

11 Jul 2018
Working Paper Summaries

Channeled Attention and Stable Errors

Keywords: by Tristan Gagnon-Bartsch, Matthew Rabin, and Joshua Schwartzstein

2011
Working Paper

The Importance of Work Context in Organizational Learning from Error

By: Lucy H. MacPhail and Amy C. Edmondson

This paper examines the implications of work context for learning from errors in organizations. Prior research has shown that attitudes and behaviors related to error vary between groups within organizations but has not investigated or theorized the ways in which... View Details

Keywords: Judgments; Learning; Business Processes; Organizational Culture; Failure; Performance Improvement; Opportunities; Complexity

August 2016
Article

The Role of (Dis)similarity in (Mis)predicting Others' Preferences

By: Kate Barasz, Tami Kim and Leslie K. John

Consumers readily indicate liking options that appear dissimilar—for example, enjoying both rustic lake vacations and chic city vacations or liking both scholarly documentary films and action-packed thrillers. However, when predicting other consumers’ tastes for the... View Details

Keywords: Perceived Similarity; Prediction Error; Preference Prediction; Self-other Difference; Social Inference; Cognition and Thinking; Perception; Forecasting and Prediction

2018
Working Paper

Channeled Attention and Stable Errors -- Previous Working Version

By: Tristan Gagnon-Bartsch, Matthew Rabin and Joshua Schwartzstein

A common critique of models of mistaken beliefs is that people should recognize their error after observations they thought were unlikely. This paper develops a framework for assessing when a given error is likely to be discovered, in the sense that the error-maker... View Details

Keywords: Perception; Behavior; Theory; Situation or Environment

February 2021
Tutorial

Assessing Prediction Accuracy of Machine Learning Models

By: Michael Toffel and Natalie Epstein

This video describes how to assess the accuracy of machine learning prediction models, primarily in the context of machine learning models that predict binary outcomes, such as logistic regression, random forest, or nearest neighbor models. After introducing and... View Details

Keywords: Statistics; Experiments; Forecasting and Prediction; Performance Evaluation; AI and Machine Learning

Purchase

August 2020 (Revised September 2020)
Technical Note

Assessing Prediction Accuracy of Machine Learning Models

By: Michael W. Toffel, Natalie Epstein, Kris Ferreira and Yael Grushka-Cockayne

The note introduces a variety of methods to assess the accuracy of machine learning prediction models. The note begins by briefly introducing machine learning, overfitting, training versus test datasets, and cross validation. The following accuracy metrics and tools... View Details

Keywords: Machine Learning; Statistics; Econometric Analyses; Experimental Methods; Data Analysis; Data Analytics; Forecasting and Prediction; Analytics and Data Science; Analysis; Mathematical Methods

Educators

Purchase

July 2019
Article

I Know Why You Voted for Trump: (Over)inferring Motives Based on Choice

By: Kate Barasz, Tami Kim and Ioannis Evangelidis

People often speculate about why others make the choices they do. This paper investigates how such inferences are formed as a function of what is chosen. Specifically, when observers encounter someone else's choice (e.g., of political candidate), they use the chosen... View Details

Keywords: Self-other Difference; Social Perception; Inference-making; Preferences; Consumer Behavior; Prediction; Prediction Error; Decision Choices and Conditions; Perception; Behavior; Forecasting and Prediction

20 Jun 2016
Research & Ideas

When Predicting Other People's Preferences, You're Probably Wrong

our account suggests, people default to the belief that others have relatively narrow and homogeneous preferences, and thus predict that dissimilar items are disliked,” the researchers write in “The Role of (Dis)similarity in... View Details

Keywords: by Carmen Nobel; Retail

October–December 2022
Article

Achieving Reliable Causal Inference with Data-Mined Variables: A Random Forest Approach to the Measurement Error Problem

By: Mochen Yang, Edward McFowland III, Gordon Burtch and Gediminas Adomavicius

Combining machine learning with econometric analysis is becoming increasingly prevalent in both research and practice. A common empirical strategy involves the application of predictive modeling techniques to "mine" variables of interest from available data, followed... View Details

Keywords: Machine Learning; Econometric Analysis; Instrumental Variable; Random Forest; Causal Inference; AI and Machine Learning; Forecasting and Prediction

Register to Read

2025
Working Paper

Enhancing Treatment Effect Prediction on Privacy-Protected Data: An Honest Post-Processing Approach

By: Ta-Wei Huang and Eva Ascarza

As firms increasingly rely on customer data for personalization, concerns over privacy and regulatory compliance have grown. Local Differential Privacy (LDP) offers strong individual-level protection by injecting noise into data before collection. While... View Details

Keywords: Targeted Intervention; Conditional Average Treatment Effect Estimation; Differential Privacy; Honest Estimation; Post-processing; Analytics and Data Science; Consumer Behavior; Marketing

January 2025
Technical Note

AI vs Human: Analyzing Acceptable Error Rates Using the Confusion Matrix

By: Tsedal Neeley and Tim Englehart

This technical note introduces the confusion matrix as a foundational tool in artificial intelligence (AI) and large language models (LLMs) for assessing the performance of classification models, focusing on their reliability for decision-making. A confusion matrix... View Details

Keywords: Reliability; Confusion Matrix; AI and Machine Learning; Decision Making; Measurement and Metrics; Performance

Educators

Purchase

Combining machine learning with econometric analysis is becoming increasingly prevalent in both research and practice. A common empirical strategy involves the application of predictive modeling techniques to "mine" variables of interest from available data,... View Details

2023
Working Paper

The Limits of Algorithmic Measures of Race in Studies of Outcome Disparities

By: David S. Scharfstein and Sergey Chernenko

We show that the use of algorithms to predict race has significant limitations in measuring and understanding the sources of racial disparities in finance, economics, and other contexts. First, we derive theoretically the direction and magnitude of measurement bias in... View Details

Keywords: Racial Disparity; Paycheck Protection Program; Measurement Error; AI and Machine Learning; Race; Measurement and Metrics; Equality and Inequality; Prejudice and Bias; Forecasting and Prediction; Outcome or Result

2023
Working Paper

Nailing Prediction: Experimental Evidence on the Value of Tools in Predictive Model Development

By: Daniel Yue, Paul Hamilton and Iavor Bojinov

Predictive model development is understudied despite its centrality in modern artificial intelligence and machine learning business applications. Although prior discussions highlight advances in methods (along the dimensions of data, computing power, and algorithms)... View Details

Keywords: Analytics and Data Science

Predictive model development is understudied despite its importance to modern businesses. Although prior discussions highlight advances in methods (along the dimensions of data, computing power, and algorithms) as the primary driver of model quality, the value of... View Details

15 Aug 2016
Research & Ideas

Black Swans and Big Trends Can Ruin Anyone's Internet Prediction

how we thought about online opportunities one year after the dot-com bubble burst. In retrospect, many of my thoughts about the internet’s future evolution missed the mark. If you think history repeats itself, then these forecasting View Details

Keywords: by Thomas R. Eisenmann; Technology

2019
Working Paper

Managing Churn to Maximize Profits

By: Aurelie Lemmens and Sunil Gupta

Customer defection threatens many industries, prompting companies to deploy targeted, proactive customer retention programs and offers. A conventional approach has been to target customers either based on their predicted churn probability, or their responsiveness to a... View Details

Keywords: Churn Management; Defection Prediction; Loss Function; Stochastic Gradient Boosting; Customer Relationship Management; Consumer Behavior; Profit