Publications

June 2023
Simulation

Artea Dashboard and Targeting Policy Evaluation

By: Ayelet Israeli and Eva Ascarza

Companies deploy A/B experiments to gain valuable insights about their customers in order to answer strategic business problems. In marketing, A/B tests are often used to evaluate marketing interventions intended to generate incremental outcomes for the firm. The Artea... View Details

Teaching Interest

Overview

Paul is primarily interested in teaching data science to management students through the case method. This includes technical topics (programming and statistics) as well as higher-level management issues (digital transformation, data governance, etc.) As a research... View Details

Keywords: A/B Testing; AI; AI Algorithms; AI Creativity; Algorithm; Algorithm Bias; Algorithmic Bias; Algorithmic Fairness; Algorithms; Analytics; Application Program Interface; Artificial Intelligence; Causality; Causal Inference; Computing; Computers; Data Analysis; Data Analytics; Data Architecture; Data As A Service; Data Centers; Data Governance; Data Labeling; Data Management; Data Manipulation; Data Mining; Data Ownership; Data Privacy; Data Protection; Data Science; Data Science And Analytics Management; Data Scientists; Data Security; Data Sharing; Data Strategy; Data Visualization; Database; Data-driven Decision-making; Data-driven Management; Data-driven Operations; Datathon; Economics Of AI; Economics Of Innovation; Economics Of Information System; Economics Of Science; Forecast; Forecast Accuracy; Forecasting; Forecasting And Prediction; Information Technology; Machine Learning; Machine Learning Models; Prediction; Prediction Error; Predictive Analytics; Predictive Models; Analysis; AI and Machine Learning; Analytics and Data Science; Applications and Software; Digital Transformation; Information Management; Digital Strategy; Technology Adoption

September 2020 (Revised February 2024)
Teaching Note

Artea (A), (B), (C), and (D): Designing Targeting Strategies

By: Eva Ascarza and Ayelet Israeli

Teaching Note for HBS No. 521-021,521-022,521-037,521-043. This collection of exercises aims to teach students about 1)Targeting Policies; and 2)Algorithmic bias in marketing—implications, causes, and possible solutions. Part (A) focuses on A/B testing analysis and... View Details

Keywords: Targeted Advertising; Targeting; Race; Gender; Diversity; Marketing; Customer Relationship Management; Prejudice and Bias; Analytics and Data Science; Retail Industry; Apparel and Accessories Industry; Technology Industry; United States

February 2024
Module Note

Data-Driven Marketing in Retail Markets

By: Ayelet Israeli

This note describes an eight-class sessions module on data-driven marketing in retail markets. The module aims to familiarize students with core concepts of data-driven marketing in retail, including exploring the opportunities and challenges, adopting best practices,... View Details

26 Apr 2023
In Practice

Is AI Coming for Your Job?

generate content that perpetuates existing biases. When we train these models at scale based on existing data, if the underlying data included biased information, the result is also likely to include that bias unless we intervene. One... View Details

Keywords: by Kristen Senz; Technology

March 2019
Case

Wattpad

By: John Deighton and Leora Kornfeld

How to run a platform to match four million writers of stories to 75 million readers? Use data science. Make money by doing deals with television and filmmakers and book publishers. The case describes the challenges of matching readers to stories and of helping writers... View Details

Educators

08 May 2023
Research & Ideas

How Trump’s Anti-Immigrant Rhetoric Crushed Crowdfunding for Minority Entrepreneurs

builds upon previous research on “systemic racial bias in entrepreneurial finance,” illustrating a “more direct” connection between shifts in public attitudes and struggles experienced by minority creators in raising money for new... View Details

Keywords: by Scott Van Voorhis

Forthcoming
Article

Human-Algorithm Collaboration with Private Information: Naïve Advice Weighting Behavior and Mitigation

By: Maya Balakrishnan, Kris Ferreira and Jordan Tong

Even if algorithms make better predictions than humans on average, humans may sometimes have private information which an algorithm does not have access to that can improve performance. How can we help humans effectively use and adjust recommendations made by... View Details

Keywords: AI and Machine Learning; Analytics and Data Science; Forecasting and Prediction; Digital Marketing

2024
Working Paper

Warnings and Endorsements: Improving Human-AI Collaboration Under Covariate Shift

By: Matthew DosSantos DiSorbo and Kris Ferreira

Problem definition: While artificial intelligence (AI) algorithms may perform well on data that are representative of the training set (inliers), they may err when extrapolating on non-representative data (outliers). These outliers often originate from covariate shift,... View Details

Keywords: AI and Machine Learning; Decision Choices and Conditions

08 May 2018
First Look

First Look at New Research and Ideas, May 8, 2018

unexpected networking opportunities, generating a tight community of German businesspeople in India. Publisher's link: https://www.hbs.edu/faculty/Pages/item.aspx?num=54465 How Scheduling Can Bias Quality Assessment: Evidence from Food... View Details

Keywords: Sean Silverthorne

September 2020 (Revised July 2022)
Supplement

Spreadsheet Supplement to Artea (B) and (C)

By: Eva Ascarza and Ayelet Israeli

Spreadsheet Supplement to "Artea (B): Including Customer-level Demographic Data" and "Artea (C): Potential Discrimination through Algorithmic Targeting" View Details

Keywords: Gender; Race; Diversity; Marketing; Customer Relationship Management; Demographics; Prejudice and Bias; Retail Industry; Apparel and Accessories Industry; Technology Industry; United States

November, 2016
Article

Fixing Discrimination in Online Marketplaces

By: Ray Fisman and Michael Luca

Online marketplaces such as eBay, Uber, and Airbnb have the potential to reduce racial, gender, and other forms of bias that affect the off-line world. And in the early days of Internet commerce, the relative anonymity of transactions did make it harder for... View Details

Keywords: Prejudice and Bias; Digital Platforms; Internet and the Web; Race; Gender

Find at Harvard

Register to Read

March 2017 (Revised September 2017)
Case

Facebook Fake News in the Post-Truth World

By: John R. Wells and Carole A. Winkler

In January 2017, Mark Zuckerberg, founder and CEO of Facebook, was surrounded by controversy. The election of Donald Trump as the next president of the United States in November 2016 had triggered a national storm of protests, and many attributed Trump’s victory to... View Details

Keywords: Facebook; Fake News; Mark Zuckerberg; Donald Trump; Algorithms; Social Networks; Partisanship; Social Media; App Development; Instagram; WhatsApp; Smartphone; Silicon Valley; Office Space; Digital Strategy; Democracy; Entry Barriers; Online Platforms; Controversy; Tencent; Agility; Social Networking; Gaming; Gaming Industry; Computer Games; Mobile Gaming; Messaging; Monetization Strategy; Advertising; Digital Marketing; Business Ventures; Acquisition; Mergers and Acquisitions; Business Growth and Maturation; Business Headquarters; Business Organization; For-Profit Firms; Trends; Communication; Communication Technology; Forms of Communication; Interactive Communication; Interpersonal Communication; Talent and Talent Management; Crime and Corruption; Voting; Demographics; Entertainment; Games, Gaming, and Gambling; Moral Sensibility; Values and Beliefs; Initial Public Offering; Profit; Revenue; Geography; Geographic Location; Global Range; Local Range; Country; Cross-Cultural and Cross-Border Issues; Globalized Firms and Management; Globalized Markets and Industries; Governing Rules, Regulations, and Reforms; Government and Politics; International Relations; National Security; Political Elections; Business History; Recruitment; Selection and Staffing; Information Management; Information Publishing; News; Newspapers; Innovation and Management; Innovation Strategy; Technological Innovation; Knowledge Dissemination; Human Capital; Law; Leadership Development; Leadership Style; Leading Change; Business or Company Management; Crisis Management; Goals and Objectives; Growth and Development Strategy; Growth Management; Management Practices and Processes; Management Style; Management Systems; Management Teams; Managerial Roles; Marketing Channels; Social Marketing; Network Effects; Market Entry and Exit; Digital Platforms; Marketplace Matching; Industry Growth; Industry Structures; Monopoly; Media; Product Development; Service Delivery; Corporate Social Responsibility and Impact; Mission and Purpose; Organizational Change and Adaptation; Organizational Culture; Organizational Structure; Public Ownership; Problems and Challenges; Business and Community Relations; Business and Government Relations; Groups and Teams; Networks; Rank and Position; Opportunities; Behavior; Emotions; Identity; Power and Influence; Prejudice and Bias; Reputation; Social and Collaborative Networks; Status and Position; Trust; Society; Civil Society or Community; Culture; Public Opinion; Social Issues; Societal Protocols; Strategy; Adaptation; Business Strategy; Commercialization; Competition; Competitive Advantage; Competitive Strategy; Corporate Strategy; Customization and Personalization; Diversification; Expansion; Horizontal Integration; Segmentation; Information Technology; Internet and the Web; Mobile and Wireless Technology; Internet and the Web; Applications and Software; Information Infrastructure; Digital Platforms; Internet and the Web; Mobile and Wireless Technology; Valuation; Advertising Industry; Communications Industry; Entertainment and Recreation Industry; Information Industry; Information Technology Industry; Journalism and News Industry; Media and Broadcasting Industry; Service Industry; Technology Industry; Telecommunications Industry; Video Game Industry; United States; California; Sunnyvale; Russia

Educators

06 Oct 2015
First Look

October 6, 2015

themselves and the charity, they respond very similarly to self risk and charity risk. By contrast, when their decisions force tradeoffs between money for themselves and the charity, participants act more averse to charity risk and less averse to self risk. These... View Details

Keywords: Sean Silverthorne

September 2019 (Revised September 2019)
Case

Facebook Fake News in the Post-Truth World

By: John R. Wells, Carole A. Winkler and Benjamin Weinstock

In August 2019, Mark Zuckerberg, founder and CEO of Facebook, was surrounded by controversy. The first major storm of protest followed the surprise election of Donald Trump as President of the United States on November 8, 2016; many put the blame at the door of fake... View Details

Keywords: Facebook; Fake News; Mark Zuckerberg; Donald Trump; Algorithms; Social Networks; Partisanship; Social Media; App Development; Instagram; WhatsApp; Smartphone; Silicon Valley; Office Space; Digital Strategy; Democracy; Entry Barriers; Online Platforms; Controversy; Tencent; Agility; Social Networking; Gaming; Gaming Industry; Computer Games; Mobile Gaming; Messaging; Monetization Strategy; Advertising; Digital Marketing; Business Ventures; Acquisition; Mergers and Acquisitions; Business Growth and Maturation; Business Headquarters; Business Organization; For-Profit Firms; Trends; Communication; Communication Technology; Forms of Communication; Interactive Communication; Interpersonal Communication; Talent and Talent Management; Crime and Corruption; Voting; Demographics; Entertainment; Games, Gaming, and Gambling; Moral Sensibility; Values and Beliefs; Initial Public Offering; Profit; Revenue; Geography; Geographic Location; Global Range; Local Range; Country; Cross-Cultural and Cross-Border Issues; Globalized Firms and Management; Globalized Markets and Industries; Governing Rules, Regulations, and Reforms; Government and Politics; International Relations; National Security; Political Elections; Business History; Recruitment; Selection and Staffing; Information Management; Information Publishing; News; Newspapers; Innovation and Management; Innovation Strategy; Technological Innovation; Knowledge Dissemination; Human Capital; Law; Leadership Development; Leadership Style; Leading Change; Business or Company Management; Crisis Management; Goals and Objectives; Growth and Development Strategy; Growth Management; Management Practices and Processes; Management Style; Management Systems; Management Teams; Managerial Roles; Marketing Channels; Social Marketing; Network Effects; Market Entry and Exit; Digital Platforms; Marketplace Matching; Industry Growth; Industry Structures; Monopoly; Media; Product Development; Service Delivery; Corporate Social Responsibility and Impact; Mission and Purpose; Organizational Change and Adaptation; Organizational Culture; Organizational Structure; Public Ownership; Problems and Challenges; Business and Community Relations; Business and Government Relations; Groups and Teams; Networks; Rank and Position; Opportunities; Behavior; Emotions; Identity; Power and Influence; Prejudice and Bias; Reputation; Social and Collaborative Networks; Status and Position; Trust; Society; Civil Society or Community; Culture; Public Opinion; Social Issues; Societal Protocols; Strategy; Adaptation; Business Strategy; Commercialization; Competition; Competitive Advantage; Competitive Strategy; Corporate Strategy; Customization and Personalization; Diversification; Expansion; Horizontal Integration; Segmentation; Information Technology; Internet and the Web; Mobile and Wireless Technology; Applications and Software; Information Infrastructure; Valuation; Advertising Industry; Communications Industry; Entertainment and Recreation Industry; Information Industry; Information Technology Industry; Journalism and News Industry; Media and Broadcasting Industry; Service Industry; Technology Industry; Telecommunications Industry; Video Game Industry; United States; California; Sunnyvale; Russia

Educators

October–December 2022
Article

Achieving Reliable Causal Inference with Data-Mined Variables: A Random Forest Approach to the Measurement Error Problem

By: Mochen Yang, Edward McFowland III, Gordon Burtch and Gediminas Adomavicius

Combining machine learning with econometric analysis is becoming increasingly prevalent in both research and practice. A common empirical strategy involves the application of predictive modeling techniques to "mine" variables of interest from available data, followed... View Details

Keywords: Machine Learning; Econometric Analysis; Instrumental Variable; Random Forest; Causal Inference; AI and Machine Learning; Forecasting and Prediction

Find at Harvard

Register to Read

2023
Working Paper

Insufficiently Justified Disparate Impact: A New Criterion for Subgroup Fairness

By: Neil Menghani, Edward McFowland III and Daniel B. Neill

In this paper, we develop a new criterion, "insufficiently justified disparate impact" (IJDI), for assessing whether recommendations (binarized predictions) made by an algorithmic decision support tool are fair. Our novel, utility-based IJDI criterion evaluates false... View Details

Keywords: AI and Machine Learning; Forecasting and Prediction; Prejudice and Bias