Ayelet Israeli

Typically, past spending in a category is a positive indicator of future purchasing. In this we show that there exist categories (which we name "canary categories") in which the reverse is true -  
When customers purchase products in canary categories, it is a signal that they will not return to that store on future trips.

We label these categories canary categories, as in a canary in a coal mine. Canaries in coal mines warn miners of the danger of carbon monoxide. Canary categories warn retailers of the dangers of customer attrition due to gaps in store assortments.

Why do these categories exist? We believe that one reason is due to incomplete assortments: When customers visit a store and their preferred brand is not available, some customers buy, but are less likely to return in the future. Under this explanation, the trigger for the effect is the absence of a customer’s favorite brand from the retailer’s assortment. The implication is that canary categories can vary from retailer to retailer (even within a chain) depending upon their assortments.

We do not suggest that retailers should stop selling canary categories! These categories provide an important source of revenue and may also contribute an important source of differentiation that attracts customers. Instead, canary categories serve an important role, helping managers identify categories in which incomplete assortments are contributing to customer attrition. Our paper offers guidance on how to identify these categories.

An inherent risk of algorithmic personalization is disproportionate targeting of individuals from certain groups (or demographic characteristics such as gender or race), even when the decision maker does not intend to discriminate based on those “protected” attributes. This unintended discrimination is often caused by underlying correlations in the data between protected attributes and other observed characteristics used by the algorithm (or machine learning (ML) tool) to create predictions and target individuals optimally. Because these correlations are hidden in high dimensional data, removing protected attributes from the database does not solve the discrimination problem; instead, removing those attributes often exacerbates the problem by making it undetectable and, in some cases, even increases the bias generated by the algorithm.

We propose BEAT (Bias-Eliminating Adapted Trees) to address these issues. This approach allows decision makers to target individuals based on differences in their predicted behavior—hence capturing value from personalization—while ensuring a balanced allocation of resources across individuals, guaranteeing both group and individual fairness. Essentially, the method only extracts heterogeneity in the data that is unrelated to protected attributes. To do so, we build on the General Random Forest (GRF) framework (S. Athey et al., Ann. Stat. 47, 1148–1178 (2019)) and develop a targeting allocation that is “balanced” with respect to protected attributes.  We validate BEAT using simulations and an online experiment with N=3,146 participants. This approach can be applied to any type of allocation decision that is based on prediction algorithms, such as medical treatments, hiring decisions, product recommendations, or dynamic pricing.