Publications

Article

Fast Subset Scan for Multivariate Spatial Biosurveillance

By: Daniel B. Neill, Edward McFowland III and Huanian Zheng

We present new subset scan methods for multivariate event detection in massive space-time datasets. We extend the recently proposed 'fast subset scan' framework from univariate to multivariate data, enabling computationally efficient detection of irregular space-time... View Details

Keywords: Algorithms; Disease Surveillance; Event Detection; Scan Statistics; Spatial Scan

2022
Article

Nonparametric Subset Scanning for Detection of Heteroscedasticity

By: Charles R. Doss and Edward McFowland III

We propose Heteroscedastic Subset Scan (HSS), a novel method for identifying covariates that are responsible for violations of the homoscedasticity assumption in regression settings. Viewing the problem as one of anomalous pattern detection, we use subset scanning... View Details

Keywords: Scan Statistics; Anomaly Detection; Regression; Model Diagnostics

Article

Fast Generalized Subset Scan for Anomalous Pattern Detection

By: Edward McFowland III, Skyler Speakman and Daniel B. Neill

We propose Fast Generalized Subset Scan (FGSS), a new method for detecting anomalous patterns in general categorical data sets. We frame the pattern detection problem as a search over subsets of data records and attributes, maximizing a nonparametric scan statistic... View Details

Keywords: Pattern Detection; Anomaly Detection; Knowledge Discovery; Bayesian Networks; Scan Statistics; Analytics and Data Science

January 2024
Article

Subset Scanning for Multi-Trait Analysis Using GWAS Summary Statistics

By: Rui Cao, Evan Olawsky, Edward McFowland III, Erin Marcotte, Logan Spector and Tianzhong Yang

Multi-trait analysis has been shown to have greater statistical power than single-trait analysis. Most of the existing multi-trait analysis methods only work with a limited number of traits and usually prioritize high statistical power over identifying relevant traits,... View Details

Keywords: Mathematical Methods; Health Disorders

November 2021
Article

Gaussian Process Subset Scanning for Anomalous Pattern Detection in Non-iid Data

By: William Herlands, Edward McFowland III, Andrew Gordon Wilson and Daniel B. Neill

Identifying anomalous patterns in real-world data is essential for understanding where, when, and how systems deviate from their expected dynamics. Yet methods that separately consider the anomalousness of each individual data point have low detection power for subtle,... View Details

Keywords: Pattern Detection; Subset Scanning; Gaussian Processes; Mathematical Methods

Article

Detecting Adversarial Attacks via Subset Scanning of Autoencoder Activations and Reconstruction Error

By: Celia Cintas, Skyler Speakman, Victor Akinwande, William Ogallo, Komminist Weldemariam, Srihari Sridharan and Edward McFowland III

Reliably detecting attacks in a given set of inputs is of high practical relevance because of the vulnerability of neural networks to adversarial examples. These altered inputs create a security risk in applications with real-world consequences, such as self-driving... View Details

Keywords: Autoencoder Networks; Pattern Detection; Subset Scanning; Computer Vision; Statistical Methods And Machine Learning; Machine Learning; Deep Learning; Data Mining; Big Data; Large-scale Systems; Mathematical Methods; Analytics and Data Science

2015
Article

Scalable Detection of Anomalous Patterns With Connectivity Constraints

By: Skyler Speakman, Edward McFowland III and Daniel B. Neill

We present GraphScan, a novel method for detecting arbitrarily shaped connected clusters in graph or network data. Given a graph structure, data observed at each node, and a score function defining the anomalousness of a set of nodes, GraphScan can efficiently and... View Details

Keywords: Biosurveillance; Event Detection; Graph Mining; Scan Statistics; Spatial Scan Statistic

2011
Article

Scalable Detection of Anomalous Patterns With Connectivity Constraints

By: Skyler Speakman, Edward McFowland III and Daniel B. Neill

We present GraphScan, a novel method for detecting arbitrarily shaped connected clusters in graph or network data. Given a graph structure, data observed at each node, and a score function defining the anomalousness of a set of nodes, GraphScan can efficiently and... View Details

Keywords: Biosurveillance; Event Detection; Graph Mining; Scan Statistics; Spatial Scan Statistic

Web

Statistical & Data Services - Research Computing Services

About Us Statistical & Data Services 4ms Research Computing Services (RCS) provides statistical and data services for the HBS research community, including consultations and expertise in: Advanced View Details

2023
Working Paper

Efficient Discovery of Heterogeneous Quantile Treatment Effects in Randomized Experiments via Anomalous Pattern Detection

By: Edward McFowland III, Sriram Somanchi and Daniel B. Neill

In the recent literature on estimating heterogeneous treatment effects, each proposed method makes its own set of restrictive assumptions about the intervention’s effects and which subpopulations to explicitly estimate. Moreover, the majority of the literature provides... View Details

Keywords: Causal Inference; Program Evaluation; Algorithms; Distributional Average Treatment Effect; Treatment Effect Subset Scan; Heterogeneous Treatment Effects