Simple graphical criteria for selection bias in general-population and selected-sample treatment effects

When analyzing a non-representative sample selected from a general population, estimated average treatment effects (ATEs) can be biased relative to not only the causal ATE for the general population, but also relative to the ATE for the selected sample. When the treatment could affect selection, different individuals comprise the selected sample when the treatment is hypothetically set to different levels. Thus, defining estimands of interest in the selected sample and establishing identification criteria has been challenging. We consider ATEs in the general population and the selected sample as well the net treatment difference, which compares average potential outcomes between the counterfactual selected samples in a world with all members of the general population treated versus a world with no members treated. We provide graphical criteria for each estimand to be nonparametrically identified, which are easily assessed using a standard single-world intervention graph. Others decomposed bias relative to the general-population ATE into: (1) bias in the ATE for the (factual) selected sample; and (2) bias due to effect heterogeneity by selection status. We provide an alternative two-way decomposition using the net treatment difference, allowing each source of bias to be assessed unambiguously in a graph, even when the treatment affects selection.