Omitted-variable bias and other matters in the defense of the category adjustment model: A reply to Crawford (2019)

The datasets from Duffy, Huttenlocher, Hedges, and Crawford (2010) [Psychonomic Bulletin & Review, 17(2), 224-230] were reanalyzed by Duffy and Smith (2018) [Psychonomic Bulletin & Review, 25(5), 1740-1750]. Duffy and Smith (2018) conclude that the datasets are not consistent with the category adjustment model (CAM). Crawford (2019) [Psychonomic Bulletin & Review, 26(2), 693-698] offered a reply to Duffy and Smith (2018) that is based on three main points. Crawford proposes regressions that are, in part, based on a “deviation” analysis. Crawford offers a different simulation of data and claims that the techniques employed by Duffy and Smith (2018) are not sufficiently sensitive to detect a specific relationship that is claimed to be consistent with CAM. Crawford also appeals to a figure showing that the responses appear to be biased toward the overall running mean, and presumably not toward recently viewed lines. We show that Crawford’s analysis suffers from an omitted-variable bias. Once this bias is corrected, the evidence in support of CAM disappears. When we produce a simulated dataset that is consistent with the specification suggested by Crawford, the techniques of Duffy and Smith (2018) correctly detect the true relationship. Despite the assertion otherwise, the simulated dataset that was analyzed by Crawford is not publicly available. Since the analysis of Crawford (2019) is incorrect, it remains our view that the datasets from Duffy, Huttenlocher, Hedges, and Crawford (2010) do not appear to be consistent with CAM or any Bayesian model of judgment.