Algorithms and Professionals May Disagree On Companies’ Reputations

This work examines (dis)agreement between an algorithm and experienced professionals in their determinations of prominent companies’ reputations. Across repeated trials, a sample of professionals in the fields of reputation management, public relations, and CSR/ESG investing were asked to rate 25 companies across industries on eight dimensions of reputation (e.g., trust, social responsibility, exchange of benefits), yielding a total of 2,986 discrete reputational judgments. These ratings were on the same scale as an algorithm developed by a tech startup which uses real-time social media data to quantify the public reputation of companies, allowing for a direct examination of agreement between professionals and the algorithm. I found that professionals and the algorithm significantly disagreed on the reputation of the companies. When examined in tandem, the algorithm’s reputation ratings positively predicted market performance, while professionals’ reputation judgments negatively predicted market performance. These results contribute a detailed account of human-algorithm disagreement and the manner in which professionals’ subject reputational judgments may be at odds with algorithmic tools designed to quantify reputational information.