Who is Tweeting? A Scoping Review of Methods to Establish Race and Ethnicity from Twitter Datasets

Background: A growing amount of health research uses social media data. Those critical of social media research often cite that it may be unrepresentative of the population. Identifying the demographics of social media users enables us to measure the representativeness. Extracting race or ethnicity from social media data can be difficult and researchers may choose from a multitude of different approaches. Methods: We present a scoping review to identify the methods used to extract race or ethnicity from Twitter datasets. We searched 16 electronic databases and carried out reference checking in order to identify relevant articles. Sifting of each record was undertaken independently by at least two researchers with any disagreement discussed. The research could be grouped by the methods applied to extract race or ethnicity. Results: From 1093 records we identified 56 that met our inclusion criteria. The majority focus on Twitter users based in the US. A range of types of data were used including Twitter profile -pictures, bios, and/or location, and the content in the tweets themselves. The methods used were wide ranging and included using manual inference, linkage to census data, commercial software, language/dialect recognition and machine learning. Not all studies evaluated their methods. Those that did found accuracy to vary from 45% to 93% with significantly lower accuracy identifying non-white race categories. There may be some ethical questions over some of the methods used, particularly using photos or dialect, as well as questions surrounding accuracy. Conclusion: There is no standard approach or guidelines for extracting race or ethnicity from Twitter or other social media. Social media researchers must use careful interpretation of race or ethnicity and not over-promise what can be achieved, as even manual screening is a subjective, imperfect method. Future research should establish the accuracy of methods to inform evidence-based best practice guidelines for social media researchers, and be guided by concerns of equity and social justice.