Determining the Number of Topics to Retain using Tools from Factor Analysis

Determining the optimal number of topics to retain in the conduct of topic modeling (TM) has received much attention over the last decade. Despite this work, issues remain regarding the best methods to use for making such determinations. Approaches involving the use of relatively simple statistics, most notably perplexity, have proven to be somewhat inconsistent. Recently, researchers have suggested the use of change in perplexity scores as a useful heuristic for determining the optimal number of topics to retain. The current study builds on this earlier work by assessing the utility of several methods borrowed from factor analysis and applied to statistics commonly used in topic modeling, including perplexity and Alpha. These new approaches are applied to several textual datasets and compared with more traditional methods for determining the number of topics to retain. Results of these analyses demonstrate that application of these methods borrowed from factor analysis does appear to be effective for identifying the number of topics to retain.