Machine learning for product choice prediction

The goal of this paper is to provide a point of empirical evidence as to how machine-learning techniques stack-up in their ability to predict consumer choices relative to traditional statistical techniques. We compare a traditional (naïve) multinomial logit to six machine-learning alternatives: learning multinomial logit, random forests, neural networks, gradient boosting, support vector machines and an ensemble learning algorithm. The comparison is done by applying these methods to beer category stock keeping unit (SKU) level panel data. Results show that machine-learning techniques tend to perform better, but not always. Ensemble learning performs best while maintaining an overall high-performance level across all SKU classes, independently of their sample size. This result builds on existing evidence about the benefits of combining multiple prediction techniques over relying on a single best performing model, as conventional wisdom would intuitively make us believe. In general, the better performance of machine learning techniques at predicting product choice should not come as a surprise. At their core, machine learning techniques are designed to augment dimensionality of models and/or scan through orders of magnitude greater model alternatives, relative to the narrower focus of traditional approaches.