Developing a Modern Retail Forecasting System: People and Processes

In the first installment of this three-part series, as technical leads in the Data Science/AI team at U.S. retailer Target, we described the overall architecture and design of a demand forecasting system capable of efficiently generating the nearly one billion weekly forecasts required for Target's operations. In this second article, we recount some of the lessons learned in the process of developing and implementing the forecasting system. As was the case with the system architecture in the first article, its location at the intersection of statistics/mathematics, software engineering, and business practice makes the actual process of developing the forecasting system especially challenging. Here we set out the challenges we encountered, along with the steps we took to address them, in the form of patterns-rules that describe problems and how they may be resolved, along with trade-offs that should be taken into account when doing so.