Forecasting Agri-food Consumption Using the Keyword Volume Index from Search Engine Data

Forecasting in the agri-food sector is an important topic. Accurate yield prediction improves farmers’ revenue and management stability. Agile demand forecasting helps agri-food producers to adjust their production properly. Also, appropriate price forecasting directly affect the profit of agri-food suppliers and distributors, and can be important information for decision making by policy makers. However, previous studies often use only a price variable as a predictor without critical variables such as yield or demand because collecting instant yield or demand data is difficult. Recently, with the progress of technology, performance of production yield forecasting using climate data have reached a high level. For example, Monsanto acquires Climate Corporation, a climate data research company for $930 million to maximize farmers’ yields. This event means that yield forecasting techniques have been matured.However, studies on forecasting agri-food demands are not vibrant due to lack of real-time consumer data. Presently, there are several sources of data on real-time economic activity available from private sector companies such as Google, MasterCard, Federal Express, UPS, and Intuit (Choi & Varian, 2012). Actually, many forecasting studies with significant results have used search engine data as an important indicator to improve prediction accuracy. Yahoo’s search query predicted stock market trade volume (Preis et al., 2013). Google Trends data forecasted the number of movie viewers (Hand & Judge, 2011), which is used as an important predictor and indicator of monthly consumption (Vosen & Schmidt, 2011; Kholodilin et al., 2010). In the commodity markets, automobile purchases (Choi & Varian, 2012; Carrière-Swallow & Labbé, 2013) and house prices (Wu & Brynjolfsson, 2014) can be explained by Google Trends data. However, there are few studies on forecasting using Google Trends data in the agri-food and agribusiness sectors. In this study, we examine Naver Trend of South Korea (similar to Google Trends), which is a real-time weekly index of the volume of queries that users enter into Naver. We aim to find that these search engine data improves accuracy in forecasting consumption of agri-foods. For a more detail explanation, we classify empirically agri-food items into several specific groups. Consumption data is collected from a panel survey of housewives from the Rural Development Administration in South Korea. The data includes five years of daily household food consumption records based on receipt from December 2009 through November 2014. We finally obtained 3.6 million purchase data of 732 panels continuously sustained during five years. The data set for estimation is processed on a weekly based amount of consumption by targeted agri-food items for 261 weeks. The prediction model of this study is based on the AR(1) and seasonal AR(1) model of Choi and Varian (2012). The baseline model only consists of the consumption of t period and lagged consumption of t-1 period. The AR(1) term (in the case of seasonal AR(1) and lagged consumption of t-52 period) and the test model have an additional predictor as a keyword volume index of t period. A 3-week moving average is used for the model estimation in order to smooth the short-term fluctuation of the measured value. For the evaluation of prediction models, we first examine that the test model is a significantly improved in-sample fit, and we next investigate whether the trends variables improve out-of-sample forecasting by the rolling window method, which is consistent with the method of Choi and Varian (2012). The model estimation and prediction performance evaluation is achieved by the agri-food items when considering the classified groups. The first classification measure is whether the agri-food item is a main (or single) dish or minor ingredients of a main dish. The second measure is whether the agri-food item is consumed at home or for eating out. The third measure is whether the agri-food item is a common product or a brand product. The fourth measure is whether the agri-food item has macro sales trends or no trends. The results are different by agri-food groups. The agri-food items often used for main or single dishes such as chicken, sweet pumpkin, sweet potato, and shiitake mushroom tend to show significant improvement for both in-sample estimation and out-of-sample forecast. However, agri-food items used mainly as minor ingredients such as carrot, cucumber, canned tuna, and enoki mushroom are not significant. Even if the item is a main or single dish, agri-food items frequently consumed out of the home such as roasted pork belly, fried chicken, and dumpling have no significant relationship. However, agri-food items mainly consumed at home such as most fruits are significant. Meanwhile, the significant difference between common products and brand products are not found. Rather, the more important criteria are macro trends. Agri-food items with gradual growth macro trends or long-term fluctuation have a significant relationship. For example, instant rice products, healthy food such as chicken breast and nut products, or newly released ramen products with a growing trend show a significant relationship. On the contrary, the steady selling of ramen products or dumpling products is not significant. Although the Naver Trends data for all of the agri-food items do not improve performance of consumption forecasting, the search engine data is correlated with actual consumption in several agri-food groups such as 1)the ingredients of main or single dishes, and 2)ingredients mainly consumed at home, and 3)products with macro sales trends. The first and second groups are directly linked with the searching activities for actual agri-food consumption. The third group has volatility for the estimation in the difference of steady selling products with low fluctuation. Therefore, search engine data of the agri-foods with these characteristics can be considered as an important predictor of agri-food consumption forecasting research. This study shows that consumers’ internet searching activity-related data rarely dealt with previous research in the agri-food sector, which can be a meaningful factor to predict more accurate actual consumption. We expect to extend many further relevant studies, and to help agri-food producers and distributors when they forecast the sales of their agri-products