IDEAS home Printed from https://ideas.repec.org/a/eee/intfor/v38y2022i1p35-50.html
   My bibliography  Save this article

A novel text-based framework for forecasting agricultural futures using massive online news headlines

Author

Listed:
  • Li, Jianping
  • Li, Guowen
  • Liu, Mingxi
  • Zhu, Xiaoqian
  • Wei, Lu

Abstract

The agricultural futures prices are generally considered difficult to forecast because the causes of fluctuations are incredibly complicated. We propose a text-based forecasting framework, which can effectively identify and quantify factors affecting agricultural futures based on massive online news headlines. A comprehensive list of influential factors can be formed using a text mining method called topic modeling. A new sentiment-analysis-based way is designed to quantify the factors such as the weather and policies that are important yet difficult to quantify. The proposed framework is empirically tested at forecasting soybean futures prices in the Chinese market. Testing was based on 9715 online news headlines from July 19, 2012 to July 9, 2018. The results show that the identified influential factors and sentiment-based variables are effective, and the proposed framework performs significantly better in medium-term and long-term forecasting than the benchmark model.

Suggested Citation

  • Li, Jianping & Li, Guowen & Liu, Mingxi & Zhu, Xiaoqian & Wei, Lu, 2022. "A novel text-based framework for forecasting agricultural futures using massive online news headlines," International Journal of Forecasting, Elsevier, vol. 38(1), pages 35-50.
    • Handle: RePEc:eee:intfor:v:38:y:2022:i:1:p:35-50
    DOI: 10.1016/j.ijforecast.2020.02.002
    as

    Download full text from publisher

    File URL: http://www.sciencedirect.com/science/article/pii/S0169207020300194
    Download Restriction: Full text for ScienceDirect subscribers only
    File URL: https://libkey.io/10.1016/j.ijforecast.2020.02.002?utm_source=ideas
    LibKey link: if access is restricted and if your library uses this service, LibKey will redirect you to where you can use your library subscription to access this item
    ---><---

    As the access to this document is restricted, you may want to search for a different version of it.

    References listed on IDEAS

    as
    1. Chao Li & Dermot J. Hayes, 2017. "Price Discovery on the International Soybean Futures Markets: A Threshold Co‐Integration Approach," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 37(1), pages 52-70, January.
    2. Grimmer, Justin & Stewart, Brandon M., 2013. "Text as Data: The Promise and Pitfalls of Automatic Content Analysis Methods for Political Texts," Political Analysis, Cambridge University Press, vol. 21(3), pages 267-297, July.
    3. Date, Paresh & Mamon, Rogemar & Tenyakov, Anton, 2013. "Filtering and forecasting commodity futures prices under an HMM framework," Energy Economics, Elsevier, vol. 40(C), pages 1001-1013.
    4. Yang, Ke & Tian, Fengping & Chen, Langnan & Li, Steven, 2017. "Realized volatility forecast of agricultural futures using the HAR models with bagging and combination approaches," International Review of Economics & Finance, Elsevier, vol. 49(C), pages 276-291.
    5. Wei, Lu & Li, Guowen & Zhu, Xiaoqian & Sun, Xiaolei & Li, Jianping, 2019. "Developing a hierarchical system for energy corporate risk factors based on textual risk disclosures," Energy Economics, Elsevier, vol. 80(C), pages 452-460.
    6. Zhao, Yang & Li, Jianping & Yu, Lean, 2017. "A deep learning ensemble approach for crude oil price forecasting," Energy Economics, Elsevier, vol. 66(C), pages 9-16.
    7. Aaron Smith, 2005. "Partially overlapping time series: a new model for volatility dynamics in commodity futures," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(3), pages 405-422.
    8. Yu, Feng & Xu, Xiaozhong, 2014. "A short-term load forecasting model of natural gas based on optimized genetic algorithm and improved BP neural network," Applied Energy, Elsevier, vol. 134(C), pages 102-113.
    9. Diebold, Francis X & Mariano, Roberto S, 2002. "Comparing Predictive Accuracy," Journal of Business & Economic Statistics, American Statistical Association, vol. 20(1), pages 134-144, January.
    10. Tian, Fengping & Yang, Ke & Chen, Langnan, 2017. "Realized volatility forecasting of agricultural commodity futures using the HAR model with time-varying sparsity," International Journal of Forecasting, Elsevier, vol. 33(1), pages 132-152.
    11. Luo, Jiawen & Klein, Tony & Ji, Qiang & Hou, Chenghan, 2022. "Forecasting realized volatility of agricultural commodity futures with infinite Hidden Markov HAR models," International Journal of Forecasting, Elsevier, vol. 38(1), pages 51-73.
    12. Ahumada, H. & Cornejo, M., 2016. "Forecasting food prices: The case of corn, soybeans and wheat," International Journal of Forecasting, Elsevier, vol. 32(3), pages 838-848.
    13. Wang, Huai-zhi & Li, Gang-qiang & Wang, Gui-bin & Peng, Jian-chun & Jiang, Hui & Liu, Yi-tao, 2017. "Deep learning based ensemble approach for probabilistic wind power forecasting," Applied Energy, Elsevier, vol. 188(C), pages 56-70.
    14. Richard E. Just & Gordon C. Rausser, 1981. "Commodity Price Forecasting with Large-Scale Econometric Models and the Futures Market," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 63(2), pages 197-208.
    15. Lu Wei & Guowen Li & Xiaoqian Zhu & Jianping Li, 2019. "Discovering bank risk factors from financial statements based on a new semi‐supervised text mining algorithm," Accounting and Finance, Accounting and Finance Association of Australia and New Zealand, vol. 59(3), pages 1519-1552, September.
    16. Martin T. Bohl, Pierre Siklos, Claudia Wellenreuther, 2018. "Speculative Activity and Returns to Volatility of Chinese Major Agricultural Commodity Futures," LCERPA Working Papers 0111, Laurier Centre for Economic Research and Policy Analysis, revised 30 Jan 2018.
    17. Bohl, Martin T. & Siklos, Pierre L. & Wellenreuther, Claudia, 2018. "Speculative activity and returns volatility of Chinese agricultural commodity futures," Journal of Asian Economics, Elsevier, vol. 54(C), pages 69-91.
    18. Yang Bao & Anindya Datta, 2014. "Simultaneously Discovering and Quantifying Risk Types from Textual Risk Disclosures," Management Science, INFORMS, vol. 60(6), pages 1371-1391, June.
    19. Liyan Han & Rong Liang & Ke Tang, 2013. "Cross-market soybean futures price discovery: does the Dalian Commodity Exchange affect the Chicago Board of Trade?," Quantitative Finance, Taylor & Francis Journals, vol. 13(4), pages 613-626, March.
    20. Xiaoyong Xiao & Qingsong Tian & Shuxia Hou & Chongguang Li, 2019. "Economic policy uncertainty and grain futures price volatility: evidence from China," China Agricultural Economic Review, Emerald Group Publishing Limited, vol. 11(4), pages 642-654, August.
    21. Jeff Fleming & Chris Kirby & Barbara Ostdiek, 2006. "Information, Trading, and Volatility: Evidence from Weather‐Sensitive Markets," Journal of Finance, American Finance Association, vol. 61(6), pages 2899-2930, December.
    22. Tang, Ling & Yu, Lean & Wang, Shuai & Li, Jianping & Wang, Shouyang, 2012. "A novel hybrid ensemble learning paradigm for nuclear energy consumption forecasting," Applied Energy, Elsevier, vol. 93(C), pages 432-443.
    23. Ballestra, Luca Vincenzo & Guizzardi, Andrea & Palladini, Fabio, 2019. "Forecasting and trading on the VIX futures market: A neural network approach based on open to close returns and coincident indicators," International Journal of Forecasting, Elsevier, vol. 35(4), pages 1250-1262.
    24. Michael K Adjemian & Scott H Irwin, 2018. "USDA Announcement Effects in Real-Time," American Journal of Agricultural Economics, Agricultural and Applied Economics Association, vol. 100(4), pages 1151-1171.
    25. Erdogdu, Erkan, 2007. "Electricity demand analysis using cointegration and ARIMA modelling: A case study of Turkey," Energy Policy, Elsevier, vol. 35(2), pages 1129-1146, February.
    26. Ying Jiang & Shamim Ahmed & Xiaoquan Liu, 2017. "Volatility forecasting in the Chinese commodity futures market with intraday data," Review of Quantitative Finance and Accounting, Springer, vol. 48(4), pages 1123-1173, May.
    27. Aaron Smith, 2005. "Partially overlapping time series: a new model for volatility dynamics in commodity futures," Journal of Applied Econometrics, John Wiley & Sons, Ltd., vol. 20(3), pages 405-422, March.
    28. Liu, Qingfu & An, Yunbi, 2011. "Information transmission in informationally linked markets: Evidence from US and Chinese commodity futures markets," Journal of International Money and Finance, Elsevier, vol. 30(5), pages 778-795, September.
    29. Philipp Adämmer & Martin T. Bohl, 2018. "Price discovery dynamics in European agricultural markets," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 38(5), pages 549-562, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sun, Jingyun & Zhao, Panpan & Sun, Shaolong, 2022. "A new secondary decomposition-reconstruction-ensemble approach for crude oil price forecasting," Resources Policy, Elsevier, vol. 77(C).
    2. Wen, Shigang & Li, Jianping & Huang, Chuangxia & Zhu, Xiaoqian, 2023. "Extreme risk spillovers among traditional financial and FinTech institutions: A complex network perspective," The Quarterly Review of Economics and Finance, Elsevier, vol. 88(C), pages 190-202.
    3. Rui Luo & Jinpei Liu & Piao Wang & Zhifu Tao & Huayou Chen, 2024. "A multisource data‐driven combined forecasting model based on internet search keyword screening method for interval soybean futures price," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 43(2), pages 366-390, March.
    4. Lili Pan & Lin Wang & Qianqian Feng, 2022. "A Bibliometric Analysis of Risk Management in Foreign Direct Investment: Insights and Implications," Sustainability, MDPI, vol. 14(12), pages 1-18, June.
    5. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    6. Yuyao Feng & Guowen Li & Xiaolei Sun & Jianping Li, 2022. "Identification of tourists’ dynamic risk perception—the situation in Tibet," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-13, December.
    7. Jujie Wang & Zhenzhen Zhuang & Liu Feng, 2022. "Intelligent Optimization Based Multi-Factor Deep Learning Stock Selection Model and Quantitative Trading Strategy," Mathematics, MDPI, vol. 10(4), pages 1-19, February.
    8. Wuyue An & Lin Wang & Yu‐Rong Zeng, 2023. "Text‐based soybean futures price forecasting: A two‐stage deep learning approach," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(2), pages 312-330, March.
    9. Li, Guowen & Jing, Zhongbo & Li, Jingyu & Feng, Yuyao, 2023. "Drivers of risk correlation among financial institutions: A study based on a textual risk disclosure perspective," Economic Modelling, Elsevier, vol. 128(C).
    10. Junshu Jiang & Jordan Richards & Raphael Huser & David Bolin, 2024. "The Efficient Tail Hypothesis: An Extreme Value Perspective on Market Efficiency," Papers 2408.06661, arXiv.org.
    11. Xiaoqian Zhu & Yinghui Wang & Jianping Li, 2022. "What drives reputational risk? Evidence from textual risk disclosures in financial statements," Palgrave Communications, Palgrave Macmillan, vol. 9(1), pages 1-15, December.
    12. Jiang, He & Hu, Weiqiang & Xiao, Ling & Dong, Yao, 2022. "A decomposition ensemble based deep learning approach for crude oil price forecasting," Resources Policy, Elsevier, vol. 78(C).
    13. Wuyue An & Lin Wang & Dongfeng Zhang, 2023. "Comprehensive commodity price forecasting framework using text mining methods," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(7), pages 1865-1888, November.
    14. Xu Gong & Keqin Guan & Qiyang Chen, 2022. "The role of textual analysis in oil futures price forecasting based on machine learning approach," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 42(10), pages 1987-2017, October.
    15. Hao, Jun & Feng, Qianqian & Yuan, Jiaxin & Sun, Xiaolei & Li, Jianping, 2022. "A dynamic ensemble learning with multi-objective optimization for oil prices prediction," Resources Policy, Elsevier, vol. 79(C).
    16. Li, Hengyun & Gao, Huicai & Song, Haiyan, 2023. "Tourism forecasting with granular sentiment analysis," Annals of Tourism Research, Elsevier, vol. 103(C).
    17. Xu Zhang & Xian Yang & Jianping Li & Jun Hao, 2023. "Contemporaneous and noncontemporaneous idiosyncratic risk spillovers in commodity futures markets: A novel network topology approach," Journal of Futures Markets, John Wiley & Sons, Ltd., vol. 43(6), pages 705-733, June.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Algirdas Justinas Staugaitis & Bernardas Vaznonis, 2022. "Financial Speculation Impact on Agricultural and Other Commodity Return Volatility: Implications for Sustainable Development and Food Security," Agriculture, MDPI, vol. 12(11), pages 1-27, November.
    2. Hardik A. Marfatia & Qiang Ji & Jiawen Luo, 2022. "Forecasting the volatility of agricultural commodity futures: The role of co‐volatility and oil volatility," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 41(2), pages 383-404, March.
    3. Petropoulos, Fotios & Apiletti, Daniele & Assimakopoulos, Vassilios & Babai, Mohamed Zied & Barrow, Devon K. & Ben Taieb, Souhaib & Bergmeir, Christoph & Bessa, Ricardo J. & Bijak, Jakub & Boylan, Joh, 2022. "Forecasting: theory and practice," International Journal of Forecasting, Elsevier, vol. 38(3), pages 705-871.
      • Fotios Petropoulos & Daniele Apiletti & Vassilios Assimakopoulos & Mohamed Zied Babai & Devon K. Barrow & Souhaib Ben Taieb & Christoph Bergmeir & Ricardo J. Bessa & Jakub Bijak & John E. Boylan & Jet, 2020. "Forecasting: theory and practice," Papers 2012.03854, arXiv.org, revised Jan 2022.
    4. Lu, Quanying & Li, Yuze & Chai, Jian & Wang, Shouyang, 2020. "Crude oil price analysis and forecasting: A perspective of “new triangle”," Energy Economics, Elsevier, vol. 87(C).
    5. Yongmei Fang & Bo Guan & Shangjuan Wu & Saeed Heravi, 2020. "Optimal forecast combination based on ensemble empirical mode decomposition for agricultural commodity futures prices," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 39(6), pages 877-886, September.
    6. Liu, Mingxi & Li, Guowen & Li, Jianping & Zhu, Xiaoqian & Yao, Yinhong, 2021. "Forecasting the price of Bitcoin using deep learning," Finance Research Letters, Elsevier, vol. 40(C).
    7. Li, Jianping & Feng, Yuyao & Li, Guowen & Sun, Xiaolei, 2020. "Tourism companies' risk exposures on text disclosure," Annals of Tourism Research, Elsevier, vol. 84(C).
    8. Jesus Crespo Cuaresma & Jaroslava Hlouskova & Michael Obersteiner, 2021. "Agricultural commodity price dynamics and their determinants: A comprehensive econometric approach," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 40(7), pages 1245-1273, November.
    9. Zhang, Shuai & Chen, Yong & Xiao, Jiuhong & Zhang, Wenyu & Feng, Ruijun, 2021. "Hybrid wind speed forecasting model based on multivariate data secondary decomposition approach and deep learning algorithm with attention mechanism," Renewable Energy, Elsevier, vol. 174(C), pages 688-704.
    10. Ke Yan & Yuting Dai & Meiling Xu & Yuchang Mo, 2019. "Tunnel Surface Settlement Forecasting with Ensemble Learning," Sustainability, MDPI, vol. 12(1), pages 1-11, December.
    11. Algirdas Justinas Staugaitis & Bernardas Vaznonis, 2022. "Short-Term Speculation Effects on Agricultural Commodity Returns and Volatility in the European Market Prior to and during the Pandemic," Agriculture, MDPI, vol. 12(5), pages 1-26, April.
    12. Quanbiao Shang & Teresa Serra & Philip Garcia & Mindy Mallory, 2021. "Looking under the surface: An analysis of iceberg orders in the U.S. agricultural futures markets," Agricultural Economics, International Association of Agricultural Economists, vol. 52(4), pages 679-699, July.
    13. Blum, Avinoam & Raviv, Alon, 2023. "The effects of the financial crisis and Basel III on banks’ risk disclosure: A textual analysis," Finance Research Letters, Elsevier, vol. 53(C).
    14. Gong, Xu & Lin, Boqiang, 2018. "Structural changes and out-of-sample prediction of realized range-based variance in the stock market," Physica A: Statistical Mechanics and its Applications, Elsevier, vol. 494(C), pages 27-39.
    15. Zhang, Yaojie & Lei, Likun & Wei, Yu, 2020. "Forecasting the Chinese stock market volatility with international market volatilities: The role of regime switching," The North American Journal of Economics and Finance, Elsevier, vol. 52(C).
    16. Suryanarayana, Gowri & Lago, Jesus & Geysen, Davy & Aleksiejuk, Piotr & Johansson, Christian, 2018. "Thermal load forecasting in district heating networks using deep learning and advanced feature selection methods," Energy, Elsevier, vol. 157(C), pages 141-149.
    17. Haas, Christian & Budin, Constantin & d’Arcy, Anne, 2024. "How to select oil price prediction models — The effect of statistical and financial performance metrics and sentiment scores," Energy Economics, Elsevier, vol. 133(C).
    18. Matteo Bonato & Oğuzhan Çepni & Rangan Gupta & Christian Pierdzioch, 2023. "El Niño, La Niña, and forecastability of the realized variance of agricultural commodity prices: Evidence from a machine learning approach," Journal of Forecasting, John Wiley & Sons, Ltd., vol. 42(4), pages 785-801, July.
    19. Wei, Lu & Li, Guowen & Li, Jianping & Zhu, Xiaoqian, 2019. "Bank risk aggregation with forward-looking textual risk disclosures," The North American Journal of Economics and Finance, Elsevier, vol. 50(C).
    20. Lu Wei & Chen Han & Yinhong Yao, 2022. "The Bias Analysis of Oil and Gas Companies’ Credit Ratings Based on Textual Risk Disclosures," Energies, MDPI, vol. 15(7), pages 1-12, March.

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:eee:intfor:v:38:y:2022:i:1:p:35-50. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: Catherine Liu (email available below). General contact details of provider: http://www.elsevier.com/locate/ijforecast .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.