IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2201.12283.html
   My bibliography  Save this paper

Predicting The Stock Trend Using News Sentiment Analysis and Technical Indicators in Spark

Author

Listed:
  • Taylan Kabbani

    (Ozyegin University
    Huawei Turkey R&D Center)

  • Fatih Enes Usta

    (Marmara University)

Abstract

Predicting the stock market trend has always been challenging since its movement is affected by many factors. Here, we approach the future trend prediction problem as a machine learning classification problem by creating tomorrow_trend feature as our label to be predicted. Different features are given to help the machine learning model predict the label of a given day; whether it is an uptrend or downtrend, those features are technical indicators generated from the stock's price history. In addition, as financial news plays a vital role in changing the investor's behavior, the overall sentiment score on a given day is created from all news released on that day and added to the model as another feature. Three different machine learning models are tested in Spark (big-data computing platform), Logistic Regression, Random Forest, and Gradient Boosting Machine. Random Forest was the best performing model with a 63.58% test accuracy.

Suggested Citation

  • Taylan Kabbani & Fatih Enes Usta, 2022. "Predicting The Stock Trend Using News Sentiment Analysis and Technical Indicators in Spark," Papers 2201.12283, arXiv.org.
    • Handle: RePEc:arx:papers:2201.12283
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2201.12283
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Fama, Eugene F, 1970. "Efficient Capital Markets: A Review of Theory and Empirical Work," Journal of Finance, American Finance Association, vol. 25(2), pages 383-417, May.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. Sushant Chari & Purva Hegde Desai & Nilesh Borde & Babu George, 2023. "Aggregate News Sentiment and Stock Market Returns in India," JRFM, MDPI, vol. 16(8), pages 1-18, August.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. David M. Ritzwoller & Joseph P. Romano, 2019. "Uncertainty in the Hot Hand Fallacy: Detecting Streaky Alternatives to Random Bernoulli Sequences," Papers 1908.01406, arXiv.org, revised Apr 2021.
    2. Shazia Ghani, 2011. "A re-visit to Minsky after 2007 financial meltdown," Post-Print halshs-01027435, HAL.
    3. Steininger, Lea & Hesse, Casimir, 2024. "Buying into new ideas: The ECB’s evolving justification of unlimited liquidity," Department of Economics Working Paper Series 357, WU Vienna University of Economics and Business.
    4. Christiane Goodfellow & Dirk Schiereck & Steffen Wippler, 2013. "Are behavioural finance equity funds a superior investment? A note on fund performance and market efficiency," Journal of Asset Management, Palgrave Macmillan, vol. 14(2), pages 111-119, April.
    5. Cagli, Efe Caglar & Taskin, Dilvin & Evrim Mandaci, Pınar, 2019. "The short- and long-run efficiency of energy, precious metals, and base metals markets: Evidence from the exponential smooth transition autoregressive models," Energy Economics, Elsevier, vol. 84(C).
    6. Andrew Weinbach & Rodney J. Paul, 2009. "National television coverage and the behavioural bias of bettors: the American college football totals market," International Gambling Studies, Taylor & Francis Journals, vol. 9(1), pages 55-66, April.
    7. Plantinga, Andrew J. & Provencher, Bill, 2001. "Internal Consistency In Models Of Optimal Resource Use Under Uncertainty," 2001 Annual meeting, August 5-8, Chicago, IL 20712, American Agricultural Economics Association (New Name 2008: Agricultural and Applied Economics Association).
    8. Growitsch Christian & Nepal Rabindra & Stronzik Marcus, 2015. "Price Convergence and Information Efficiency in German Natural Gas Markets," German Economic Review, De Gruyter, vol. 16(1), pages 87-103, February.
    9. Oxelheim, Lars & Rafferty, Michael, 2005. "On the static efficiency of secondary bond markets," Journal of Multinational Financial Management, Elsevier, vol. 15(2), pages 117-135, April.
    10. Baoqiang Zhan & Shu Zhang & Helen S. Du & Xiaoguang Yang, 2022. "Exploring Statistical Arbitrage Opportunities Using Machine Learning Strategy," Computational Economics, Springer;Society for Computational Economics, vol. 60(3), pages 861-882, October.
    11. Shi, Huai-Long & Zhou, Wei-Xing, 2022. "Factor volatility spillover and its implications on factor premia," Journal of International Financial Markets, Institutions and Money, Elsevier, vol. 80(C).
    12. Gaio, Luiz Eduardo & Stefanelli, Nelson Oliveira & Pimenta, Tabajara & Bonacim, Carlos Alberto Grespan & Gatsios, Rafael Confetti, 2022. "The impact of the Russia-Ukraine conflict on market efficiency: Evidence for the developed stock market," Finance Research Letters, Elsevier, vol. 50(C).
    13. Anastasios Evgenidis & Stephanos Papadamou, 2021. "The impact of unconventional monetary policy in the euro area. Structural and scenario analysis from a Bayesian VAR," International Journal of Finance & Economics, John Wiley & Sons, Ltd., vol. 26(4), pages 5684-5703, October.
    14. Nuruddeen Usman & Kodili Nwanneka & Nduka, 2023. "Announcement Effect of COVID-19 on Cryptocurrencies," Asian Economics Letters, Asia-Pacific Applied Economics Association, vol. 3(3), pages 1-4.
    15. Olayemi O Adu & Blessing O Idakwoji, 2024. "Commodity Market Efficiency - New Evidence From the Russia-Ukraine War," Energy RESEARCH LETTERS, Asia-Pacific Applied Economics Association, vol. 5(2), pages 1-6.
    16. Tihana Škrinjarić, 2019. "Time Varying Spillovers between the Online Search Volume and Stock Returns: Case of CESEE Markets," IJFS, MDPI, vol. 7(4), pages 1-30, October.
    17. Carol Alexander & Anca Dimitriu, 2003. "Equity Indexing: Conitegration and Stock Price Dispersion: A Regime Switiching Approach to market Efficiency," ICMA Centre Discussion Papers in Finance icma-dp2003-02, Henley Business School, University of Reading.
    18. Robert C. Merton, 2006. "Paul Samuelson and Financial Economics," The American Economist, Sage Publications, vol. 50(2), pages 9-31, October.
    19. Butt, Prof. Khursheed A & Pandow, Bilal Ahmad, 2013. "An analysis into the Stock Selectivity skill of Indian Fund Managers," MPRA Paper 83500, University Library of Munich, Germany, revised 2013.
    20. Alagidede, Paul & Panagiotidis, Theodore, 2009. "Modelling stock returns in Africa's emerging equity markets," International Review of Financial Analysis, Elsevier, vol. 18(1-2), pages 1-11, March.

    More about this item

    NEP fields

    This paper has been announced in the following NEP Reports:

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2201.12283. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.