IDEAS home Printed from https://ideas.repec.org/p/arx/papers/1805.06126.html
   My bibliography  Save this paper

Market Self-Learning of Signals, Impact and Optimal Trading: Invisible Hand Inference with Free Energy

Author

Listed:
  • Igor Halperin
  • Ilya Feldshteyn

Abstract

We present a simple model of a non-equilibrium self-organizing market where asset prices are partially driven by investment decisions of a bounded-rational agent. The agent acts in a stochastic market environment driven by various exogenous "alpha" signals, agent's own actions (via market impact), and noise. Unlike traditional agent-based models, our agent aggregates all traders in the market, rather than being a representative agent. Therefore, it can be identified with a bounded-rational component of the market itself, providing a particular implementation of an Invisible Hand market mechanism. In such setting, market dynamics are modeled as a fictitious self-play of such bounded-rational market-agent in its adversarial stochastic environment. As rewards obtained by such self-playing market agent are not observed from market data, we formulate and solve a simple model of such market dynamics based on a neuroscience-inspired Bounded Rational Information Theoretic Inverse Reinforcement Learning (BRIT-IRL). This results in effective asset price dynamics with a non-linear mean reversion - which in our model is generated dynamically, rather than being postulated. We argue that our model can be used in a similar way to the Black-Litterman model. In particular, it represents, in a simple modeling framework, market views of common predictive signals, market impacts and implied optimal dynamic portfolio allocations, and can be used to assess values of private signals. Moreover, it allows one to quantify a "market-implied" optimal investment strategy, along with a measure of market rationality. Our approach is numerically light, and can be implemented using standard off-the-shelf software such as TensorFlow.

Suggested Citation

  • Igor Halperin & Ilya Feldshteyn, 2018. "Market Self-Learning of Signals, Impact and Optimal Trading: Invisible Hand Inference with Free Energy," Papers 1805.06126, arXiv.org.
  • Handle: RePEc:arx:papers:1805.06126
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/1805.06126
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Avinash K. Dixit & Robert S. Pindyck, 1994. "Investment under Uncertainty," Economics Books, Princeton University Press, edition 1, number 5474.
    2. Igor Halperin, 2018. "The QLBS Q-Learner Goes NuQLear: Fitted Q Iteration, Inverse RL, and Option Portfolios," Papers 1801.06077, arXiv.org.
    3. Poterba, James M. & Summers, Lawrence H., 1988. "Mean reversion in stock prices : Evidence and Implications," Journal of Financial Economics, Elsevier, vol. 22(1), pages 27-59, October.
    4. Frédéric Abergel & Anirban Chakraborti & Hideaki Aoyama & B.K. Chakrabarti & Asim Gosh, 2014. "Econophysics of agent-based models," Post-Print hal-01006419, HAL.
    Full references (including those not matched with items on IDEAS)

    Citations

    Citations are extracted by the CitEc Project, subscribe to its RSS feed for this item.
    as


    Cited by:

    1. A. Itkin & A. Lipton & D. Muravey, 2020. "From the Black-Karasinski to the Verhulst model to accommodate the unconventional Fed's policy," Papers 2006.11976, arXiv.org, revised Jan 2021.
    2. Matthew Dixon & Igor Halperin, 2020. "G-Learner and GIRL: Goal Based Wealth Management with Reinforcement Learning," Papers 2002.10990, arXiv.org.
    3. Jacobo Roa-Vicens & Yuanbo Wang & Virgile Mison & Yarin Gal & Ricardo Silva, 2019. "Adversarial recovery of agent rewards from latent spaces of the limit order book," Papers 1912.04242, arXiv.org.
    4. Andrey Itkin & Dmitry Muravey, 2023. "American options in time-dependent one-factor models: Semi-analytic pricing, numerical methods and ML support," Papers 2307.13870, arXiv.org.

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Yoon, Byung-Sam & Brorsen, B. Wade, 2005. "Can Multiyear Rollover Hedging Increase Mean Returns?," Journal of Agricultural and Applied Economics, Cambridge University Press, vol. 37(1), pages 65-78, April.
    2. Yasuyuki Itoh, 2007. "A Class of Gaussian Hybrid Processes for Modeling Financial Markets," Asia-Pacific Financial Markets, Springer;Japanese Association of Financial Economics and Engineering, vol. 14(3), pages 185-199, September.
    3. Brett Olsen & Jeffrey Stokes, 2015. "Is Farm Real Estate The Next Bubble?," The Journal of Real Estate Finance and Economics, Springer, vol. 50(3), pages 355-376, April.
    4. Rios, Daniel & Blanco, Gerardo & Olsina, Fernando, 2019. "Integrating Real Options Analysis with long-term electricity market models," Energy Economics, Elsevier, vol. 80(C), pages 188-205.
    5. Ben Abdallah, Skander & Lasserre, Pierre, 2016. "Asset retirement with infinitely repeated alternative replacements: Harvest age and species choice in forestry," Journal of Economic Dynamics and Control, Elsevier, vol. 70(C), pages 144-164.
    6. Oscar Gutiérrez & Francisco Ruiz-Aliseda, 2011. "Real options with unknown-date events," Annals of Finance, Springer, vol. 7(2), pages 171-198, May.
    7. Timothy Erickson & Toni M. Whited, 2000. "Measurement Error and the Relationship between Investment and q," Journal of Political Economy, University of Chicago Press, vol. 108(5), pages 1027-1057, October.
    8. Arve, Malin & Zwart, Gijsbert, 2023. "Optimal procurement and investment in new technologies under uncertainty," Journal of Economic Dynamics and Control, Elsevier, vol. 147(C).
    9. Shively, Gerald E., 2001. "Price thresholds, price volatility, and the private costs of investment in a developing country grain market," Economic Modelling, Elsevier, vol. 18(3), pages 399-414, August.
    10. Marks, Phillipa & Marks, Brian, 2007. "Spectrum Allocation, Spectrum Commons and Public Goods: the Role of the Market," MPRA Paper 6785, University Library of Munich, Germany.
    11. Stern, Nicholas, 2018. "Public economics as if time matters: Climate change and the dynamics of policy," Journal of Public Economics, Elsevier, vol. 162(C), pages 4-17.
    12. Krause, M.U., 2002. "Inter-Industry Wage Differentials and Job Flows," Discussion Paper 2002-3, Tilburg University, Center for Economic Research.
    13. Wong, Kit Pong & Yi, Long, 2013. "Irreversibility, mean reversion, and investment timing," Economic Modelling, Elsevier, vol. 30(C), pages 770-775.
    14. Pierre‐Richard Agénor, 2004. "Macroeconomic Adjustment and the Poor: Analytical Issues and Cross‐Country Evidence," Journal of Economic Surveys, Wiley Blackwell, vol. 18(3), pages 351-408, July.
    15. Atal, Vidya & Bar, Talia & Gordon, Sidartha, 2016. "Project selection: Commitment and competition," Games and Economic Behavior, Elsevier, vol. 96(C), pages 30-48.
    16. Yonggu Kim & Keeyoung Shin & Joseph Ahn & Eul-Bum Lee, 2017. "Probabilistic Cash Flow-Based Optimal Investment Timing Using Two-Color Rainbow Options Valuation for Economic Sustainability Appraisement," Sustainability, MDPI, vol. 9(10), pages 1-16, October.
    17. Bergendahl, Goran, 2005. "Models for investment in electronic commerce--financial perspectives with empirical evidence," Omega, Elsevier, vol. 33(4), pages 363-376, August.
    18. Prelipcean, Gabriela & Boscoianu, Mircea, 2019. "Aspect Regarding the Design of Active Strategies for Venture Capital Financing – the Flexible Adjustment for Romania as a Frontier Capital Market," Proceedings of the ENTRENOVA - ENTerprise REsearch InNOVAtion Conference (2019), Rovinj, Croatia, in: Proceedings of the ENTRENOVA - ENTerprise REsearch InNOVAtion Conference, Rovinj, Croatia, 12-14 September 2019, pages 187-196, IRENET - Society for Advancing Innovation and Research in Economy, Zagreb.
    19. Wong, Kit Pong, 2011. "Progressive taxation and the intensity and timing of investment," Economic Modelling, Elsevier, vol. 28(1-2), pages 100-108, January.
    20. Chahim, M. & Hartl, R.F. & Kort, P.M., 2011. "The Deterministic Impulse Control Maximum Principle in Operations Research : Necessary and Sufficient Optimality Conditions (replaces CentER DP 2011-052)," Discussion Paper 2011-133, Tilburg University, Center for Economic Research.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:1805.06126. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.