IDEAS home Printed from https://ideas.repec.org/p/arx/papers/2306.06087.html
   My bibliography  Save this paper

Learning Not to Spoof

Author

Listed:
  • David Byrd

Abstract

As intelligent trading agents based on reinforcement learning (RL) gain prevalence, it becomes more important to ensure that RL agents obey laws, regulations, and human behavioral expectations. There is substantial literature concerning the aversion of obvious catastrophes like crashing a helicopter or bankrupting a trading account, but little around the avoidance of subtle non-normative behavior for which there are examples, but no programmable definition. Such behavior may violate legal or regulatory, rather than physical or monetary, constraints. In this article, I consider a series of experiments in which an intelligent stock trading agent maximizes profit but may also inadvertently learn to spoof the market in which it participates. I first inject a hand-coded spoofing agent to a multi-agent market simulation and learn to recognize spoofing activity sequences. Then I replace the hand-coded spoofing trader with a simple profit-maximizing RL agent and observe that it independently discovers spoofing as the optimal strategy. Finally, I introduce a method to incorporate the recognizer as normative guide, shaping the agent's perceived rewards and altering its selected actions. The agent remains profitable while avoiding spoofing behaviors that would result in even higher profit. After presenting the empirical results, I conclude with some recommendations. The method should generalize to the reduction of any unwanted behavior for which a recognizer can be learned.

Suggested Citation

  • David Byrd, 2023. "Learning Not to Spoof," Papers 2306.06087, arXiv.org.
  • Handle: RePEc:arx:papers:2306.06087
    as

    Download full text from publisher

    File URL: http://arxiv.org/pdf/2306.06087
    File Function: Latest version
    Download Restriction: no
    ---><---

    References listed on IDEAS

    as
    1. Luisa Mendonça & Alan De Genaro, 2020. "Detection and analysis of occurrences of spoofing in the Brazilian capital market," Journal of Financial Regulation and Compliance, Emerald Group Publishing Limited, vol. 28(3), pages 369-408, March.
    2. Lee, Eun Jung & Eom, Kyong Shik & Park, Kyung Suh, 2013. "Microstructure-based manipulation: Strategic behavior and performance of spoofing traders," Journal of Financial Markets, Elsevier, vol. 16(2), pages 227-252.
    3. Tālis J. Putniņš, 2012. "Market Manipulation: A Survey," Journal of Economic Surveys, Wiley Blackwell, vol. 26(5), pages 952-967, December.
    Full references (including those not matched with items on IDEAS)

    Most related items

    These are the items that most often cite the same works as this one and are cited by the same works as this one.
    1. Oliver Linton & Soheil Mahmoodzadeh, 2018. "Implications of High-Frequency Trading for Security Markets," Annual Review of Economics, Annual Reviews, vol. 10(1), pages 237-259, August.
    2. Cao, Zhiqi & Lv, Dayong & Sun, Zhenzhen, 2021. "Stock price manipulation, short-sale constraints, and breadth-return relationship," Pacific-Basin Finance Journal, Elsevier, vol. 67(C).
    3. Kemme, David M. & McInish, Thomas H. & Zhang, Jiang, 2022. "Market fairness and efficiency: Evidence from the Tokyo Stock Exchange," Journal of Banking & Finance, Elsevier, vol. 134(C).
    4. Haochen Li & Maria Polukarova & Carmine Ventre, 2023. "Detecting Financial Market Manipulation with Statistical Physics Tools," Papers 2308.08683, arXiv.org.
    5. Luisa Mendonça & Alan De Genaro, 2020. "Detection and analysis of occurrences of spoofing in the Brazilian capital market," Journal of Financial Regulation and Compliance, Emerald Group Publishing Limited, vol. 28(3), pages 369-408, March.
    6. Liu, Jie & Wu, Chonglin & Yuan, Lin & Liu, Jia, 2022. "Opening price manipulation and its value influences," International Review of Financial Analysis, Elsevier, vol. 83(C).
    7. Kadıoğlu, Eyüp & Frömmel, Michael, 2022. "Manipulation in the bond market and the role of investment funds: Evidence from an emerging market," International Review of Financial Analysis, Elsevier, vol. 79(C).
    8. James, Robert & Leung, Henry & Prokhorov, Artem, 2023. "A machine learning attack on illegal trading," Journal of Banking & Finance, Elsevier, vol. 148(C).
    9. Imisiker, Serkan & Tas, Bedri Kamil Onur, 2018. "Wash trades as a stock market manipulation tool," Journal of Behavioral and Experimental Finance, Elsevier, vol. 20(C), pages 92-98.
    10. Hauser, Shmuel & Kedar-Levy, Haim & Milo, Orit, 2022. "Price discovery during parallel stocks and options preopening: Information distortion and hints of manipulation," Journal of Financial Markets, Elsevier, vol. 59(PA).
    11. Sheridan Titman & Chishen Wei. Wei & Bin Zhao, 2021. "Corporate Actions and the Manipulation of Retail Investors in China: An Analysis of Stock Splits," NBER Working Papers 29212, National Bureau of Economic Research, Inc.
    12. Tiago Cardao-Pito, 2023. "How to identify norms, laws, and regulations that facilitate illicit financial flows and related financial crimes," Journal of Money Laundering Control, Emerald Group Publishing Limited, vol. 27(4), pages 674-686, August.
    13. Gregor Schwerhoff & Ottmar Edenhofer & Marc Fleurbaey, 2020. "Taxation Of Economic Rents," Journal of Economic Surveys, Wiley Blackwell, vol. 34(2), pages 398-423, April.
    14. Giancarlo Corsetti & Romain Lafarguette & Arnaud Mehl, 2019. "Fast Trading and the Virtue of Entropy: Evidence from the Foreign Exchange Market," Discussion Papers 1914, Centre for Macroeconomics (CFM).
    15. Dan Gabriel ANGHEL & Elena Valentina ŢILICĂ & Victor DRAGOTĂ, 2020. "Intraday Patterns in Returns on the Romanian and Bulgarian Stock Markets," Journal for Economic Forecasting, Institute for Economic Forecasting, vol. 0(2), pages 92-114, July.
    16. Luca Gelsomini, 2024. "On the Profitability of Rumors," Working Papers 2024: 06, Department of Economics, University of Venice "Ca' Foscari".
    17. Xintong Wang & Christopher Hoang & Yevgeniy Vorobeychik & Michael P. Wellman, 2021. "Spoofing the Limit Order Book: A Strategic Agent-Based Analysis," Games, MDPI, vol. 12(2), pages 1-43, May.
    18. William Quinn, 2019. "Squeezing the bears: cornering risk and limits on arbitrage during the ‘British bicycle mania’, 1896–8," Economic History Review, Economic History Society, vol. 72(4), pages 1286-1311, November.
    19. Dongyu Chen & Xiaolin Li & Fujun Lai, 2023. "Shill bidding in lenders’ eyes? A cross-country study on the influence of large bids in online P2P lending," Electronic Commerce Research, Springer, vol. 23(2), pages 1089-1114, June.
    20. Kumhof, Michael & Tideman, Nicolaus & Hudson, Michael & Goodhart, Charles, 2021. "Post-Corona Balanced-Budget Super-Stimulus: The Case for Shifting Taxes onto Land," CEPR Discussion Papers 16652, C.E.P.R. Discussion Papers.

    More about this item

    Statistics

    Access and download statistics

    Corrections

    All material on this site has been provided by the respective publishers and authors. You can help correct errors and omissions. When requesting a correction, please mention this item's handle: RePEc:arx:papers:2306.06087. See general information about how to correct material in RePEc.

    If you have authored this item and are not yet registered with RePEc, we encourage you to do it here. This allows to link your profile to this item. It also allows you to accept potential citations to this item that we are uncertain about.

    If CitEc recognized a bibliographic reference but did not link an item in RePEc to it, you can help with this form .

    If you know of missing items citing this one, you can help us creating those links by adding the relevant references in the same way as above, for each refering item. If you are a registered author of this item, you may also want to check the "citations" tab in your RePEc Author Service profile, as there may be some citations waiting for confirmation.

    For technical questions regarding this item, or to correct its authors, title, abstract, bibliographic or download information, contact: arXiv administrators (email available below). General contact details of provider: http://arxiv.org/ .

    Please note that corrections may take a couple of weeks to filter through the various RePEc services.

    IDEAS is a RePEc service. RePEc uses bibliographic data supplied by the respective publishers.