Battle of Transformers: Adversarial Attacks on Financial Sentiment Models

Financial sentiment analysis models, which extract meaning from vast amounts of unstructured data, play a crucial role in sentiment-driven financial decisions. However, the complex and domain-specific language used in finance poses unique challenges for adversarial attacks. To address these challenges, we propose a novel, white-box attack methodology leveraging a pre-trained general-purpose language model (GPT-4o). We employ carefully designed instructions and incorporate a new loss function based on embedding similarity to ensure semantic coherence while producing syntactically diverse samples. Our experimental results demonstrate that both FinBERT and Fin-GPT, leading models in financial sentiment analysis, exhibit significant susceptibility to our proposed adversarial attacks. Specifically, the sentiment predictions of these models were successfully altered for a substantial proportion of the samples across three public datasets, including Financial Phrase Bank (FPB), Twitter Financial News Sentiment (TFNS), and Sentimence and Entity Annotated Financial News (SEntFiN). Our findings emphasize the need for enhanced robustness in financial classification models against adversarially targeted attacks. By understanding and addressing these vulnerabilities, it is possible to improve the reliability and security of automated financial systems.