SMALL BUT MIGHTY: UNLOCKING FINANCIAL SENTI- MENT ANALYSIS WITH COMPACT LANGUAGE MODELS

Pre-trained large language models (LLMs) have dominated the landscape of finan-cial sentiment analysis, as demonstrated by studies such as (Luo & Gong, 2024)that leverage models like LLaMA-2 7B to achieve high classification accuracy onfinancial texts. Despite the extensive use and impressive performance of theseLLMs, the potential of small language models (SLMs) remains relatively under-explored. In this work, we investigate the viability of SLMs for Financial Sen-timent Analysis (FSA) by employing parameter-efficient fine-tuning techniques.Specifically, we adapt Microsoft’s phi-3 mini model using Low-Rank Adaptation(LoRA) to classify financial texts into positive, negative, or neutral sentiments.Leveraging the Financial PhraseBank dataset and strategic prompt engineering,our approach fine-tunes the model under various low-rank settings (r = 16, 8, and4). Experimental results demonstrate that phi-3 mini, despite not being pre-trainedon extensive financial data, achieves competitive performance with an overall ac-curacy of 88%, closely matching FinBERT’s 88.93% accuracy. Notably, phi-3mini attains a superior F1-score in detecting negative sentiment, underscoring itsrobustness in critical analysis scenarios. These findings highlight the promise ofwell-tuned, general-purpose language models as cost-effective and flexible alter-natives to specialized, resource-intensive models in financial applications.