A Synergistic Integration Between Large Language Models and the Best-Worst Method

Large language models (LLMs) are increasingly being deployed for a variety of tasks, including recommendation systems. They are uniquely suited to making inferences in zero-shot circumstances given a purported capacity to reason across domains. However, there are several drawbacks to their use that limit real-world applicability. Namely, LLMs make opaque judgements that are not easily understood by users or subject to human control. Additionally, when many alternatives are evaluated, a robust external procedure is needed to control the model’s predictions. The Best-Worst Method (BWM) is an MCDA technique that can be used to extract preference weights and evaluate alternatives in a pairwise manner. This work proposes a hybrid model using BWM and LLMs that provides a human-interpretable framework for AI-ML recommendation. The system can function independently or as a refinement layer on top of existing retrieval systems. Due to the use of natural language as a mediator, it can process a broad spectrum of user and alternative information. An evaluation is conducted using the MovieLens dataset, showing a positive monotonic relationship between our predicted scores and user-submitted reviews.