Anxiety Prediction in Narrative Texts Based on Large Language Models: A comparison of Model Selection and Prompting Styles

Abstract: Anxiety mainly involves subjective anxious experiences, cognitive vigilance, and somatic symptoms. With the development of social media, identifying anxiety from textual data to assist clinical intervention has broad application prospects. Previous studies have predominantly relied on traditional machine learning approaches to detect anxiety in text, with limited accuracy. The development of large language models (LLMs) has provided a novel and promising pathway for textual anxiety identification. The present study examined the performance of LLMs in identifying anxiety in self-reported narrative texts under different model types and prompt styles. Four experts first independently pre-rated 50 texts, and satisfactory inter-rater reliability was achieved. Subsequently, expert ratings were conducted for a final set of 252 texts. Under the guidance of four prompt strategies (zero-shot, few-shot, chain of thought, and few-shot + chain of thought), Qwen-max and Deepseek-reasoner were employed to generate model-based ratings. The results indicated that Qwen-max achieved an agreement of approximately 0.8 with expert ratings, significantly outperforming Deepseek-reasoner (0.6–0.7). Few-shot + chain of thought achieved the best evaluation performance, whereas zero-shot performed the worst. Qwen-max was less sensitive to variations in prompt strategies than Deepseek-reasoner. In addition, for Deepseek-reasoner, the inclusion of examples in prompts (few-shot), compared with chain-of-thought, contributed more to performance improvement. This study provides a preliminary technical framework for future assessment and screening of anxiety in textual data.