Decoding the phenomenology of spontaneous thought using large language-model ratings on verbal retrospective free reports

Summary

Spontaneous thoughts make up most of everyday inner experience, but studying them is difficult because traditional methods disrupt the natural flow of thinking or introduce motor artifacts. An alternative approach combined delayed verbal retrospective free reports with automated ratings from large language models. Twenty-two participants performed an eyes-closed free-thinking task, and their reports were evaluated on ten dimensions by four LLMs and human raters. Machine-learning models trained on EEG features achieved above-chance accuracy for predicting emotional valence. LLMs showed higher inter-rater agreement than humans, supporting their use for scalable annotation and suggesting that affective dimensions of spontaneous thoughts can be decoded from brain activity.

Study at a glance

Abstract

Spontaneous thoughts constitute most of everyday inner experience, yet long-standing methodological challenges obscure a thorough exploration of their content and neurophysiological underpinnings. Traditional approaches relying on thought probes impose strict constraints on phenomenological reports, whereas online verbal reports disrupt the natural flow of experience while interfering neural signals with motor artifacts. Here, we designed and tested an alternative approach to assess the neural basis of spontaneous thoughts combining delayed verbal retrospective free reports (RFR) with automated phenomenological ratings generated by large language models (LLMs). Twenty-two participants performed an eyes-closed free-thinking task, providing reports that were evaluated along ten phenomenological dimensions by four state-of-the-art LLMs and a panel of human raters. Machine-learning models (ML) were then trained to decode LLM-derived ratings from EEG spectral, complexity, and connectivity features. Our analyses showed that inter-rater agreement among LLMs exceeded that of human raters whereas ML models achieved above-chance accuracy for the prediction of emotional valence. These findings provide support for the use of LLMs for a scalable phenomenological annotation of spontaneous thoughts and suggest that their affective dimensions can be decoded from concurrent EEG activity.