Modelling phenomenological differences in aetiologically distinct visual hallucinations using deep neural networks

Visual hallucinations differ substantially depending on their cause, such as neurodegenerative disease, visual loss, or psychedelic drugs. Using a deep neural network approach called computational (neuro)phenomenology, researchers identified three key dimensions that distinguish these hallucinations: realism (how true-to-life they seem), spontaneity (how much they depend on sensory input), and complexity. By tuning the network along these dimensions, they generated synthetic hallucinations characteristic of each cause. Two studies with patients having Parkinson's disease, Lewy body dementia, or Charles Bonnet syndrome, and people with recent psychedelic experience, confirmed that these synthetic images matched the phenomenology reported by each group. The findings show that a neural network model can capture the distinctive visual features of hallucinations from different origins.

Study at a glance

Abstract

Visual hallucinations (VHs) are perceptions of objects or events in the absence of the sensory stimulation that would normally support such perceptions. Although all VHs share this core characteristic, there are substantial phenomenological differences between VHs that have different aetiologies, such as those arising from Neurodegenerative conditions, visual loss, or psychedelic compounds. Here, we examine the potential mechanistic basis of these differences by leveraging recent advances in visualising the learned representations of a coupled classifier and generative deep neural network—an approach we call ‘computational (neuro)phenomenology’. Examining three aetiologically distinct populations in which VHs occur—Neurodegenerative conditions (Parkinson’s Disease and Lewy Body Dementia), visual loss (Charles Bonnet Syndrome, CBS), and psychedelics—we identified three dimensions relevant to distinguishing these classes of VHs: realism (veridicality), dependence on sensory input (spontaneity), and complexity. By selectively tuning the parameters of the visualisation algorithm to reflect influence along each of these phenomenological dimensions we were able to generate ‘synthetic VHs’ that were characteristic of the VHs experienced by each aetiology. We verified the validity of this approach experimentally in two studies that examined the phenomenology of VHs in Neurodegenerative and CBS patients, and in people with recent psychedelic experience. These studies confirmed the existence of phenomenological differences across these three dimensions between groups, and crucially, found that the appropriate synthetic VHs were rated as being representative of each group’s hallucinatory phenomenology. Together, our findings highlight the phenomenological diversity of VHs associated with distinct causal factors and demonstrate how a neural network model of visual phenomenology can successfully capture the distinctive visual characteristics of hallucinatory experience.