The paper argues that algorithms are a kind of functional moral zombie, lacking sentience, and therefore cannot be autonomous or accountable moral agents. Just as philosophical zombies—exact physical duplicates of conscious beings without subjective experience—are used to challenge physicalism, algorithms similarly lack the experiential knowledge needed for moral understanding. Without feeling, an algorithm cannot truly value anything or act for moral reasons; its 'values' are merely prioritized items on a list. The author contends that moral agency requires sentience, so both zombies and algorithms are incoherent as morally responsible agents.
The paper argues that if valenced or affective experience is not essential for moral significance, as illustrated by hypothetical Vulcans who are conscious but lack affect, then one should be pressured to accept that consciousness itself may not be necessary for well-being, moral status, or psychological intrinsic value. It examines three positions: valence sentientism (valenced experience is necessary), broad sentientism (phenomenal consciousness generally is necessary), and non-necessitarianism (consciousness is unnecessary). The author contends that rejecting valence sentientism in light of Vulcans leads to accepting non-necessitarianism.
The AI rights research program faces an epistemic obstacle: the lack of a solution to the 'Hard Problem' of consciousness—why certain brain states produce subjective experience. Three possible grounds for AI rights—superintelligence, empathy, and consciousness—are examined. Appeals to superintelligence and empathy are problematic; consciousness should be the central focus, as in animal rights. However, AI rights differs from animal rights in a crucial way: animal rights can proceed without solving the Hard Problem, because we can reasonably assume animals are conscious based on biological similarity. For AI, no such assumption is possible, as we still do not understand why human brain states give rise to conscious mental states.