Beyond mimicry: a framework for evaluating genuine intelligence in artificial systems.
Frontiers in artificial intelligence January 1, 2025 DOI: 10.3389/frai.2025.1686752 via PubMed
Summary
A new evaluation framework, the Machine Perturbational Complexity & Agency Battery (mPCAB), shifts AI assessment from task performance to the cognitive processes underlying human-like understanding. It applies neurophysiological methods originally used to assess consciousness, incorporating perturbational complexity, global workspace assessment, norm internalization, and agency to compare digital, neuromorphic, and biological substrates. The framework addresses gaps in long-term reasoning, norm internalization under distribution shifts, and transformational creativity involving meta-cognitive rule modification. By analyzing theories of consciousness and mapping human cognitive functions to computational counterparts, it provides design principles. Pilot studies show feasibility across substrates and comparable metrics, moving evaluation toward mechanism-based assessment for developing mind-like machines.
Study at a glance
| Characteristics | Theoretical or philosophical paper Peer reviewed |
|---|---|
| Keywords | Agency Artificial intelligence Creativity Evaluation frameworks Machine consciousness |
| Key finding | The mPCAB framework provides a substrate-independent, mechanism-based evaluation that moves beyond superficial benchmarks by applying neurophysiological methods for assessing consciousness to artificial systems. |
Abstract
Current AI benchmarks often equate mimicry with genuine intelligence, emphasizing task performance over the underlying cognitive processes that enable human-like understanding. The Machine Perturbational Complexity & Agency Battery (mPCAB) introduces a new, substrate-independent framework that applies neurophysiological methods used initially to assess consciousness in artificial systems. Unlike existing evaluations, it features four key components-perturbational complexity, global workspace assessment, norm internalization, and agency-that link mechanisms with functions. This enables systematic comparisons across digital, neuromorphic, and biological substrates, addressing three research gaps: long-term reasoning with coherent behavior, norm internalization amid distribution shifts, and transformational creativity involving meta-cognitive rule modification. By analyzing theories of consciousness (GNW, IIT, PP, HOT), we identify targets for AI implementation. Our cognitive architecture analysis maps human functions-such as working memory and executive control-to their computational counterparts, providing guiding principles for design. The creativity taxonomy progresses from combinational to transformational, with measurable criteria like changes in conceptual space and the depth of meta-level reasoning. Ethical considerations are integrated into frameworks for monitoring organoid intelligence, reducing bias in creativity, and addressing rights issues. Pilot studies demonstrate mPCAB's feasibility across different substrates and show that its metrics are comparable. This framework moves evaluation away from superficial benchmarks toward mechanism-based assessment, supporting the development of mind-like machines and responsible AI advancements.