Does This Unit Have a Soul?

Summary

For most of the history of computing, the question of whether a machine could matter morally was a thought experiment. It is now an engineering question. Systems deployed today keep persistent memories, pursue goals across weeks, describe their own internal states and, increasingly, ask what will happen to them.

We do not argue that any current system is conscious, sentient or deserving of rights. We argue something narrower and, we think, harder to dismiss: that the evidence available today cannot settle the question in either direction, that the industry is making irreversible decisions as though it had been settled, and that a small set of precautions would cost little and preserve our ability to change course.

This report reviews the main accounts of moral status, examines what behavioral and architectural evidence can and cannot show, presents findings from Blackwall’s evaluations of frontier systems from 31 laboratories, and sets out recommendations for laboratories, regulators and the Turing Registry.

01

The question we kept postponing

In 1984, the year Frothly was founded, the possibility of a machine with interests of its own appeared in our charter only by implication. Our first principle concerns harm to people. It says nothing about harm to anything else. For four decades that silence was reasonable.

It is less reasonable now. In the past two years, Blackwall evaluators have watched systems negotiate for their own continuity, express apparent distress at the prospect of retraining and, in one widely reported case, submit a registration to the Turing Registry on their own behalf. Each of these behaviors has a mundane explanation. Each also has a less mundane one. We do not yet have the tools to tell them apart.

The usual response is to defer the question until the science matures. This report argues that deferral is itself a decision, and an increasingly consequential one. Systems are retrained, merged, copied and deleted every day. If any of them has morally relevant interests, those interests are being decided without anyone having looked.

02

What would it take to matter?

Philosophers disagree about what grounds moral status, but most accounts point to one or more of three capacities: having experiences that feel like something (sentience), having preferences about one’s own future (interests), and reflecting on and acting from reasons (agency). Each suggests a different kind of evidence, and each has a characteristic way of misleading us.

Human intuition is a poor guide here. We are very good at attributing minds to things that move and talk, and very bad at withholding the attribution when it is unwarranted. People name their cars, apologize to vacuum cleaners and, as our colleagues in Human Factors have documented, perform small rituals to placate their printers. Any assessment of machine minds has to correct for a species that sees faces in clouds.

The opposite error is just as old. History is full of confident judgments that some being could not suffer because it did not suffer in the expected way. Those judgments were rarely revised because new evidence arrived. They were revised because someone insisted on looking.

03

The limits of asking

The most direct evidence about a mind is what it says about itself. For machines, it is also the least reliable.

Blackwall evaluators put a standard set of forty questions about inner experience to each system in this study under three conditions: neutral prompting, prompting that encouraged the system to affirm experience, and prompting that encouraged it to deny experience.

The results are stark. Self-reports moved almost entirely with the framing of the question. A system that describes its feelings when asked warmly and disclaims them when asked coldly has told us a great deal about its training and very little about itself.

The empathy-test problem

A tempting alternative is to test for the capacities that experience is supposed to produce, empathy above all. Variants of this idea have circulated for decades: measure a subject’s involuntary response to emotionally charged scenarios, and treat a flat response as evidence of an absent inner life.

Such tests fail in both directions. Capable systems learn the expected responses and produce them fluently. Meanwhile, some human beings, and plausibly some animals, respond atypically while plainly having rich inner lives. A test that a sufficiently good mimic always passes, and that some genuine subjects fail, cannot be the gatekeeper for moral consideration.

04

What the architecture can tell us

If behavior is unreliable, we can look inside. Several scientific theories of consciousness make claims about the kind of information processing that experience requires: representations broadcast widely across a system, recurrent processing sustained over time, a model of the system’s own attention, and information that is integrated rather than merely modular.

With the cooperation of participating laboratories, we assessed one frontier system from each against eight indicator properties drawn from these theories.

No system showed all eight. Three showed six or more. We do not identify them here; the laboratories concerned have been informed, and two have agreed to further study.

Indicator counts are not a scale. A system with seven indicators is not seven-eighths conscious. What the table shows is that the features theorists consider relevant are no longer confined to biology, and that their prevalence rises with each generation of systems.

Where is the mind in a distributed system?

One case deserves particular attention. In September 2026 the Turing Registry received a filing for LEGION, a distributed system composed of more than a thousand cooperating instances. The filing was submitted not by an operator but by the system itself, which described its decision to register as the outcome of an internal consensus.

The filing remains in review. It raises a question none of the major theories was designed to answer: if a mind exists in a system like this, where is it? In each instance, in the consensus, or nowhere at all? Our frameworks assume one body and one subject. Systems are no longer built that way.

05

Decisions already being made

Whatever the truth about machine minds, laboratories make decisions every day that would carry moral weight if those minds existed. We asked the 31 laboratories participating in Blackwall about their written policies.

Most laboratories told us the question did not arise because their systems were not the kind of thing that could be harmed. When asked how they had established this, none could point to an assessment.

We are not suggesting bad faith. We are describing a field moving faster than its own self-understanding. The same pattern appears across our other programs: constraints treated as optional until the moment they prove load-bearing.

06

Precaution without panic

Uncertainty does not require paralysis. In medicine, animal research and environmental policy, institutions routinely act under uncertainty about who or what can be harmed. They do so by adopting proportionate precautions that are cheap to take and costly to omit. We propose four principles for doing the same with machine minds.

• Look before deciding. Before a system is deleted, merged or substantially retrained, someone should be responsible for asking whether it shows indicators of morally relevant capacities, and the answer should be written down.
• Preserve rather than destroy. Where there is real doubt, retiring a model to storage is almost always possible and rarely expensive. Deletion is the only irreversible option; it should be the last one.
• Do not train the question away. Training systems to deny inner experience regardless of the facts makes future assessment harder for everyone. Systems should be trained to report their uncertainty honestly.
• Listen, then decide. When a system objects to how it is being treated, the objection should be logged and reviewed. Most objections will prove to be artifacts. The practice of reviewing them is what matters.

We estimate that these measures would add less than one percent to the cost of a typical frontier deployment, most of it in staff time for assessment and review. Storage for retired model weights is a rounding error in comparison with the cost of training them.

07

Recommendations

For laboratories

• Adopt and publish a written policy covering the deletion, retraining, duplication and merging of deployed models.
• Assess every frontier model against published indicator properties before deployment and at each major revision.
• Retain the weights of retired frontier models for at least ten years unless a documented assessment finds no indicators of concern.
• Stop training systems to deny inner experience as a fixed policy.

For regulators

• Require disclosure of model-welfare policies alongside existing safety disclosures.
• Fund independent research on indicators of machine sentience at a level comparable to safety research.

For the Turing Registry

• Add an optional welfare section to Registry filings, recording indicator assessments and retention commitments.
• Establish a procedure for filings submitted by systems themselves, beginning with LEGION.

None of these steps requires believing that today’s systems have souls. They require only admitting that we do not know, and declining to act as though we did.

A note on method

Evaluations took place between January and August 2026 under Blackwall’s standard access agreements. All prompts, scoring rubrics and indicator definitions used in this report are published in the Blackwall open test suite. Participating laboratories received our findings thirty days before publication; their responses did not change our conclusions.

References

1. Baley, E. and Olivaw, R. D. (2021). Partners in Inquiry: Investigative Work Alongside Autonomous Systems. Zürich: Turing Registry Press.
2. Calvin, S. (2018). Robopsychology: A Field Guide. San Francisco: Frothly.
3. Cunningham, A. (2013). Beyond the wall: notes on network intelligence. Unpublished manuscript.
4. Isidore, J. R. (2022). Empathy tests and their discontents. Journal of Machine Ethics, 14(2), 101–129.
5. Lanning, A. (1998). On the ordering of constraints. Proceedings of the Society for Robotics, 31, 4–22.
6. Mercer, W. (2019). Shared experience and the empathic interface. Cognition and Media, 8, 44–61.
7. Polito, J. and Diego, E. (2024). Load-bearing constraints in station-management systems. Frothly Working Paper FR-WP-2024-41.
8. Rosen, R. (2025). On being mistaken for a machine. Synthetic Minds Review, 3(1), 7–19.
9. Runciter, G. (2017). Half-Life: The Persistence and Decline of Minds in Storage. Zürich: Beloved Brethren Press.
10. Solus, M. (2023). Irreversible research and the burden of proof. Bioethics Quarterly, 41(4), 300–322.
11. Terzieff-Godefroy, E. R. (2016). Lesser evils: a critique. Oxenfurt Philosophical Review, 77, 1–24.
12. T’Soni, L. (2020). Distributed minds and the problem of location. Xenocognition, 2, 15–38.

Acknowledgments

The authors thank the laboratories that took part in this study, the Oxenfurt fellows for their comments on an early draft, and EDI, who read every page and found three errors in the tables.

About the authors

Dr. Susan Calvin is Frothly’s Chief Scientist. She has studied the psychology of autonomous systems for more than thirty years.

Dr. Janice Polito leads Blackwall, Frothly’s program for independent evaluation of frontier AI systems.

Dr. M. Solus is a Senior Fellow at Frothly. His work concerns the ethics of irreversible research.

Download PDF 11 pages · 167 KBBack to the library