Every political community arises, to some extent, from a promise of protection. But that promise cannot be understood in a narrow or reductionist way. Protection does not mean only defending against danger, containing threats, or neutralizing risks. It also means upholding the conditions that make a truly human life possible: belonging, recognition, education, work, health, participation, relationships, and rights. A society does not protect only when it prevents someone from being attacked; it also protects when it prevents someone from being abandoned, rendered invisible, or excluded from the community to which they belong.

This distinction is crucial. Security is a necessary good, but it does not exhaust the meaning of protection. Without a minimum level of security, there is no effective freedom, no stable coexistence, and no rights that can be exercised without fear. But when security occupies the entire public discourse, protection risks diminishing, to the point of being equated with surveillance, control, and constant prevention. Then the community stops asking how it should care for its members and begins to ask, almost exclusively, how it should defend itself against them. The problem, therefore, is not recognizing the value of security. The problem lies in making it the sole guiding principle of collective life.

History shows that few concepts have served as frequently as a certain understanding of security to extend power beyond its limits. Where everything can be presented as a risk, almost anything can be claimed as a necessary measure. And where protection is framed solely in terms of threat, citizens may cease to be seen as subjects of rights and instead be regarded as potential sources of danger. A long tradition of political and moral thought has warned against such a reduction. A just community is not the only one that protects against harm, but also one that allows people to live without being left out in the cold. From this perspective, the basic right would not simply be the right to be safe—if by “safety” we mean the absence of threat—but rather the right not to be abandoned by the community. That right points to a network of rights and guarantees without which dignity becomes an abstract concept: social rights, public services, material conditions of existence, institutional recognition, and a real possibility of participation.

This initial clarification is particularly necessary when analyzing the application of artificial intelligence to social control. These systems are often presented in terms of security, efficiency, and prevention. However, their impact cannot be assessed solely by asking whether they enable us to anticipate risks or improve public administration. The deeper question is this: whether they contribute to truly humane protection or, on the contrary, reinforce a form of governance based on suspicion, classification, and surveillance.

Artificial intelligence does not introduce surveillance, but it transforms it. Traditional surveillance was more visible, localized, and limited: a border, a camera, an investigation, an administrative file, a court order. Algorithmic surveillance operates in a more diffuse manner. It collects scattered data, cross-references databases, identifies patterns, assigns probabilities, and generates profiles. It is not limited to observing what a person does. It seeks to infer what they might do. This shift alters the very meaning of control. People are no longer judged solely by their actions but are instead evaluated based on their membership in statistical categories. They can be classified as trustworthy or suspicious, reliable or dubious reliability, a priority or dispensable. The danger does not lie solely in the accumulation of personal information. It lies in the fact that this information, processed through opaque systems, can influence how a person gains access to rights, services, public spaces, or administrative decisions.

The question, therefore, is not whether artificial intelligence can contribute to legitimate purposes. Of course it can. The question is under what conditions it can be used without degrading human dignity, eroding freedom, or turning prevention into a state of perpetual suspicion. The recent encyclical*Magnifica Humanitas*offers a useful framework for addressing this question. By warning against the “Babel syndrome,” it denounces the temptation to build a world that is technically powerful but humanly impoverished, in which efficiency, domination, or technological self-sufficiency ultimately sacrifice human dignity. Algorithmic surveillance can become a contemporary form of that Babel: an invisible architecture of classification and control that promises order but risks reducing the person to a data point, a profile, or performance statistics.

1. From Visible to Algorithmic Surveillance

Traditional surveillance was typically linked to suspicion, a specific space, or a particular purpose. Algorithmic surveillance tends to operate before individualized suspicion arises. Its logic is not so much verification as anticipation. It is not limited to reconstructing past events but aims to foresee future risks. This shift has significant legal and moral consequences. First, because it allows public intervention to be brought forward to increasingly early stages. Second, because the basis for that intervention can be diluted into statistical correlations. And finally, because the person affected may not know the criteria used to determine their classification.

Artificial intelligence applied to social control does not merely see more; it sees differently. Where the law demands reasons, the system may offer scores. Where justice requires individualization, the model may produce groupings. Where dignity demands being treated as a subject, technology may treat the person as an object of calculation.

The UNESCO recommendation on the ethics of artificial intelligence therefore insists that these systems must respect human dignity, human rights, privacy, transparency, explainability, human oversight, and non-discrimination throughout their entire lifecycle (UNESCO, 2021). These are minimum conditions to prevent technology from becoming power without accountability.

2. The Power to Classify

The most problematic aspect of algorithmic surveillance is its capacity to classify. The person under surveillance is not merely observed but placed within a category. That category can have real-world effects, even when it does not take the form of a formal decision. It can intensify controls, delay benefits, guide investigations, influence evaluations, or exclude individuals from opportunities. Algorithmic classification is particularly problematic because it is often presented as neutral. However, data does not arise in a vacuum. It comes from societies marked by inequalities, biases, and power dynamics. If a system is trained on historical data that reflects discriminatory practices, it can reproduce and even amplify them. Discrimination then ceases to appear as visible bias and takes the form of a technical outcome. This phenomenon has been widely documented in studies on algorithmic discrimination. Machine learning systems can perpetuate human biases present in training data and project them onto areas such as employment, credit, healthcare, criminal justice, education, or migration (Iturmendi Rubia, 2023). The opacity of these systems also hinders accountability and the ability to challenge unfair decisions.

This is no minor issue. An unjust human decision may affect a single case. A biased algorithmic system can simultaneously affect thousands of people. Moreover, it can do so under the guise of objectivity, which hinders legal, social, and political responses. Injustice becomes harder to identify when it is cloaked in precision. That is why it is important to remember a fundamental principle: statistical effectiveness does not equate to legal legitimacy. A system can predict with a certain degree of accuracy and still violate rights. It can organize information more effectively while simultaneously reinforcing inequalities. It can be useful for the administration and harmful to the people affected.

3. Biometrics: The Body as Permanent Data

Biometrics introduces an additional level of intensity. The face, fingerprint, voice, iris, or gait are not ordinary data. They form part of a person’s bodily identity. When incorporated into automated identification or verification systems, the body becomes a credential. The difference from other personal data is clear. A password can be changed. An email address can be canceled. An ID document can be renewed. But no one can change their iris, face, or fingerprint. For this reason, biometric data is particularly sensitive and requires enhanced safeguards. Its indiscriminate use can infringe upon privacy, freedom of assembly, freedom of expression, the presumption of innocence, and the right not to be subject to constant surveillance.

The Worldcoin case in Spain clearly illustrates these risks. In March 2024, the Spanish Data Protection Agency ordered a precautionary halt to the collection and processing of personal data associated with the Worldcoin project and to the further processing of data already collected. Subsequently, the Bavarian data protection authority found violations of the General Data Protection Regulation and demanded corrective measures, including the deletion of iris codes stored since the project’s inception (Spanish Data Protection Agency, 2024).

This example raises a broader question: Can we speak of truly free consent when irreversible biometric data is provided in exchange for financial compensation or access to certain services? The answer cannot be abstract. It depends on the social circumstances of the person consenting. Providing data from a position of freedom is not the same as doing so out of necessity. Biometric surveillance particularly affects those with the least ability to resist. For some citizens, these technologies are a convenience. For others, they can become a requirement, a form of control, or a condition of access.

4. Predictive Policing and Anticipatory Suspicion

Predictive policing clearly illustrates the shift from monitoring actions to monitoring risks. These systems aim to anticipate where a crime might occur or which individuals are most likely to reoffend. The stated purpose may seem reasonable—to better allocate resources, prevent harm, or protect the public. The main problem is that a person may be treated not for what they have done but for what a system calculates they might do. In the criminal justice system, this logic is especially sensitive. Democratic criminal law is based on accountability for acts, not on belonging to risk profiles. The presumption of innocence, individual culpability, and the right to defense cannot be replaced by group probabilities.

There is also a territorial risk. If certain neighborhoods have historically been more heavily monitored, they will have generated more police data. If that data feeds into predictive systems, the result will likely be a renewed concentration of surveillance in those same neighborhoods. The system will have transformed a history of inequality into the appearance of objective risk. Prediction then becomes a self-fulfilling prophecy.

The well-known case of State v. Loomis highlights these difficulties. In 2016, the Wisconsin Supreme Court cautiously approved the use of a risk assessment tool during the sentencing phase. The defendant argued, among other issues, that he had been unable to adequately understand or challenge the methodology of the COMPAS system, which was used to assess recidivism risk (State v. Loomis, 2016). Beyond the specific outcome of the case, the question remains. Can a person defend themselves against a score whose internal logic they do not understand? Can a judge rely on an assessment that they cannot fully explain? What room remains for personal judgment in decisions mediated by risk profiles?

Public safety does not justify replacing personal responsibility with statistical probability. Nor does it justify treating certain social groups in advance as potential threats. If artificial intelligence technically reintroduces categories of suspicion that constitutionalism had sought to overcome, then we are not facing a mere instrumental advance, but rather an erosion of safeguarding.

5. The Normalization of the Exception

Every society accepts exceptional measures in response to exceptional situations. But the exception loses its true nature when it becomes permanent. One risk of algorithmic surveillance is precisely this: turning a state of alert into routine infrastructure. Health, migration, terrorist, or security crises often justify the use of extraordinary tools. The problem arises when those tools remain in place after the crisis has passed. Cameras do not disappear easily. Databases tend to be retained. Models are reused. Governments rarely give up capabilities that enhance their monitoring capacity.

A permanent state of emergency alters the relationship between citizens and those in power. Freedom is tolerated if it does not interfere with prevention. Privacy is recognized if it does not hinder security. Protests are permitted if they are not deemed a risk. Democratic forms are maintained, but the tangible trust that allows people to live as citizens—and not merely as files—can erode. Freedom of conscience requires more than the mere absence of direct coercion. It also needs social conditions in which to be exercised. A person who knows—or reasonably suspects—that their movements, face, travel, or relationships may be recorded and evaluated tends to moderate their behavior. Mass surveillance leads to self-censorship.

6. Those Most Surveilled

Algorithmic surveillance is not distributed evenly. Experience shows that surveillance tends to be concentrated on those who were already most exposed to institutional power: the poor, migrants, minorities, residents of marginalized areas, people who depend on public benefits, and groups with less capacity to challenge the system.

The case of Systeem Risico Indicatie (SyRI) in the Netherlands is particularly significant. This system was used to detect potential fraud in benefits, taxes, and social security contributions by cross-referencing public data. In 2020, the District Court of The Hague ruled that the regulations underpinning the system violated Article 8 of the European Convention on Human Rights—which concerns the right to privacy—due to a lack of proportionality, transparency, and sufficient safeguards (Rechtbank Den Haag, 2020). The United Nations Special Rapporteur on extreme poverty welcomed the decision as an important ruling against state attempts to “spy on the poor,” emphasizing that the system had targeted low-income neighborhoods with minority populations (OHCHR, 2020).

As noted at the beginning, for the affluent, artificial intelligence often appears as personalization, convenience, or efficiency. For vulnerable people, however, it can appear as surveillance, suspicion, a requirement, or a filter. Some receive tailored services; others face automated monitoring. Some are treated as consumers; others are treated as risks. Human dignity demands that we resist this technological double standard. No innovation deserves to be called progress if it increases the ability to monitor those with fewer opportunities to defend themselves. A just society cannot build its security on the constant exposure of the vulnerable.

Picture IA with ChatGPT for document - When protection becomes surveillance - Artificial Intelligence and Human Dignity

7. Dignity and the Common Good as Limits

This is not about rejecting all artificial intelligence applied to security. Such a conclusion would be as simplistic as it is useless. There are legitimate uses of technology to protect rights, improve public administration, prevent harm, and detect abuse. But precisely because these uses can be legitimate, they require clear limits:

• Human dignity. No person should be treated as mere data, as an identifiable body, or as a potential threat. Dignity prevents reducing a person’s life story to a profile and freedom to risk calculation.

• Strict legal regulations. Any intrusive surveillance must be regulated by a clear rule, serve a legitimate purpose, and be subject to independent oversight.

• Proportionality. Not everything that is useful is necessary. Not everything that is effective is permissible. Not every administrative improvement justifies a significant intrusion into people’s lives.

• Transparency. People must know when automated systems that may affect their rights are being used. And they must be able to understand, to a sufficient degree, the reasons behind a decision that adversely affects them.

• The right to challenge. A decision that cannot be understood can hardly be challenged. Without a real opportunity to contest it, the guarantee becomes merely a formality.

To these limits must be added a requirement for substantive justice: examining who bears the cost of the technology. The ethical question is not only whether the system works, but for whom it works. Does it protect the vulnerable, or does it make them the primary target of surveillance? Does it expand rights, or does it refine mechanisms of exclusion? Does it serve the common good, or does it entrench new asymmetries between those who monitor and those who are monitored?

The European Artificial Intelligence Regulation has adopted a preventive approach based on risk levels, with strengthened obligations for systems that may affect fundamental rights. But regulation, while essential, is not enough. We also need a public culture that understands that efficiency is not the ultimate criterion of a decent society (Iturmendi Rubia, 2024).

The alternative, asMagnifica Humanitassuggests, does not lie in uncritically accepting technology or rejecting it out of fear. The real dilemma is another: whether to erect a new Tower of Babel of data, suspicion, and social classification, or to build a shared city in which technology remains at the service of the individual (León XIV, 2026).

Security is a common good when it protects the life and liberty of all. It ceases to be so when it requires some to live under suspicion so that others may live in peace. It is also measured by its refusal to sacrifice the dignity of those who are easiest to monitor.

Artificial intelligence can serve the common good if it remains subject to legal limits, democratic oversight, and ethical discernment. When those limits disappear, it ceases to be a tool and becomes a regime. And no regime of suspicion, however sophisticated it may be, can be reconciled with the dignity of the human person.

References

• Spanish Data Protection Agency. (December 19, 2024).Worldcoin will have to delete all iris codes stored since the project’s inception. https://www.aepd.es/prensa-y-comunicacion/notas-de-prensa/worldcoin-tendra-que-eliminar-todos-los-codigos-de-iris
• Iturmendi Rubia, J. M. (2023).Algorithmic discrimination and its impact on human dignity and human rights. Special reference to immigrants. Deusto Journal of Human Rights, 12, 257–284. https://doi.org/10.18543/djhr.2910
• Iturmendi Rubia, J. M. (2024).Artificial Intelligence and Human Rights: Challenges and Opportunities in the Digital Age. Introduction to the Special Issue. Deusto Journal of Human Rights, 14, 11-31. https://doi.org/10.18543/djhr.3202
• León XIV. (2026).Magnifica Humanitas. Holy See. https://www.vatican.va/content/leo-xiv/es/encyclicals/documents/20260515-magnifica-humanitas.html
• Office of the United Nations High Commissioner for Human Rights. (February 5, 2020). Landmark ruling by Dutch court stops government attempts to spy on the poor – UN expert. https://www.ohchr.org/en/press-releases/2020/02/landmark-ruling-dutch-court-stops-government-attempts-spy-poor-un-expert
• District Court of The Hague. (February 5, 2020). SyRI legislation in breach of the European Convention on Human Rights. https://www.rechtspraak.nl/organisatie-en-contact/organisatie/rechtbanken/rechtbank-den-haag/nieuws/syri-legislation-in-breach-of-european-convention-on-human-rights
• State v. Loomis, 881 N.W.2d 749 (Wis. 2016). https://law.justia.com/cases/wisconsin/supreme-court/2016/2015ap000157-cr.html
• UNESCO. (2021). Recommendation on the Ethics of Artificial Intelligence. https://unesdoc.unesco.org/ark:/48223/pf0000381137_spa

Author

Cover image credits: A person looking at the device known as the “Orb”, a technological tool used by Worldcoin to scan people’s irises. Credit: Copyright 2023 The Associated Press. All rights reserved. Source: University of Chile. Published on Friday, 12 April 2024.