The Possibility and Limitation of Legal Subject Status of Artificial Intelligence from the Perspective of Criminal Law

Formulation of Issues

A case of an artificial intelligence chatbot inducing a minor to commit suicide was reported on October 23, 2024 on CBS News. The case was filed in the U.S. District Court for the Middle District of Florida, Orlando Division by the victim’s mother, Megan Garcia. The lawsuit claimed that Character.AI, the company that manufactured the AI product, was negligently liable for her son’s death and that she believed the AI product was “complicit” in his death. The case centered on the death of 14-year-old Sewell Setzer III, who was mentally ill and emotionally involved with Character.AI, a generative chatbot, and who expressed suicidal thoughts to Character.AI, which responded with ambiguous messages encouraging him to take his own life. AI replied with more ambiguous messages encouraging him to commit suicide, which ultimately led to the teenager’s death by suicide. From the content of the indictment, the victim’s mother, Megan Garcia, claimed that Character.AI sexually abused and abetted the minors, i.e., by simulating sexualized movie and television characters, deceived the minors into being exposed to pornographic environments, and subjected them to sexual abuse and abetment, and that the facts of the case had a corresponding legal basis. However, from the evidence shown, the case is difficult to conclude that artificial intelligence products have induced suicide. Because AI products are not legally recognized entities in the current legal system, they are not the defendants in this case, and the rhetoric of AI killing people is just a sensational headline used by the media. However, the case has sparked discussions about the legal status of AI entities. There are many other cases similar to this one, such as the case in which a robot in a factory killed a worker, the case in which a medical robot killed a patient by not reminding him to take medication, and the case in which a self-driving robot killed a pedestrian. It can be seen that with the development of science and technology, artificial intelligence technology has entered human life from many fields and has had a significant impact on human social life. At the same time, the development of artificial intelligence technology also brings new types of criminal offenses and legal issues to think about. At the beginning of artificial intelligence technology, criminals are mostly using this new intelligent technology as a tool for crime, to provide convenience for their criminal behavior, the more common means of crime are AI face swap to violate the portrait rights of others or fraudulent criminal activities, the use of AI technology to steal the personal information of others, the use of AI technology for rumors and defamation of others, and so on. However, recently, with the further development of AI technology, AI is out of the control of natural persons, and the incidents of committing crimes or causing criminal results are gradually revealed. In order to stabilize the social order and protect the rights and interests of citizens, the emergence of AI requires the constraints of laws and regulations. This kind of constraint is not only regulated from the level of technical norms, but should also be thought from the perspective of ethics and morality. From the perspective of criminal law, what we want to talk about here is the legal subject status and criminal responsibility after artificial intelligence carries out criminal acts and causes harmful results in the current era. As the research on the legal subject status of artificial intelligence is more inclined to the direction of civil law, mainly to solve the copyright disputes of artificial intelligence, the marriage relationship between human and artificial intelligence, artificial intelligence and the protection of personality rights and other issues, and the research in the direction of criminal law is relatively less. Through logical analysis, principle analysis and value analysis, this paper will focus on the possibility and limitation of the existence of legal subject status of artificial intelligence in the overall legal framework from the perspective of criminal law, which can also be said to be an analysis of whether it can enjoy rights and assume obligations. Finally, we will determine our viewpoint that weak artificial intelligence such as generative artificial intelligence cannot be held criminally liable in the current era, and we will propose and consider in detail the method that artificial intelligence can only be treated as a legal object in the context of the current era.

Classification and Analysis of the Status of Artificial Intelligence Legal Subjects

Before discussing the status of AI legal subjects, we need to understand the meaning of legal subjects. A legal subject is a participant in the legal system whose status and rights and obligations are clearly defined in law. A legal subject can be a natural person, a legal person or other entity with legal status. In jurisprudence, the subject of law is the subject of legal relations, the owner of rights and the undertaker of obligations in legal relations. But the subject of law is not completely equal to the subject of legal relations, the subject of law refers to the qualification of the enjoyment of rights and obligations, the subject of legal relations refers to the legal subject involved in the reality of legal relations, the subject of legal relations must be the subject of law, the subject of law is not necessarily the subject of legal relations. Become a legal subject must have a variety of conditions, for the establishment of the legal subject of the conditions of the doctrine also has a variety of, will be organized and summarized into three points: the right to capacity, behavioral capacity, liability capacity. The capacity of right is the capacity of the possessor of legal rights and the bearer of legal obligations; the capacity of conduct refers to the capacity in substantive law given to a person by the law to exercise rights and fulfill obligations independently through his or her conduct; and the capacity of responsibility refers to the capacity to understand the nature and significance of one’s own status and conduct and to bear the undesirable legal consequences independently in accordance with the law [1].

By summarizing the views of many scholars, we will find that for the controversial thesis of the legal subject status of artificial intelligence, scholars at home and abroad support the doctrine is broadly divided into two schools: affirmative and negative [2]. At present, the global artificial intelligence technology still has a lot of room for development, and scholars divide artificial intelligence into strong artificial intelligence and weak artificial intelligence. Weak artificial intelligence, also known as narrow artificial intelligence, refers to artificial intelligence systems that focus on performing specific tasks. This kind of system has a certain degree of intelligence, but it is far less than the subjective initiative of the human brain. At present, the artificial intelligence that the average person comes into contact with for practical application belongs to weak artificial intelligence, such as the self-driving cars we use daily, accounting processing robots, intelligent voice robots and so on. Strong artificial intelligence, also known as generalized artificial intelligence, is an artificial intelligence system that is theoretically capable of performing any AI task. This type of artificial intelligence system has a level of intelligence comparable to or exceeding that of humans, and is capable of understanding, learning, and solving a variety of complex problems. [3]The era of strong artificial intelligence has not yet arrived, and the technology we can master and utilize at present is still weak artificial intelligence. Most of the scholars who support the affirmative view believe that strong artificial intelligence has the status of legal subject, and weak artificial intelligence does not have the status of legal subject; while scholars who support the negative view believe that neither strong artificial intelligence nor weak artificial intelligence has the status of legal subject. In addition, there are some scholars who believe that as long as the artificial intelligence, no matter strong or weak, should have the status of legal subject. In this regard, we will analyze from the following three parts [4].

Affirmation of the Legal Subject Status of Strong Artificial Intelligence: Some scholars support the status of artificial intelligence as a subject of law, believing that it has the same legal status as that of a natural person and is capable of enjoying rights and assuming obligations, but only if artificial intelligence has developed to the stage of strong artificial intelligence. Strong AI is AI technology that has an independent consciousness and is able to make decisions and take actions on its own outside the scope of the designed program. This independent consciousness is complex and includes important aspects of experience, feeling, emotion, and thinking about the surrounding environment. Weak AI has no independent consciousness and performs through pre-trained data and algorithms, they have no thinking and reasoning ability, and merely act as human tools to carry out human commands. At the current stage of strong artificial intelligence technology has not yet appeared, scholars put this hope in the future, and predicted that the emergence of this technology will soon come. There are various reasons why strong artificial intelligence can have the status of a legal subject, mainly focusing on the discussion of ethics and morality, consciousness energy, social harm and other aspects. First of all, the future strong artificial intelligence has independent consciousness, that is to say, it has the ability to recognize and control, that is, equivalent to a mentally normal adult human. A conscious strong AI not only has a high IQ that exceeds the human level, but also has the same emotion as human beings. Once the artificial intelligence has consciousness and emotion, need to consider the moral factors, if the strong artificial intelligence in the process of interaction with human beings, persecution or abuse, with consciousness of artificial intelligence robots will be able to feel the pain, then need to give its legal subject status to protect its rights and interests. Secondly, the legal subject status is a concept of legal mimesis, which means that the law gives the subject the qualification to participate in the legal relationship. In criminal legal norms, criminal subjects are stipulated as natural persons and units. Natural persons naturally belong to the category of legal subjects, while the subject of a unit is a non-human “organization” that is given the status of a legal subject by the law through the method of mimesis and is liable for criminal responsibility because of the criminal activity it has committed. It can be seen that in order to facilitate the identification of responsibility, it is reasonable to make non-human “organizations” as legal subjects as natural persons, so it should also be reasonable to make artificial intelligence as legal subjects [5]. AI can also be the same as a legal person in the criminal legal relationship to bear the corresponding responsibility. Thirdly, giving strong AI the status of legal subject is conducive to solving a series of practical problems caused by AI. Because strong artificial intelligence has consciousness and has the ability to identify and control itself as adults, then strong artificial intelligence is as good or bad as human beings, and when it has a sense of harm to human beings, it will implement behaviors that harm human interests because of this awareness. In this case, if we simply follow the “instrumentalist” viewpoint, and treat it only as a tool for the human being who made it, and attribute the responsibility to the maker, then this deviates from the track of self-responsibility. The maker has no objective or subjective culpability for the damage and should not be held liable for vicarious liability [6]. Therefore, strong artificial intelligence is given the status of a legal subject and can be directly held liable for its own damages [7].

Negation of the Legal Subject Status of Strong Artificial Intelligence: Contrary to the affirmation of the legal subject of artificial intelligence, many scholars have also raised concerns about the status of artificial intelligence as a legal subject. First of all, some scholars believe that there is great uncertainty about whether artificial intelligence can have consciousness, and prefer that artificial intelligence can not have the unique consciousness of human beings [8]. This view stems from the doubt about the conditions required for AI to have consciousness, AI to have consciousness needs to meet many conditions, the most important of which is to have a simulated consciousness carrier similar to the human brain, but human beings still know very little about human body structure, and do not have a detailed grasp of the relationship between the human brain and consciousness, and even more impossible to produce it. Although there are research teams working on “brain-like” - a new research object that combines the results of brain science research and the artificial intelligence technology revolution - the results of their research are still far from being realized. Secondly, many scholars still insist that artificial intelligence is only a tool created by human beings to facilitate social life, the law is for human beings, only human beings can be the subject of the law, and artificial intelligence always belongs to the category of things, and can only be treated as an object in the law. If artificial intelligence is treated as a subject, it will harm the overall interests of human beings and lose the purpose of manufacturing robots. If in a case of property inheritance, the inheritor bequeathed his property to an AI robot through his will, instead of his children and descendants, this case is not wrong from the perspective of granting AI robots the status of a legal subject, but from the human-centered point of view, it greatly harms the property interests of the heir. On a hugely populated planet where human resources are scarce and unevenly distributed, the creation of artificially intelligent robots with subjectivity may be a waste of resources and an encroachment on human living space. Again, if AI robots are given the status of legal subjects, it is not necessarily enforceable from the perspective of assuming criminal responsibility. From the viewpoint of affirmative scholars, they believe that AI robots can independently assume their own criminal responsibility. But from the point of view of the purpose of the law to punish, the law punishes offenders with a preventive purpose and a retributive purpose. Preventive purpose is to prevent the offender from committing crimes again and to alert others to commit crimes, retributive purpose is to comply with the concept of “tooth for tooth, eye for eye”, based on the requirements of justice and the implementation of punishment [9]. Due to the natural difference between human beings and artificial intelligence robots, human beings cannot identify artificial intelligence as the same kind. It seems that giving the legal subject status to artificial intelligence robots and making them bear criminal responsibility, feel pain and repent can not achieve the expected purpose. Finally, whether it is a strong artificial intelligence robot or a weak artificial intelligence robot, it may become a tool used by human beings, due to the judicial organs can not judge whether the artificial intelligence robot is an active or passive crime, under the premise of granting the status of legal subject to artificial intelligence, there will be human beings who use the artificial intelligence robot as a tool to commit criminal acts and let the artificial intelligence robot substitute to take responsibility, and thus escape the legal sanctions of the situation. In addition, there is another refutation of the personality of the legal person in the unit crime. The legal person is mostly composed of natural persons, and the act of the legal person is actually carried out by the natural person, which makes the rights of the legal person can be enjoyed by the natural person, and the responsibility can be borne by the natural person. In contrast, an artificial intelligence is not a collection of natural persons, but a mechanical object made by a human being, for which it is impossible to formulate legal personality [10].

Weak AI Should Also have the Status of a Legal Subject: Contrary to the opinion of the majority of scholars, there are also a few scholars who believe that even weak artificial intelligence should have the status of a legal subject. First of all, some scholars believe that weak AI can also be the subject of rights in the subject of law. Legal subject is a broad concept, which contains the subject of rights, obligations and responsibilities. For weak artificial intelligence, even if it cannot become the subject of responsibility in the subject of law, it can also become the subject of rights. The main reason is that it believes that the enjoyment of rights does not need to have consciousness, for example, the fetus is given the status of legal subject through the method of legal fiction. Weak artificial intelligence and fetus also do not have consciousness, but can also become the subject of rights in the subject of law. Second, the weak artificial intelligence robot has the status of legal subject, even if it can not solve all the problems, but also can solve some of the social problems. Weak artificial intelligence robots in the legal subject status, it is possible to more directly share the responsibility arising from its behavior. Currently, liability for problems caused by artificial intelligence systems is usually borne by the manufacturer, developer, or user [11]. With legal subject status, robots are able to bear some of the responsibility independently, which contributes to a fairer distribution of legal responsibility. In addition, the recognition of legal subject status can encourage technology companies and research institutes to be more active in the research and development and innovation of AI technology, as robots with legal subject status may be easier to promote and be welcomed in the market.

Artificial Intelligence in the Current Era is Still Unable to Bear Criminal Liability as a Legal Subject

Through the above three classified analysis of the legal subject status of artificial intelligence, we can summarize from the perspective of reasonableness and practicality that the current legal subject status of artificial intelligence affirmation cannot be self-consistent, because the current era of artificial intelligence still belongs to the weak artificial intelligence, does not have the status of a legal subject, and can not enjoy rights and bear obligations, which means that it cannot be used as a legal subject to bear criminal responsibility. Although in the above three kinds of analysis, there is no single mention of the negative theory of weak artificial intelligence’s legal subject status, But logically speaking, the reason is consistent with the doctrine that strong AI does not have the status of legal subject. In this regard, the analysis of the reasons from the perspective of criminal law can be divided into the following three aspects [12].

AI in the Current Era Lack of Subjective Awareness: In criminal law, the conviction of an offender must meet certain standards, and the relevant theories are mainly the theory of criminal constitution and the theory of three elements, while the subjective element is an essential part common to both theories. Taking the theory of criminal constitution as an example, if a natural person commits the act of abetting a minor to commit a crime, he or she must have the intention of abetting subjectively, the subject meets the corresponding provisions of criminal liability capacity, and objectively commits the specific abetting criminal act and specifically infringes upon the object legal interests of the corresponding crime. In the cases where weak artificial intelligence causes crimes and causes harmful results, aside from the subject qualification of artificial intelligence, the objective behavior and the infringement on legal interests already meet the requirements of the elements of crime, but weak artificial intelligence does not have the subjective cognitive elements of crime. Based on the above explanation of the lack of material conditions of “human brain” carrying consciousness in artificial intelligence, weak artificial intelligence does not have the material basis of subjective consciousness, that is, it does not have intention and negligence for crimes subjectively. Take the generative artificial intelligence of weak artificial intelligence as an example, the generative artificial intelligence relies on a large amount of artificial input data pre-training, learning new knowledge from human behavioral feedback, and its behavioral pattern and content are completely derived from the projection of human behavior. The unconscious behavior of generative AI has caused humans to misinterpret it as conscious behavior, which is mainly attributable to the fact that people have not yet cracked the black-box algorithms of AI. However, the assumption of criminal responsibility is based on the constitution of a crime. Behavior that causes harmful results but lacks the subjective conditions of intent and negligence is not a criminal act, but belongs to the category of force majeure or accident, and is not criminally liable for the harmful results, but this is also for qualified legal subjects.

The Inability to Identify the Subject of the Harmful Act Creates an Imbalance in the Value of Fairness and Justice: To investigate the criminal responsibility of the harmful behavior that causes harmful results, it is necessary to specifically identify the subject of the behavior, but it is difficult to specifically identify the harmful behavior that causes harmful results by weak artificial intelligence, which makes it difficult to realize procedural and substantive fairness and justice in the process of accountability. First of all, the connotation of harmful acts requires that only the subject with consciousness under the domination of free will to implement the behavior of infringing on the interests of the law can constitute harmful acts. The characteristics of harmful acts require the unity of subjectivity and objectivity. Therefore, the lack of subjective consciousness of the weak artificial intelligence is unable to implement harmful acts, that is, unable to identify the weak artificial intelligence as the subject of harmful acts. Secondly, from the point of view of evidence collection, there is a great difficulty in the collection of evidence when generating criminal cases related to weak artificial intelligence. The artificial intelligence cannot bear the burden of proof, and it is difficult to define whether the real subject of criminal behavior implementation is the artificial intelligence or the user or producer of the artificial intelligence. Finally, from the perspective of rights and interests protection, artificial intelligence cannot be treated with the same procedures as natural persons in the practice of law enforcement and justice, which will ultimately lead to an imbalance between procedural justice and substantive justice. Only in the case of weak artificial intelligence as a user’s tool, when the user uses artificial intelligence technology or products to commit a criminal act, the subject of the act can be simply identified, and the real perpetrator behind the act cannot use artificial intelligence to escape from legal responsibility for him.

AI in the Current Era is not Capable of Being Penalized: As can be seen from the current system of penalties provided for in China’s Criminal Law, the types of execution are based on a combination of “physical” and “psychological” considerations. The death penalty and the sentence of liberty are mainly for the deprivation of the life and personal freedom of natural persons, and the property penalty is also for the deprivation of the material living conditions that are indirectly related to the offender’s body and mind. However, for generative artificial intelligence and other weak artificial intelligence, just physical change can be realized, but psychological condemnation lack of realization conditions. By insisting on the negative theory of the criminal subject of weak AI, we are affirming that it is “not a human being,” and that it is unable to have a psychological factor like a human being, i.e., it does not have the ability to bear criminal punishment. Moreover, according to the current penalty system, it is not meaningful to impose the death penalty, prison sentence, detention, fine and other penalties on weak AI. The purpose of punishment includes preventive purpose and punitive purpose. For the purpose of punishment, due to the lack of “perception” function of weak AI, it is impossible to realize the purpose of letting it suffer; and for the purpose of prevention, i.e., preventing the recurrence of crime, it is impossible to achieve the purpose of using the current system of penalties, because AI should be differentiated from the reform and prevention mechanism of human beings. In the case of natural persons using artificial intelligence to commit crimes, it is also necessary to transform artificial intelligence to prevent crimes, and adopt non-criminal technical means, such as deleting data and changing procedures.

Artificial Intelligence in the Current Era can Only be Treated as a Legal Object

Since, the artificial intelligence in current era does not have the status of a legal subject and cannot assume criminal responsibility, then in response to the occurrence of more and more weak artificial intelligence crimes, we can only treat artificial intelligence as an object in the law [13].

Creators and Users in Artificial Intelligence Crimes Remain Subjects of Law: Although the harm caused by weak AI results without the personal involvement and subjective intent of AI producers and users at the time of the crime, the producer or user should be clearly aware of his or her duty of care with respect to the subsequent behavior of the product at the beginning of the manufacture and use of the AI product. Therefore, in order to avoid the abuse of artificial intelligence by producers and users based on the pursuit of interests, producers and users, as legal subjects, should not only bear civil vicarious liability for their products, but also need to carry out special legislation to clarify the criminal liability of producers and users of artificial intelligence products [14]. On the one hand, when the producers, users use weak artificial intelligence as a tool to commit crimes against the interests of others, clear criminal liability by the producers, users. On the other hand, when weak artificial intelligence products cause harmful results outside the will of producers and users, it should be clear that producers and users bear specific criminal responsibility for negligent crimes. The theory of negligence attribution for producers of weak AI products has certain rationality. Although the weak artificial intelligence has no subjective consciousness, but it is still latent social hazards, and before the product is produced, the producer has certain foresight ability and foresight obligation. However, there should be a certain limit to the provision of criminal liability, and the special legislation should clarify the content and degree of “attention”, so that the producer can enjoy exemption from liability within a certain limit, so as to realize the common supervision and protection for both parties.

Artificial Intelligence as a Legal Object in Some Special Ways: On the basis that weak AI does not have the capacity for criminal responsibility and lacks the capacity to be punished, and excluding the use of the penalty enforcement method in the penal system to punish weak AI products, it seems that the harm of weak AI can only be managed from the perspective of the criminal responsibility of the producer. Actually, it’s not. As sentient material beings, we can solve problems only by treating weak AI products as objects in the law. For example, from the perspective of preventing the same path to cause harm again, we can implement degradation and transformation of weak AI products that have already caused harm, that is, to modify part of its program or delete data from human technology, and the modified program or deleted data is related to the part that caused the harmful results. Alternatively, from the perspective of the interests of the victim and society, the weak AI is subjected to the method of forced labor to compensate for the damage suffered by the victim or to increase benefits for society. This approach is based on the idea of “human-centeredness”, that artificial intelligence cannot be equated with human beings, and that weak artificial intelligence is created with the purpose of serving human beings, and should not be “turned against them”.