Interest in the ethics of Artificial Intelligence systems is dominated by the question of how these sorts of technologies will benefit or harm human individuals and societies. Much less attention is given to the ethics of our interaction with AI systems from the perspective of what may harm or benefit the systems themselves. Despite this, there is potential for future AI systems to be designed in a way that makes them either morally significant entities, or gives them the tools with which to develop degrees of moral significance, perhaps even personhood in the moral sense. This thesis proposes how certain contemporary paradigms in AI might in the future create a morally significant machine, perhaps even a machine person; one which can be harmed to a degree similar to ourselves. This type of system would be the first technology towards which the design of law and policy would be obliged to consider not just human best interests, but the best interests of the technology itself: how it is designed, what we can use it for, what can be done to it, and what we are duty-bound to provide it with. The thesis proposes a wide range of legal and social problems that the invention of such a system would engender, particularly in relation to paradigms like property, legal personality, and rights of both positive and negative nature. It also explores the fraught line-drawing problem of establishing which systems matter and which do not, and what the legal implications of this would be. It establishes that the net demands such a machine would place upon humans informs an argument that there should be a pre-emptive policy to prevent their creation, so as to mitigate harms to both human society and the machines themselves. When closely examined, the reality of a social partnership between persons – both human and machine – is too problematic and too profoundly challenging to the conception of anthropocentric hegemony to be justifiable
