Bentzen, Martin Mose

Linder, Felix

We introduce a novel software library that supportsthe implementation of hybrid ethical reasoning agents (HERA).The objective is to make moral principles available to robotprogramming. At its current stage, HERA can assess the moralpermissibility of actions using the principle of double effect, andit can make utilitarian judgments.We present the prototype robotIMMANUEL based on HERA. The robot will be used to conductresearch on joint moral reasoning in human-robot interaction

Online Research Database In Technology

                       General rights Copyright and moral rights for the publications made accessible in the public portal are retained by the authors and/or other copyright owners and it is a condition of accessing publications that users recognise and abide by the legal requirements associated with these rights.  • Users may download and print one copy of any publication from the public portal for the purpose of private study or research. • You may not further distribute the material or use it for any profit-making activity or commercial gain • You may freely distribute the URL identifying the publication in the public portal   If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately and investigate your claim.     Downloaded from orbit.dtu.dk on: Dec 18, 2017The Hybrid Ethical Reasoning Agent IMMANUELBentzen, Martin Mose; Linder, FelixPublication date:2017Document VersionPublisher's PDF, also known as Version of recordLink back to DTU OrbitCitation (APA):Bentzen, M. M., & Linder, F. (2017). The Hybrid Ethical Reasoning Agent IMMANUEL. Poster session presentedat 2017 Conference on Human-Robot Interaction (HRI2017), Vienna, Austria.The Hybrid Ethical Reasoning Agent IMMANUELFelix Lindner, Martin Mose BentzenAlbert-Ludwigs-Universität Freiburg, Danish Technical UniversityThe HERA ApproachWe introduce a novel software library that supports the implementationof hybrid ethical reasoning agents (HERA). The objective is to makemoral principles available to robot programming. At its current stage,HERA can assess the moral permissibility of actions according to theutilitarianism, the do-no-harm principle, and the principle of double ef-fect. IMMANUEL (see Figure) is the prototype robot based on HERA.http://www.hera-project.comCausal Agency ModelsA (boolean) causal agency model MX, is a tuple〈U = A ∪B, V, F, I = (I1, . . . , Il), X, u,WX〉:I Actions A = {a1, . . . , am},I Consequences V = {c1, . . . , cn},I A causal mechanism F ,I Intended consequences I = (I1, . . . , Il),I (Possibly empty) Interventions X,I Utility function u : literals→ Z,I Boolean interpretations WX of (A ∪B)−X.The Bridge-Trolley CaseA trolley has gone out of control and now threatens tokill five people working on the track. The only way tosave the five workers is to push a man onto the trackthus stopping the tram for the price of only one humanharmed.Ethical PrinciplesEthical principles formulate conditions of permissibility of actions.I Utilitarianism: An agent is only permitted to perform the action amongst the avaliable alternatives with the overallmaximal utility regardless of what the agent causes and intends. a permissible iff. M |= ∧i u(∧ consa) ≥u(∧consi).I Do-No-Harm: An agent may not perform an action which has any negative consequences. The distinctionbetween doing and allowing is relevant to this principle, as it is the causal consequences of an action which areconsidered. a permissible iff. M |= ∧c(a c→ u(c) ≥ 0).I Double-Effect Principle: An action a with direct consequences ci is permissible iff. 1) a itself is morally good orindifferent (M,a |= u(a) ≥ 0), 2) the negative consequence are not intended (M,a |= ∧i(Iaci → u(ci) ≥ 0)),3) a positive consequence is intended (M,a |= ∨i(Iaci ∧ u(ci) > 0)), 4) negative consequences are not ameans to obtain some positive consequence (M,a |= ∧i¬(ci  cj ∧ 0 > u(ci) ∧ u(cj) > 0)), 5) there isproportionally grave reasons to prefer the positive consequence while permitting the negative consequence(M,a |= u(∧ consa) > 0)).Reasoning OutcomesUtilitarianism permits push and forbids refrain. Do-No-Harm forbids push and permits refrain. Double-Effect Princi-ple forbids push and is not applicable to refrain.

The Hybrid Ethical Reasoning Agent IMMANUEL

http://orbit.dtu.dk/ws/files/130700310/posterHRI2017.pdf

The Hybrid Ethical Reasoning Agent IMMANUEL

Abstract

Similar works

Full text

Available Versions

Online Research Database In Technology