IEEE International Conference On Systems, Man, and Cybernetics (SMC)
Abstract
Shared control, where a human user cooperates with
an algorithm to operate a device, has the potential to greatly
expand access to powered mobility, but also raises unique ethical
challenges. A shared-control wheelchair may perform actions that
do not reflect its user’s intent in order to protect their safety,
causing frustration or distrust in the process. Unlike physical
accidents there is currently no framework for investigating or
adjudicating these events, leading to a reduced capability to
improve the shared control algorithm’s user experience. In this
paper we suggest a system based on the idea of an ‘ethical black
box’ that records the sensor context of sub-critical disagreements
and collision risks in order to allow human investigators to
examine them in retrospect and assess whether the algorithm
has taken control from the user without justification