The Responsibility Quantification (ResQu) Model of Human Interaction with Automation

Intelligent systems and advanced automation are involved in information collection and evaluation, in decision-making and in the implementation of chosen actions. In such systems, human responsibility becomes equivocal. Understanding human casual responsibility is particularly important when intelligent autonomous systems can harm people, as with autonomous vehicles or, most notably, with autonomous weapon systems (AWS). Using Information Theory, we develop a responsibility quantification (ResQu) model of human involvement in intelligent automated systems and demonstrate its applications on decisions regarding AWS. The analysis reveals that human comparative responsibility to outcomes is often low, even when major functions are allocated to the human. Thus, broadly stated policies of keeping humans in the loop and having meaningful human control are misleading and cannot truly direct decisions on how to involve humans in intelligent systems and advanced automation. The current model is an initial step in the complex goal to create a comprehensive responsibility model, that will enable quantification of human causal responsibility. It assumes stationarity, full knowledge regarding the characteristic of the human and automation and ignores temporal aspects. Despite these limitations, it can aid in the analysis of systems designs alternatives and policy decisions regarding human responsibility in intelligent systems and advanced automation

Theoretical, Measured and Subjective Responsibility in Aided Decision Making

When humans interact with intelligent systems, their causal responsibility for outcomes becomes equivocal. We analyze the descriptive abilities of a newly developed responsibility quantification model (ResQu) to predict actual human responsibility and perceptions of responsibility in the interaction with intelligent systems. In two laboratory experiments, participants performed a classification task. They were aided by classification systems with different capabilities. We compared the predicted theoretical responsibility values to the actual measured responsibility participants took on and to their subjective rankings of responsibility. The model predictions were strongly correlated with both measured and subjective responsibility. A bias existed only when participants with poor classification capabilities relied less-than-optimally on a system that had superior classification capabilities and assumed higher-than-optimal responsibility. The study implies that when humans interact with advanced intelligent systems, with capabilities that greatly exceed their own, their comparative causal responsibility will be small, even if formally the human is assigned major roles. Simply putting a human into the loop does not assure that the human will meaningfully contribute to the outcomes. The results demonstrate the descriptive value of the ResQu model to predict behavior and perceptions of responsibility by considering the characteristics of the human, the intelligent system, the environment and some systematic behavioral biases. The ResQu model is a new quantitative method that can be used in system design and can guide policy and legal decisions regarding human responsibility in events involving intelligent systems

Sociotechnical imaginaries of connected and automated vehicle technology: Comparative analysis of governance cultures in Finland, Germany, and the UK

As an emerging technology, the potential deployment of connected and automated vehicles (CAVs) in cities is attributed with significant uncertainties and anticipated consequences requiring responsible governance of innovation processes. Despite a growing number of studies on policies and governance arrangements for managing the introduction of CAVs, there is a gap in understanding about country-specific governance strategies and approaches. This research aims to contribute addressing this gap by presenting a comparative analysis of CAV-related policy documents in Finland, UK, and Germany, three countries which are actively seeking to promote the introduction of CAVs and which have distinct administrative traditions. Our analytical framework is based on the set of premises about technology as a complex socio-technical phenomenon, operationalized using governance cultures and sociotechnical imaginaries concepts. Our comparative policy document analysis focuses on the assumed roles for CAV technology, the identified domains and mechanisms of governance, and the assumed actors responsible for steering the development process. The results highlight similarities in pro-automation values across three different countries, while also uncovering important differences outside the domain of traditional transport policy instruments. In addition, the results identify different types of potential technological determinism, which could restrict opportunities for responsiveness and divergent visions of mobility futures in Europe. Concluding with a warning against further depolitization of technological development and a dominant focus on economic growth, we identify several necessary directions for further developing governance and experimentation processes

Les collaborations interarmées en contexte opérationnel : l’exemple de l’utilisation des véhicules aériens sans pilotes

Travail dirigé présenté à la Faculté des études supérieures et postdoctorales en vue de l’obtention du grade de Maître ès sciences (M.Sc) en Criminologie – Option sécurité intérieureL’utilisation des véhicules aériens sans pilote lors d’opérations militaires à l’étranger se fait grandissante. Les forces armées membres de l’Organisation du Traité de l’Atlantique Nord ont perçu l’absolue nécessité d’une surveillance des groupes d’insurgés situés outre-mer. Ces technologies modernes, étant exploitées par les forces opérationnelles d’un grand nombre de nations, demandent une étroite collaboration entre militaires d’une même coalition. De ce fait, la littérature soutient que les collaborations interarmées tendent à être influencées par l’usage des drones, ces nouvelles technologies menant soit à une meilleure intégration des pratiques entre militaires de pays alliés ou bien à une séparation nette des tâches et des responsabilités diminuant ainsi la collaboration entre les troupes de diverses nations. En plus de cette absence de consensus émanant de la littérature, les perceptions des militaires canadiens sur ce sujet sont absentes des écrits académiques. Par conséquent, cette étude de cas porte sur l’expérience de onze militaires des Forces armées canadiennes ayant été déployés à l’étranger et ayant travaillé avec des drones en coopération avec d’autres nations. Les résultats de ce travail dirigé mettent de l’avant, entre autres, l’importance des suggestions pour une meilleure intégration du travail lorsqu’il est question de l’usage des drones. De plus, les militaires sondés affirment qu’une séparation des tâches existe lors de mission impliquant des véhicules aériens, mais que cette réalité est perçue comme essentielle à la réussite d’une opération. Le travail collaboratif est toutefois complexifié par des règles d’engagement discordantes entre les pays ainsi que par un éloignement géographique des militaires analysant les données collectées par les drones.The use of unmanned aerial vehicles in military operations is growing. The armed forces who are members of the North Atlantic Treaty Organization have perceived an absolute necessity of monitoring oversea insurgent groups. These modern technologies, exploited by the operational forces of many nations, require close collaboration between the military forces of a same coalition. Consequently, the literature maintains that international collaborations tend to be influenced by the use of drones, new technologies leading to either a better integration of military practices of allied countries or a clear separation of duties and responsibilities, which decreases collaboration between the troops of various nations. In addition to this lack of consensus in the literature, perceptions of Canadian military personnel on this issue are absent from academic writings. Thus, this case study examines the experience of eleven Canadian Forces members who have been deployed overseas and have worked with drones in cooperation with other nations. The results of this research show, among other things, the importance of feedback for better work integration when it comes to the use drones. Moreover, the military interviewed stated that segregation of duties exists on missions involving unmanned aerial vehicles, but this reality is perceived as essential to the success of an operation. Collaborative work is, however, complicated due to conflicting rules of engagement between countries as well as a geographical remoteness of military personnel analyzing data collected by drones

Disassembling the Trust Machine, three cuts on the political matter of blockchain

Blockchain technology is, in part, a proposal to resolve ‘the political’ through technical means: decentralised networks to solve the problem of authority; cryptography to coordinate and secure the network; and game theory and incentive design to solve network behaviour. This PhD thesis draws on theoretical work by Karen Barad (2007) and Jacques Rancière (Rancière, 2010) to ask the question of what matters politically in blockchain technology – both in the sense of matter as becoming material of a new mediation of the political, but also mattering in the sense of being of political importance to engineers, developers and communities forming around blockchain as a potential. Rather than treating blockchain as coherent thing to be either celebrated or criticised, this thesis proposes and attempts to draw out the ways in which the potentials of blockchain are negotiated as part of its political effects, looking towards these negotiations to understand how political differences are made and sought materialised. Three approaches to the political are articulated to analyse Bitcoin and Ethereum as case studies and shift their terms of debate. Firstly, addressing the question of algorithmic determinacy, an approach is proposed for critically understanding a blockchain proposition that does not immediately revert to a competition of control between ‘human’ and ‘machine’ through the notion of the insensible, drawing on work by geographer of the inhuman Yusoff (2013a). Secondly, drawing on political theorist Rancière (2010) a particular blockchain sensibility is articulated, addressing the question of the particular kind of ‘disruption’ that blockchain presents. Its specific provenance in political histories of decentralised network computation opens up political significance beyond its intersections with financial capitalism. Finally, addressing the question of blockchain as a resolution to the political, the thesis introduces the concept of dissensible as an ongoing potential for incompatible sensibilities and their negotiation