Algorithms that Remember: Model Inversion Attacks and Data Protection Law

Many individuals are concerned about the governance of machine learning systems and the prevention of algorithmic harms. The EU's recent General Data Protection Regulation (GDPR) has been seen as a core tool for achieving better governance of this area. While the GDPR does apply to the use of models in some limited situations, most of its provisions relate to the governance of personal data, while models have traditionally been seen as intellectual property. We present recent work from the information security literature around `model inversion' and `membership inference' attacks, which indicate that the process of turning training data into machine learned systems is not one-way, and demonstrate how this could lead some models to be legally classified as personal data. Taking this as a probing experiment, we explore the different rights and obligations this would trigger and their utility, and posit future directions for algorithmic governance and regulation.Comment: 15 pages, 1 figur

Acoustic-channel attack and defence methods for personal voice assistants

Personal Voice Assistants (PVAs) are increasingly used as interface to digital environments. Voice commands are used to interact with phones, smart homes or cars. In the US alone the number of smart speakers such as Amazon’s Echo and Google Home has grown by 78% to 118.5 million and 21% of the US population own at least one device. Given the increasing dependency of society on PVAs, security and privacy of these has become a major concern of users, manufacturers and policy makers. Consequently, a steep increase in research efforts addressing security and privacy of PVAs can be observed in recent years. While some security and privacy research applicable to the PVA domain predates their recent increase in popularity and many new research strands have emerged, there lacks research dedicated to PVA security and privacy. The most important interaction interface between users and a PVA is the acoustic channel and acoustic channel related security and privacy studies are desirable and required. The aim of the work presented in this thesis is to enhance the cognition of security and privacy issues of PVA usage related to the acoustic channel, to propose principles and solutions to key usage scenarios to mitigate potential security threats, and to present a novel type of dangerous attack which can be launched only by using a PVA alone. The five core contributions of this thesis are: (i) a taxonomy is built for the research domain of PVA security and privacy issues related to acoustic channel. An extensive research overview on the state of the art is provided, describing a comprehensive research map for PVA security and privacy. It is also shown in this taxonomy where the contributions of this thesis lie; (ii) Work has emerged aiming to generate adversarial audio inputs which sound harmless to humans but can trick a PVA to recognise harmful commands. The majority of work has been focused on the attack side, but there rarely exists work on how to defend against this type of attack. A defence method against white-box adversarial commands is proposed and implemented as a prototype. It is shown that a defence Automatic Speech Recognition (ASR) can work in parallel with the PVA’s main one, and adversarial audio input is detected if the difference in the speech decoding results between both ASR surpasses a threshold. It is demonstrated that an ASR that differs in architecture and/or training data from the the PVA’s main ASR is usable as protection ASR; (iii) PVAs continuously monitor conversations which may be transported to a cloud back end where they are stored, processed and maybe even passed on to other service providers. A user has limited control over this process when a PVA is triggered without user’s intent or a PVA belongs to others. A user is unable to control the recording behaviour of surrounding PVAs, unable to signal privacy requirements and unable to track conversation recordings. An acoustic tagging solution is proposed aiming to embed additional information into acoustic signals processed by PVAs. A user employs a tagging device which emits an acoustic signal when PVA activity is assumed. Any active PVA will embed this tag into their recorded audio stream. The tag may signal a cooperating PVA or back-end system that a user has not given a recording consent. The tag may also be used to trace when and where a recording was taken if necessary. A prototype tagging device based on PocketSphinx is implemented. Using Google Home Mini as the PVA, it is demonstrated that the device can tag conversations and the tagging signal can be retrieved from conversations stored in the Google back-end system; (iv) Acoustic tagging provides users the capability to signal their permission to the back-end PVA service, and another solution inspired by Denial of Service (DoS) is proposed as well for protecting user privacy. Although PVAs are very helpful, they are also continuously monitoring conversations. When a PVA detects a wake word, the immediately following conversation is recorded and transported to a cloud system for further analysis. An active protection mechanism is proposed: reactive jamming. A Protection Jamming Device (PJD) is employed to observe conversations. Upon detection of a PVA wake word the PJD emits an acoustic jamming signal. The PJD must detect the wake word faster than the PVA such that the jamming signal still prevents wake word detection by the PVA. An evaluation of the effectiveness of different jamming signals and overlap between wake words and the jamming signals is carried out. 100% jamming success can be achieved with an overlap of at least 60% with a negligible false positive rate; (v) Acoustic components (speakers and microphones) on a PVA can potentially be re-purposed to achieve acoustic sensing. This has great security and privacy implication due to the key role of PVAs in digital environments. The first active acoustic side-channel attack is proposed. Speakers are used to emit human inaudible acoustic signals and the echo is recorded via microphones, turning the acoustic system of a smartphone into a sonar system. The echo signal can be used to profile user interaction with the device. For example, a victim’s finger movement can be monitored to steal Android unlock patterns. The number of candidate unlock patterns that an attacker must try to authenticate herself to a Samsung S4 phone can be reduced by up to 70% using this novel unnoticeable acoustic side-channel

MODELLING VIRTUAL ENVIRONMENT FOR ADVANCED NAVAL SIMULATION

This thesis proposes a new virtual simulation environment designed as element of an interoperable federation of simulator to support the investigation of complex scenarios over the Extended Maritime Framework (EMF). Extended Maritime Framework is six spaces environment (Underwater, Water surface, Ground, Air, Space, and Cyberspace) where parties involved in Joint Naval Operations act. The amount of unmanned vehicles involved in the simulation arise the importance of the Communication modelling, thus the relevance of Cyberspace. The research is applied to complex cases (one applied to deep waters and one to coast and littoral protection) as examples to validate this approach; these cases involve different kind of traditional assets (e.g. satellites, helicopters, ships, submarines, underwater sensor infrastructure, etc.) interact dynamically and collaborate with new autonomous systems (i.e. AUV, Gliders, USV and UAV). The use of virtual simulation is devoted to support validation of new concepts and investigation of collaborative engineering solutions by providing a virtual representation of the current situation; this approach support the creation of dynamic interoperable immersive framework that could support training for Man in the Loop, education and tactical decision introducing the Man on the Loop concepts. The research and development of the Autonomous Underwater Vehicles requires continuous testing so a time effective approach can result a very useful tool. In this context the simulation can be useful to better understand the behaviour of Unmanned Vehicles and to avoid useless experimentations and their costs finding problems before doing them. This research project proposes the creation of a virtual environment with the aim to see and understand a Joint Naval Scenario. The study will be focusing especially on the integration of Autonomous Systems with traditional assets; the proposed simulation deals especially with collaborative operation involving different types of Autonomous Underwater Vehicles (AUV), Unmanned Surface Vehicles (USV) and UAV (Unmanned Aerial Vehicle). The author develops an interoperable virtual simulation devoted to present the overall situation for supervision considering also the sensor capabilities, communications and mission effectiveness that results dependent of the different asset interaction over a complex heterogeneous network. The aim of this research is to develop a flexible virtual simulation solution as crucial element of an HLA federation able to address the complexity of Extended Maritime Framework (EMF). Indeed this new generation of marine interoperable simulation is a strategic advantage for investigating the problems related to the operational use of autonomous systems and to finding new ways to use them respect to different scenarios. The research deal with the creation of two scenarios, one related to military operations and another one on coastal and littoral protection where the virtual simulation propose the overall situation and allows to navigate into the virtual world considering the complex physics affecting movement, perception, interaction and communication. By this approach, it becomes evident the capability to identify, by experimental analysis within the virtual world, the new solutions in terms of engineering and technological configuration of the different systems and vehicles as well as new operational models and tactics to address the specific mission environment. The case of study is a maritime scenario with a representation of heterogeneous network frameworks that involves multiple vehicles both naval and aerial including AUVs, USVs, gliders, helicopter, ships, submarines, satellite, buoys and sensors. For the sake of clarity aerial communications will be represented divided from underwater ones. A connection point for the latter will be set on the keel line of surface vessels representing communication happening via acoustic modem. To represent limits in underwater communications, underwater signals have been considerably slowed down in order to have a more realistic comparison with aerial ones. A maximum communication distance is set, beyond which no communication can take place. To ensure interoperability the HLA Standard (IEEE 1516 evolved) is adopted to federate other simulators so to allow its extensibility for other case studies. Two different scenarios are modelled in 3D visualization: Open Water and Port Protection. The first one aims to simulate interactions between traditional assets in Extended Maritime Framework (EMF) such as satellite, navy ships, submarines, NATO Research Vessels (NRVs), helicopters, with new generation unmanned assets as AUV, Gliders, UAV, USV and the mutual advantage the subjects involved in the scenario can have; in other word, the increase in persistence, interoperability and efficacy. The second scenario models the behaviour of unmanned assets, an AUV and an USV, patrolling a harbour to find possible threats. This aims to develop an algorithm to lead patrolling path toward an optimum, guaranteeing a high probability of success in the safest way reducing human involvement in the scenario. End users of the simulation face a graphical 3D representation of the scenario where assets would be represented. He can moves in the scenario through a Free Camera in Graphic User Interface (GUI) configured to entitle users to move around the scene and observe the 3D sea scenario. In this way, players are able to move freely in the synthetic environment in order to choose the best perspective of the scene. The work is intended to provide a valid tool to evaluate the defencelessness of on-shore and offshore critical infrastructures that could includes the use of new technologies to take care of security best and preserve themselves against disasters both on economical and environmental ones

The Dark Web: Cyber-Security Intelligence Gathering Opportunities, Risks and Rewards

We offer a partial articulation of the threats and opportunities posed by the so-called Dark Web (DW). We go on to propose a novel DW attack detection and prediction model. Signalling aspects are considered wherein the DW is seen to comprise a low cost signaling environment. This holds inherent dangers as well as rewards for investigators as well as those with criminal intent. Suspected DW perpetrators typically act entirely in their own self-interest (e.g. illicit financial gain, terrorism, propagation of extremist views, extreme forms of racism, pornography, and politics; so-called ‘radicalisation’). DWinvestigators therefore need to be suitably risk aware such that the construction of a credible legally admissible, robust evidence trail does not expose investigators to undue operational or legal risk

The future of Cybersecurity in Italy: Strategic focus area

This volume has been created as a continuation of the previous one, with the aim of outlining a set of focus areas and actions that the Italian Nation research community considers essential. The book touches many aspects of cyber security, ranging from the definition of the infrastructure and controls needed to organize cyberdefence to the actions and technologies to be developed to be better protected, from the identification of the main technologies to be defended to the proposal of a set of horizontal actions for training, awareness raising, and risk management

An emergent security risk : critical infrastructures and information warfare

This paper examines the emergent security risk that information warfare poses to critical infrastructure systems, particularly as governments are increasingly concerned with protecting these assets against attack or disruption. Initially it outlines critical infrastructure systems and the notion of information warfare. It then discusses the potential implications and examining the concerns and vulnerabilities such cyber attacks would pose, utilising exemplar online attack occurrences. It then examines the current Australian situation before suggesting some considerations to mitigate the potential risk that information warfare poses to critical infrastructure systems, and by association: government, industry and the wider community.<br /

State of the art 2015: a literature review of social media intelligence capabilities for counter-terrorism

Overview This paper is a review of how information and insight can be drawn from open social media sources. It focuses on the specific research techniques that have emerged, the capabilities they provide, the possible insights they offer, and the ethical and legal questions they raise. These techniques are considered relevant and valuable in so far as they can help to maintain public safety by preventing terrorism, preparing for it, protecting the public from it and pursuing its perpetrators. The report also considers how far this can be achieved against the backdrop of radically changing technology and public attitudes towards surveillance. This is an updated version of a 2013 report paper on the same subject, State of the Art. Since 2013, there have been significant changes in social media, how it is used by terrorist groups, and the methods being developed to make sense of it.&nbsp; The paper is structured as follows: Part 1 is an overview of social media use, focused on how it is used by groups of interest to those involved in counter-terrorism. This includes new sections on trends of social media platforms; and a new section on Islamic State (IS). Part 2 provides an introduction to the key approaches of social media intelligence (henceforth ‘SOCMINT’) for counter-terrorism. Part 3 sets out a series of SOCMINT techniques. For each technique a series of capabilities and insights are considered, the validity and reliability of the method is considered, and how they might be applied to counter-terrorism work explored. Part 4 outlines a number of important legal, ethical and practical considerations when undertaking SOCMINT work

Looking towards the future: the changing nature of intrusive surveillance and technical attacks against high-profile targets

In this thesis a novel Bayesian model is developed that is capable of predicting the probability of a range of eavesdropping techniques deployed, given an attacker's capability, opportunity and intent. Whilst limited attention by academia has focused on the cold war activities of Soviet bloc and Western allies' bugging of embassies, even less attention has been paid to the changing nature of the technology used for these eavesdropping events. This thesis makes four contributions: through the analysis of technical eavesdropping events over the last century, technological innovation is shown to have enriched the eavesdropping opportunities for a range of capabilities. The entry barrier for effective eavesdropping is lowered, while for the well resourced eavesdropper, the requirement for close access has been replaced by remote access opportunities. A new way to consider eavesdropping methods is presented through the expert elicitation of capability and opportunity requirements for a range of present-day eavesdropping techniques. Eavesdropping technology is shown to have life-cycle stages with the technology exploited by different capabilities at different times. Three case studies illustrate that yesterday’s secretive government method becomes today’s commodity. The significance of the egress transmission path is considered too. Finally, by using the expert elicitation information derived for capability, opportunity and life-cycle position, for a range of eavesdropping techniques, it is shown that it is possible to predict the probability of particular eavesdropping techniques being deployed. This novel Bayesian inferencing model enables scenarios with incomplete, uncertain or missing detail to be considered. The model is validated against the previously collated historic eavesdropping events. The development of this concept may be scaled with additional eavesdropping techniques to form the basis of a tool for security professionals or risk managers wishing to define eavesdropping threat advice or create eavesdropping policies based on the rigour of this technological study.Open Acces

Protecting the infrastructure: 3rd Australian information warfare & security conference 2002

The conference is hosted by the We-B Centre (working with a-business) in the School of Management Information System, the School of Computer & Information Sciences at Edith Cowan University. This year\u27s conference is being held at the Sheraton Perth Hotel in Adelaide Terrace, Perth. Papers for this conference have been written by a wide range of academics and industry specialists. We have attracted participation from both national and international authors and organisations. The papers cover many topics, all within the field of information warfare and its applications, now and into the future. The papers have been grouped into six streams: • Networks • IWAR Strategy • Security • Risk Management • Social/Education • Infrastructur