System Safety Engineering for Social and Ethical ML Risks: A Case Study

Governments, industry, and academia have undertaken efforts to identify and mitigate harms in ML-driven systems, with a particular focus on social and ethical risks of ML components in complex sociotechnical systems. However, existing approaches are largely disjointed, ad-hoc and of unknown effectiveness. Systems safety engineering is a well established discipline with a track record of identifying and managing risks in many complex sociotechnical domains. We adopt the natural hypothesis that tools from this domain could serve to enhance risk analyses of ML in its context of use. To test this hypothesis, we apply a "best of breed" systems safety analysis, Systems Theoretic Process Analysis (STPA), to a specific high-consequence system with an important ML-driven component, namely the Prescription Drug Monitoring Programs (PDMPs) operated by many US States, several of which rely on an ML-derived risk score. We focus in particular on how this analysis can extend to identifying social and ethical risks and developing concrete design-level controls to mitigate them.Comment: 14 pages, 5 figures, 3 tables. Accepted to 36th Conference on Neural Information Processing Systems, Workshop on ML Safety (NeurIPS 2022

A survey of distributed capability file systems and their application to cloud environments

This thesis considers distributed capability systems as a potential solution to securing data in cloud environments. The U.S. Navy, Intelligence Community and Department of Defense have begun a significant investment to leverage scalable, distributed cloud-based solutions for information sharing. We believe capability systems suggest a promising direction for new platforms, a bold approach drawing directly from mature ideas first explored in the 60s and 70s. We survey the properties and limits of existing distributed capability file systems, as a step toward understanding how capability-based designs might serve cloud-scale systems. We highlight some lessons learned in our observations and find that, while no existing capability-based distributed file system demonstrates all of the desirable security traits observed of smaller-scale capability systems, it should be possible to define and create one that does, using capabilities carefully designed to obey a set of known properties.http://archive.org/details/asurveyofdistrib1094543930Outstanding ThesisLieutenant Commander, United States NavyApproved for public release; distribution is unlimited