Towards Validating Risk Indicators Based on Measurement Theory (Extended version)

Due to the lack of quantitative information and for cost-efficiency, most risk assessment methods use partially ordered values (e.g. high, medium, low) as risk indicators. In practice it is common to validate risk indicators by asking stakeholders whether they make sense. This way of validation is subjective, thus error prone. If the metrics are wrong (not meaningful), then they may lead system owners to distribute security investments inefficiently. For instance, in an extended enterprise this may mean over investing in service level agreements or obtaining a contract that provides a lower security level than the system requires. Therefore, when validating risk assessment methods it is important to validate the meaningfulness of the risk indicators that they use. In this paper we investigate how to validate the meaningfulness of risk indicators based on measurement theory. Furthermore, to analyze the applicability of the measurement theory to risk indicators, we analyze the indicators used by a risk assessment method specially developed for assessing confidentiality risks in networks of organizations

Elicitation of structured engineering judgement to inform a focussed FMEA

The practical use of Failure Mode and Effects Analysis (FMEA) has been criticised because it is often implemented too late and in a manner that does not allow information to be fed-back to inform the product design. Lessons learnt from the use of elicitation methods to gather structured expert judgement about engineering concerns for a new product design has led to an enhancement of the approach for implementing design and process FMEA. We refer to this variant as a focussed FMEA since the goal is to enable relevant engineers to contribute to the analysis and to act upon the outcomes in such a way that all activities focus upon the design needs. The paper begins with a review of the proposed process to identify and quantify engineering concerns. The pros and cons of using elicitation methods, originally designed to support construction of a Bayesian prior, to inform a focussed FMEA are analysed and a comparison of the proposed process in relation to the existing standards is made. An industrial example is presented to illustrate customisation of the process and discuss the impact on the design process

Towards Validating Risk Indicators Based on Measurement Theory

Due to the lack of quantitative information and for cost-efficiency purpose, most risk assessment methods use partially ordered values (e.g. high, medium, low) as risk indicators.\ud In practice it is common to validate risk scales by asking stakeholders whether they make sense. This way of validation is subjective, thus error prone. If the metrics are wrong (not meaningful), then they may lead system owners to distribute security investments inefficiently. Therefore, when validating risk assessment methods it is important to validate the meaningfulness of the risk scales that they use. In this paper we investigate how to validate the meaningfulness of risk indicators based on measurement theory. Furthermore, to analyze the applicability of measurement theory to risk indicators, we analyze the indicators used by a particular risk assessment method specially developed for assessing confidentiality risks in networks of organizations

Using Ontologies for the Design of Data Warehouses

Obtaining an implementation of a data warehouse is a complex task that forces designers to acquire wide knowledge of the domain, thus requiring a high level of expertise and becoming it a prone-to-fail task. Based on our experience, we have detected a set of situations we have faced up with in real-world projects in which we believe that the use of ontologies will improve several aspects of the design of data warehouses. The aim of this article is to describe several shortcomings of current data warehouse design approaches and discuss the benefit of using ontologies to overcome them. This work is a starting point for discussing the convenience of using ontologies in data warehouse design.Comment: 15 pages, 2 figure

Discourse through making: eliciting knowledge to support craft skills learning

In this paper I describe the development of techniques for eliciting craft knowledge for use in an interactive learning resource, illustrated with examples from two practical research projects. In each I explored the skills of both expert and novice craft practitioners, firstly in the field of traditional bowl turning and secondly in the field of traditional clog making. The result of this work was to highlight the personal, context-specific and highly tacit nature of such craft knowledge which is explained through a review of the writings of Michael Polanyi and Donald Schön. I conclude by redefining the concept of knowledge elicitation and transmission in this context. I describe the role of the designer-researcher as helping the expert to articulate their tacit knowledge through stimulating reciprocal reflection between the expert and a novice, and designing interpretation to help bridge the knowledge gap between the two.</p

Using Computational Agents to Design Participatory Social Simulations

In social science, the role of stakeholders is increasing in the development and use of simulation models. Their participation in the design of agent-based models (ABMs) has widely been considered as an efficient solution to the validation of this particular type of model. Traditionally, "agents" (as basic model elements) have not been concerned with stakeholders directly but via designers or role-playing games (RPGs). In this paper, we intend to bridge this gap by introducing computational or software agents, implemented from an initial ABM, into a new kind of RPG, mediated by computers, so that these agents can interact with stakeholders. This interaction can help not only to elicit stakeholders' informal knowledge or unpredicted behaviours, but also to control stakeholders' focus during the games. We therefore formalize a general participatory design method using software agents, and illustrate it by describing our experience in a project aimed at developing agent-based social simulations in the field of air traffic management.Participatory Social Simulations, Agent-Based Social Simulations, Computational Agents, Role-Playing Games, Artificial Maieutics, User-Centered Design

Validation of Expert Systems: Personal Choice Expert -- A Flexible Employee Benefit System

A method for validating expert systems, based on psychological validation literature and Turing\u27s imitation game, is applied to a flexible benefits expert system. Expert system validation entails determining if a difference exists between expert and novice decisions (construct validity), if the system uses the same inputs and processes to make its decisions as experts (content validity), and if the system produces the same results as experts (criterionrelated validity). If these criteria are satisfied, then the system is indistinguishable from experts for its domain and satisfies Turing\u27s imitation game. The methods developed in this paper are applied to a human resource expert system, Personal Choice Expert (PCE), designed to help employees choose a benefits package in a flexible benefits system. Expert and novice recommendations are compared to those generated by PCE. PCE\u27s recommendations do not significantly differ from those given by experts. High inter-expert agreement exists for some benefit recommendations (e.g. Dental Care and Long-Term Disability) but not for others (e.g. Short-Term Disability and Life Insurance). Insights offered by this method are illustrated and examined

ILR Research in Progress 2011-12

The production of scholarly research continues to be one of the primary missions of the ILR School. During a typical academic year, ILR faculty members published or had accepted for publication over 25 books, edited volumes, and monographs, 170 articles and chapters in edited volumes, numerous book reviews. In addition, a large number of manuscripts were submitted for publication, presented at professional association meetings, or circulated in working paper form. Our faculty's research continues to find its way into the very best industrial relations, social science and statistics journals.Research_in_Progress_2011_12.pdf: 46 downloads, before Oct. 1, 2020

ILR Research in Progress 2013-14

The production of scholarly research continues to be one of the primary missions of the ILR School. During a typical academic year, ILR faculty members published or had accepted for publication over 25 books, edited volumes, and monographs, 170 articles and chapters in edited volumes, numerous book reviews. In addition, a large number of manuscripts were submitted for publication, presented at professional association meetings, or circulated in working paper form. Our faculty's research continues to find its way into the very best industrial relations, social science and statistics journals.Research_in_Progress_2013_14.pdf: 54 downloads, before Oct. 1, 2020