Determinants of linear judgment: A meta-analysis of lens model studies

The mathematical representation of Brunswik’s lens model has been used extensively to study human judgment and provides a unique opportunity to conduct a meta-analysis of studies that covers roughly five decades. Specifically, we analyze statistics of the “lens model equation” (Tucker, 1964) associated with 259 different task environments obtained from 78 papers. In short, we find – on average – fairly high levels of judgmental achievement and note that people can achieve similar levels of cognitive performance in both noisy and predictable environments. Although overall performance varies little between laboratory and field studies, both differ in terms of components of performance and types of environments (numbers of cues and redundancy). An analysis of learning studies reveals that the most effective form of feedback is information about the task. We also analyze empirically when bootstrapping is more likely to occur. We conclude by indicating shortcomings of the kinds of studies conducted to date, limitations in the lens model methodology, and possibilities for future research.Judgment, lens model, linear models, learning, bootstrapping

Expert Elicitation for Reliable System Design

This paper reviews the role of expert judgement to support reliability assessments within the systems engineering design process. Generic design processes are described to give the context and a discussion is given about the nature of the reliability assessments required in the different systems engineering phases. It is argued that, as far as meeting reliability requirements is concerned, the whole design process is more akin to a statistical control process than to a straightforward statistical problem of assessing an unknown distribution. This leads to features of the expert judgement problem in the design context which are substantially different from those seen, for example, in risk assessment. In particular, the role of experts in problem structuring and in developing failure mitigation options is much more prominent, and there is a need to take into account the reliability potential for future mitigation measures downstream in the system life cycle. An overview is given of the stakeholders typically involved in large scale systems engineering design projects, and this is used to argue the need for methods that expose potential judgemental biases in order to generate analyses that can be said to provide rational consensus about uncertainties. Finally, a number of key points are developed with the aim of moving toward a framework that provides a holistic method for tracking reliability assessment through the design process.Comment: This paper commented in: [arXiv:0708.0285], [arXiv:0708.0287], [arXiv:0708.0288]. Rejoinder in [arXiv:0708.0293]. Published at http://dx.doi.org/10.1214/088342306000000510 in the Statistical Science (http://www.imstat.org/sts/) by the Institute of Mathematical Statistics (http://www.imstat.org

On heuristic and linear models of judgment: Mapping the demand for knowledge

Research on judgment and decision making presents a confusing picture of human abilities. For example, much research has emphasized the dysfunctional aspects of judgmental heuristics, and yet, other findings suggest that these can be highly effective. A further line of research has modeled judgment as resulting from “as if” linear models. This paper illuminates the distinctions in these approaches by providing a common analytical framework based on the central theoretical premise that understanding human performance requires specifying how characteristics of the decision rules people use interact with the demands of the tasks they face. Our work synthesizes the analytical tools of “lens model” research with novel methodology developed to specify the effectiveness of heuristics in different environments and allows direct comparisons between the different approaches. We illustrate with both theoretical analyses and simulations. We further link our results to the empirical literature by a meta-analysis of lens model studies and estimate both human and heuristic performance in the same tasks. Our results highlight the trade-off between linear models and heuristics. Whereas the former are cognitively demanding, the latter are simple to use. However, they require knowledge – and thus “maps” – of when and which heuristic to employ.Decision making; heuristics; linear models; lens model; judgmental biases

Designing as Interpretation

The paper suggests an interpretative approach to the empirical study of design processes. Design processes are conceived as social processes of interpretation and construction of meaning, and potentially of context generation. In contrast to models which conceive designing as a goal-directed process, an interpretative approach suggests a methodological reorientation. It assumes that design goals are more or less incomplete and vague at the beginning of a design process and are interpreted in contexts and in part are created by designers in the design process on the basis of their experience, embodied skills, and practices. The interpretative paradigm in design research seeks to observe, investigate, and describe practices that designers use in the process. Rather than attempting to determine and prescribe how practitioners ought to do their work, the research question is on how work is actually done - how interpretation is achieved by designers in particular design processes. An extract is analysed in some detail in the paper. These data are taken from the transcript of a case study of a design process in practice. Sociological and socio-linguistic (‘sensitizing’) concepts such as frames and contexts are adopted to describe and analyze some practices observed in the episodes. The paper focuses on an aspect of designing – various forms of involvement and stances designers’ take on in the meaning making process of interpretative design work. Interpretative analysis takes into account designers’ alignments which constitute “participation frameworks” and ground designers’ multimodal practices in different media (language, drawing, gesture). Goffman’s (1981) concept of “footing” is used to reveal more subtle shifts in stances that designers take in designing. Investigation of referential practices designers use in some utterances in the observed design conversation suggests that designers step into, displace, and position themselves in transformed, “keyed” situations to experience the solicitations of design situations more directly and to take the role of others as well as the role of objects. These practices appear to be part of designers’ ability to construct meaning by establishing perspectives and getting “maximal grip” on design situations so as to exert their skills. Analysis of types of stances designers take in an observed design process, some of which addressed in the paper, may provide a way to describe an aspect of designers’ artistry and to characterize the particularities of unique design processes. The suggested approach is intended to contribute to a better theoretical understanding of designing and to the methodology of design research as an ‘epistemology of practice’. Interpretative analysis also aims to provide description of designers’ practices which may, as its practical benefits, contribute to ‘the reflective turn’ in design research. Keywords: Design Research Methodology; Design Practices; Framing; Case Study</p

The design with intent method: A design tool for influencing user behaviour

The official published version can be found at the link below.Using product and system design to influence user behaviour offers potential for improving performance and reducing user error, yet little guidance is available at the concept generation stage for design teams briefed with influencing user behaviour. This article presents the Design with Intent Method, an innovation tool for designers working in this area, illustrated via application to an everyday human–technology interaction problem: reducing the likelihood of a customer leaving his or her card in an automatic teller machine. The example application results in a range of feasible design concepts which are comparable to existing developments in ATM design, demonstrating that the method has potential for development and application as part of a user-centred design process

Research and Applications of the Processes of Performance Appraisal: A Bibliography of Recent Literature, 1981-1989

[Excerpt] There have been several recent reviews of different subtopics within the general performance appraisal literature. The reader of these reviews will find, however, that the accompanying citations may be of limited utility for one or more reasons. For example, the reference sections of these reviews are usually composed of citations which support a specific theory or practical approach to the evaluation of human performance. Consequently, the citation lists for these reviews are, as they must be, highly selective and do not include works that may have only a peripheral relationship to a given reviewer\u27s target concerns. Another problem is that the citations are out of date. That is, review articles frequently contain many citations that are fifteen or more years old. The generation of new studies and knowledge in this field occurs very rapidly. This creates a need for additional reference information solely devoted to identifying the wealth of new research, ideas, and writing that is changing the field

Reinventing learning: a design-research odyssey

Design research is a broad, practice-based approach to investigating problems of education. This approach can catalyze the development of learning theory by fostering opportunities for transformational change in scholars’ interpretation of instructional interactions. Surveying a succession of design-research projects, I explain how challenges in understanding students’ behaviors promoted my own recapitulation of a historical evolution in educators’ conceptualizations of learning—Romantic, Progressivist, and Synthetic (Schön, Intuitive thinking? A metaphor underlying some ideas of educational reform (working paper 8). Division for Study and Research in Education, MIT, Cambridge, 1981)—and beyond to a proposed Systemic view. In reflection, I consider methodological adaptations to design-research practice that may enhance its contributions in accord with its objectives

Improving The Service Design Process: Process Integration, Conflict Reduction And Customer Involvement

Service design is the science of creating service experiences based on the customer’s perspective, to make it useful, enjoyable and cost-effective for the customer. Although the field of service design is relatively new, it has been rapidly expanding in research and practice. Most researchers focus on the usefulness of the service, cost efficiency, meeting customers’ needs, or service strategy. However, all service elements can benefit from improving the service design process. Current service design processes are suffering a lack of integration of activities, conflicts in decision-making processes, and exclusion of practitioners’ methods. In prior research, information models were created to integrate the service design process across the enterprise. As an extension, this dissertation introduces Petri Nets to improve the service design process. Petri Nets provide a uniform environment for modeling, analysis, and design of discrete event systems. Petri Nets are used to develop a new service design process that enhances the multidisciplinary approach and includes the practitioner methods. Additionally, this dissertation uses the Lens Model to improve the decision-making mechanism. The Lens Model is to characterize decision-making policy in service design. Research shows that there is a conflict between the designer and the manager in service design decision-making. Single Lens Model systems are designed to capture the decision policy for the service designer and the service manager. A double Lens Model system is used to compare the perspectives. Finally, this research suggests a new role for the customer in the design by applying an Asset-Based approach. Asset-based System Engineering (ABSE) is a recently introduced concept that attempts to synthesize systems around their key assets and strengths. ABSE is developed with as an innovative approach that views customers as a primary asset. Customer integration in the design process is achieved through several new service design tools