Sustainability in design: now! Challenges and opportunities for design research, education and practice in the XXI century

Copyright @ 2010 Greenleaf PublicationsLeNS project funded by the Asia Link Programme, EuropeAid, European Commission

Open Challenges in Modelling, Analysis and Synthesis of Human Behaviour in Human--Human and Human--Machine Interactions

Modelling, analysis and synthesis of behaviour are the subject of major efforts in computing science, especially when it comes to technologies that make sense of human–human and human–machine interactions. This article outlines some of the most important issues that still need to be addressed to ensure substantial progress in the field, namely (1) development and adoption of virtuous data collection and sharing practices, (2) shift in the focus of interest from individuals to dyads and groups, (3) endowment of artificial agents with internal representations of users and context, (4) modelling of cognitive and semantic processes underlying social behaviour and (5) identification of application domains and strategies for moving from laboratory to the real-world products

Managing creativity in magazine publishing: the 4Ps of creativity

With deep historical roots in culture and even myth, ‘creativity’ travels from its first formal English dictionary entry at the end of 19th Century (Kaufman & Glaveneau 2019) to becoming central in the re-branding of cultural industries as creative (Hesmondhalgh & Baker 2011) in the proceeding century. However, despite being studied with academic rigour since the 1950s (Sawyer 2012), there is no agreed theory of creativity (Sternberg & Lubart 1999, Hennessey & Amabile 2010). Given this epic rise in its cultural and industrial importance, there have been many attempts in recent decades at explaining the nature of creativity, including whether it can be aided or even managed from an economic perspective. This thesis aims to join this this specific and on-going challenge by looking at creativity in the industrial context of magazine publishing: an established cultural industry and one on the cusp of what could be called ‘digital change’ in the last decade. In an attempt at adding to the definitional and methodological understanding of creativity in media, this thesis adopts a componential or holistic ‘4 Ps’ conceptualisation (first discussed by Mel Rhodes, 1961), by considering creativity in People, Process, Place and Products. A key element of this work, the ‘4Ps’ model developed is used to interpret a theorised relationship between four variables by way of proximal ‘measures’ – ones instrumentalised by reviewing cultural theory (such as genius and ‘Big C’ creativity of Weisburg 1993 and Csikszentmihaly 1988) and the social psychology confluence theories of ‘Small C’, everyday creativity (Sternberg & Lubart 1991, Amabile 1983, 1996). Through qualitative research methods, individual and group interviews, as well as organisational autoethnographic accounts from magazine publishers, editors, entrepreneurs and employees (including two ‘historical’ magazines cases), nine case study findings have been used to form structured qualitative datasets, aiding interpretation of interrelatedness of the ‘4 Ps of creativity’. A primary contribution of the thesis is therefore located within the ontological contention of empirical ‘measurement’ – and its approach to creativity judgement in the fields of cultural studies, the emerging field of media management (Malmelin & Virta, 2016, Virta & Malmelin 2017) and creative industries study (Kung 2008a, 2008b). In addition to the research’s approach, the creativity and media management insights aim to shine a light on an industry facing an existential threat from the digital shift. Where magazines today are shown to perhaps need less ‘heroic’ personality-defined People creativity (unlike the 1990s), creativity in the digital era is important in other ways. In nuancing different types of ‘Big C’ and ‘Small C’ creativity in contemporary magazine work, the study makes a case for adapting creativity management models such as Amabile’s (1996 and Amabile & Pratt 2016) and Tan (1998) by focusing on Process aspects of skills and knowledge development, aided through differentiated environmental Place culture factors suited to magazine genres, clients and audiences in changing fields with new domains of knowledge

Enhancing interaction in mixed reality

With continuous technological innovation, we observe mixed reality emerging from research labs into the mainstream. The arrival of capable mixed reality devices transforms how we are entertained, consume information, and interact with computing systems, with the most recent being able to present synthesized stimuli to any of the human senses and substantially blur the boundaries between the real and virtual worlds. In order to build expressive and practical mixed reality experiences, designers, developers, and stakeholders need to understand and meet its upcoming challenges. This research contributes a novel taxonomy for categorizing mixed reality experiences and guidelines for designing mixed reality experiences. We present the results of seven studies examining the challenges and opportunities of mixed reality experiences, the impact of modalities and interaction techniques on the user experience, and how to enhance the experiences. We begin with a study determining user attitudes towards mixed reality in domestic and educational environments, followed by six research probes that each investigate an aspect of reality or virtuality. In the first, a levitating steerable projector enables us to investigate how the real world can be enhanced without instrumenting the user. We show that the presentation of in-situ instructions for navigational tasks leads to a significantly higher ability to observe and recall real-world landmarks. With the second probe, we enhance the perception of reality by superimposing information usually not visible to the human eye. In amplifying the human vision, we enable users to perceive thermal radiation visually. Further, we examine the effect of substituting physical components with non-functional tangible proxies or entirely virtual representations. With the third research probe, we explore how to enhance virtuality to enable a user to input text on a physical keyboard while being immersed in the virtual world. Our prototype tracked the user’s hands and keyboard to enable generic text input. Our analysis of text entry performance showed the importance and effect of different hand representations. We then investigate how to touch virtuality by simulating generic haptic feedback for virtual reality and show how tactile feedback through quadcopters can significantly increase the sense of presence. Our final research probe investigates the usability and input space of smartphones within mixed reality environments, pairing the user’s smartphone as an input device with a secondary physical screen. Based on our learnings from these individual research probes, we developed a novel taxonomy for categorizing mixed reality experiences and guidelines for designing mixed reality experiences. The taxonomy is based on the human sensory system and human capabilities of articulation. We showcased its versatility and set our research probes into perspective by organizing them inside the taxonomic space. The design guidelines are divided into user-centered and technology-centered. It is our hope that these will contribute to the bright future of mixed reality systems while emphasizing the new underlining interaction paradigm.Mixed Reality (vermischte Realitäten) gehen aufgrund kontinuierlicher technologischer Innovationen langsam von der reinen Forschung in den Massenmarkt über. Mit der Einführung von leistungsfähigen Mixed-Reality-Geräten verändert sich die Art und Weise, wie wir Unterhaltungsmedien und Informationen konsumieren und wie wir mit Computersystemen interagieren. Verschiedene existierende Geräte sind in der Lage, jeden der menschlichen Sinne mit synthetischen Reizen zu stimulieren. Hierdurch verschwimmt zunehmend die Grenze zwischen der realen und der virtuellen Welt. Um eindrucksstarke und praktische Mixed-Reality-Erfahrungen zu kreieren, müssen Designer und Entwicklerinnen die künftigen Herausforderungen und neuen Möglichkeiten verstehen. In dieser Dissertation präsentieren wir eine neue Taxonomie zur Kategorisierung von Mixed-Reality-Erfahrungen sowie Richtlinien für die Gestaltung von solchen. Wir stellen die Ergebnisse von sieben Studien vor, in denen die Herausforderungen und Chancen von Mixed-Reality-Erfahrungen, die Auswirkungen von Modalitäten und Interaktionstechniken auf die Benutzererfahrung und die Möglichkeiten zur Verbesserung dieser Erfahrungen untersucht werden. Wir beginnen mit einer Studie, in der die Haltung der nutzenden Person gegenüber Mixed Reality in häuslichen und Bildungsumgebungen analysiert wird. In sechs weiteren Fallstudien wird jeweils ein Aspekt der Realität oder Virtualität untersucht. In der ersten Fallstudie wird mithilfe eines schwebenden und steuerbaren Projektors untersucht, wie die Wahrnehmung der realen Welt erweitert werden kann, ohne dabei die Person mit Technologie auszustatten. Wir zeigen, dass die Darstellung von in-situ-Anweisungen für Navigationsaufgaben zu einer deutlich höheren Fähigkeit führt, Sehenswürdigkeiten der realen Welt zu beobachten und wiederzufinden. In der zweiten Fallstudie erweitern wir die Wahrnehmung der Realität durch Überlagerung von Echtzeitinformationen, die für das menschliche Auge normalerweise unsichtbar sind. Durch die Erweiterung des menschlichen Sehvermögens ermöglichen wir den Anwender:innen, Wärmestrahlung visuell wahrzunehmen. Darüber hinaus untersuchen wir, wie sich das Ersetzen von physischen Komponenten durch nicht funktionale, aber greifbare Replikate oder durch die vollständig virtuelle Darstellung auswirkt. In der dritten Fallstudie untersuchen wir, wie virtuelle Realitäten verbessert werden können, damit eine Person, die in der virtuellen Welt verweilt, Text auf einer physischen Tastatur eingeben kann. Unser Versuchsdemonstrator detektiert die Hände und die Tastatur, zeigt diese in der vermischen Realität an und ermöglicht somit die verbesserte Texteingaben. Unsere Analyse der Texteingabequalität zeigte die Wichtigkeit und Wirkung verschiedener Handdarstellungen. Anschließend untersuchen wir, wie man Virtualität berühren kann, indem wir generisches haptisches Feedback für virtuelle Realitäten simulieren. Wir zeigen, wie Quadrokopter taktiles Feedback ermöglichen und dadurch das Präsenzgefühl deutlich steigern können. Unsere letzte Fallstudie untersucht die Benutzerfreundlichkeit und den Eingaberaum von Smartphones in Mixed-Reality-Umgebungen. Hierbei wird das Smartphone der Person als Eingabegerät mit einem sekundären physischen Bildschirm verbunden, um die Ein- und Ausgabemodalitäten zu erweitern. Basierend auf unseren Erkenntnissen aus den einzelnen Fallstudien haben wir eine neuartige Taxonomie zur Kategorisierung von Mixed-Reality-Erfahrungen sowie Richtlinien für die Gestaltung von solchen entwickelt. Die Taxonomie basiert auf dem menschlichen Sinnessystem und den Artikulationsfähigkeiten. Wir stellen die vielseitige Verwendbarkeit vor und setzen unsere Fallstudien in Kontext, indem wir sie innerhalb des taxonomischen Raums einordnen. Die Gestaltungsrichtlinien sind in nutzerzentrierte und technologiezentrierte Richtlinien unterteilt. Es ist unsere Anliegen, dass diese Gestaltungsrichtlinien zu einer erfolgreichen Zukunft von Mixed-Reality-Systemen beitragen und gleichzeitig die neuen Interaktionsparadigmen hervorheben

The Racial Rhetoric of Cuteness as Decorative Decorum

This work looks at the trope of cuteness as a means of investigating the topological phenomena of race and public space, particularly in regards to African American rhetorical modes of visual and spatial practice. By introducing a sociological coinage known as the \u27teddy-bear effect,\u27 this work explores how racialized expressions of cuteness give off the impression of a demurring civility surrounding the social expectations associated with the cultural norms of gender and class. As a preferred characteristic of information design and strategically deployed for the tactic of racialized passings in the face of increasingly regulated forms of \u27post-racial\u27 gate-keeping and contemporary color politics, this research interrogates how racial cutification animates certain generational differences within African American communities while simultaneously shaping mainstream conceptualizations of what constitutes appropriate public decorum. Of specific concern is the cultural logic of \u27minoritization\u27 on people of color as far as the techno-spatial processes of race and racism for how it serves as a means by which global citizenship continues to be fashioned, especially in civic politics, black women\u27s hair care and identity, social networking, and multimodal writing and pedagogy. Finally, this work asserts the ascendance of cuteness as a paradoxical sign of excess and miniaturization related to notions of multicultural authority and power and tracks the influence of this popularly imagined iconography of African Americanicity across the public sphere

Health State Estimation

Life's most valuable asset is health. Continuously understanding the state of our health and modeling how it evolves is essential if we wish to improve it. Given the opportunity that people live with more data about their life today than any other time in history, the challenge rests in interweaving this data with the growing body of knowledge to compute and model the health state of an individual continually. This dissertation presents an approach to build a personal model and dynamically estimate the health state of an individual by fusing multi-modal data and domain knowledge. The system is stitched together from four essential abstraction elements: 1. the events in our life, 2. the layers of our biological systems (from molecular to an organism), 3. the functional utilities that arise from biological underpinnings, and 4. how we interact with these utilities in the reality of daily life. Connecting these four elements via graph network blocks forms the backbone by which we instantiate a digital twin of an individual. Edges and nodes in this graph structure are then regularly updated with learning techniques as data is continuously digested. Experiments demonstrate the use of dense and heterogeneous real-world data from a variety of personal and environmental sensors to monitor individual cardiovascular health state. State estimation and individual modeling is the fundamental basis to depart from disease-oriented approaches to a total health continuum paradigm. Precision in predicting health requires understanding state trajectory. By encasing this estimation within a navigational approach, a systematic guidance framework can plan actions to transition a current state towards a desired one. This work concludes by presenting this framework of combining the health state and personal graph model to perpetually plan and assist us in living life towards our goals.Comment: Ph.D. Dissertation @ University of California, Irvin

The Multimodal Tutor: Adaptive Feedback from Multimodal Experiences

This doctoral thesis describes the journey of ideation, prototyping and empirical testing of the Multimodal Tutor, a system designed for providing digital feedback that supports psychomotor skills acquisition using learning and multimodal data capturing. The feedback is given in real-time with machine-driven assessment of the learner's task execution. The predictions are tailored by supervised machine learning models trained with human annotated samples. The main contributions of this thesis are: a literature survey on multimodal data for learning, a conceptual model (the Multimodal Learning Analytics Model), a technological framework (the Multimodal Pipeline), a data annotation tool (the Visual Inspection Tool) and a case study in Cardiopulmonary Resuscitation training (CPR Tutor). The CPR Tutor generates real-time, adaptive feedback using kinematic and myographic data and neural networks