Quantifying diversity in user experience

Evaluation should be integral to any design activity. Evaluation in innovative product development practices however is highly complicated. It often needs to be applied to immature prototypes, while at the same time users’ responses may greatly vary across different individuals and situations. This thesis has focused on methods and tools for inquiring into users’ experiences with interactive products. More specifically, it had three objectives: a) to conceptualize the notion of diversity in subjective judgments of users’ experiences with interactive products, b) to establish empirical evidence for the prevalence of diversity, and c) to provide a number of methodological tools for the study of diversity in the context of product development. Two critical sources of diversity in the context of users’ experiences with interactive products were identified and respective methodological solutions were proposed: a) understanding interpersonal diversity through personal attribute judgments, and b) understanding the dynamics of experience through experience narratives. Personal Attribute Judgments, and in particular, the Repertory Grid Technique, is proposed as an alternative to standardized psychometric scales, in measuring users’ responses to artifacts in the context of parallel design. It is argued that traditional approaches that rely on the a-priori definition of the measures by the researchers have at least two limitations. First, such approaches are inherently limited as researchers might fail to consider a given dimension as relevant for the given product and context, or they might simply lack validated measurement scales for a relevant dimension. Secondly, such approaches assume that participants are able to interpret and position a given statement that is defined by the researcher to their own context. Recent literature has challenged this assumption, suggesting that in certain cases participants are unable to interpret the personal relevance of the statement in their own context, and might instead employ shallow processing, that is respond to surface features of the language rather than attaching personal relevance to the question. In contrast, personal attributes are elicited from each individual respondent, instead of being a-priori imposed by the experimenter, and thus are supposed to be highly relevant to the individual. However, personal attributes require substantially more complex quantitative analysis procedures. It is illustrated that traditional analysis procedures fail to bring out the richness of the personal attribute judgments and two new Multi-Dimensional Scaling procedures that extract multiple complementary views from such datasets are proposed. An alternative approach for the measurement of the dynamics of experience over time is proposed that relies on a) the retrospective elicitation of idiosyncratic selfreports of one’s experiences with a product, the so-called experience narratives, and b) the extraction of generalized knowledge from these narratives through computational content analysis techniques. iScale, a tool that aims at increasing users’ accuracy and effectiveness in recalling their experiences with a product is proposed. iScale uses sketching in imposing a structured process in the reconstruction of one’s experiences from memory. Two different versions of iScale, each grounded in a distinct theory of how people reconstruct emotional experiences from memory, were developed and empirically tested. A computational approach for the extraction of generalized knowledge from experience narratives, that combines traditional coding procedures with computational approaches for assessing the semantic similarity between documents, is proposed and compared with traditional content analysis. Through these two methodological contributions, this thesis argues against averaging in the subjective evaluation of interactive products. It proposes the development of interactive tools that can assist designers in moving across multiple levels of abstractions of empirical data, as design-relevant knowledge might be found on all these levels

Towards an understanding of human behaviour for design action

It can be shown that exceeding both utilitarian and hedonic needs of consumers leads towards greater satisfaction, delight and enduring consumer loyalty. If designers are to meet the progressively diverse needs of consumers, then access to consumer values, aspirations and the underlying logic of their social practice become increasingly important. If we accept that what people say, do and think are often different things, gaining access to these requirements is clearly a challenge. The challenge is not only concerned with how these requirements are accessed at source, through widely adopted ethnographically inspired techniques, but more towards how these requirements are communicated to the designer. There is a clear disconnect between the collection of consumer requirements and how these requirements are arranged and communicated as implications for design. This thesis details a governance framework for the output of ethnographically inspired research methods to provide an understanding of the arrangement and attributes a communication tool for ethnographic work should possess, particularly towards the more technical area of new product development. The framework bridges a gap between consumer research and design action, which may be used as an approach to facilitate innovation, targeted problem solving and offer creative direction for new product development. Following an exploratory review of the literature and a series of way-finding interviews with domestic appliance and consumer goods manufacturers, a pilot study was conducted to identify the philosophical and practical barriers faced by designers, when designing for consumer requirements beyond the functional. A detailed second level literature review explored the emergent themes and led towards a desktop review of over 30 different creative thinking design tools from the design & emotion movement, 24 different communication approaches for ethnographic work in design and a two year case study on communication within the design process

Promoting Andean children's learning of science through cultural and digital tools

Conference Theme: To see the world and a grain of sand: Learning across levels of space, time, and scaleIn Peru, there is a large achievement gap in rural schools. In order to overcome this problem, the study aims to design environments that enhance science learning through the integration of ICT with cultural artifacts, respecting the Andean culture and empower rural children to pursue lifelong learning. This investigation employs the Cultural-Historical Activity Theory (CHAT) framework, and the Design-Based Research (DBR) methodology using an iterative process of design, implementation and evaluation of the innovative practice.published_or_final_versio

Creating the eDesign Assessment Tool (eDAT) to Represent and Evaluate Online Distance Learning Designs: A Mixed Methods Study

There are concerns that retention is lower in ODL (online distance learning) than in face-to-face courses and that this has a negative impact on both student and institution. Learning design has been shown to have an impact on retention, but a tool for a variety of developers, designers and tutors to consistently describe and compare learning designs of ODL courses remains elusive. This study argues that the lack of a common terminology and different tutor perspectives about learning hamper both development and representation of ODL designs. Existing research suggests that a complex mixture of student demographics, characteristics and skills, as well as course and institution features including level, subject and amount and type of tutor support may impact on ODL student retention. In particular, research suggests that activities that include interaction and feedback in ODL have a positive impact on retention. However, much of this research has been conducted using a range of student surveys that do not allow for comparison to retention data, or across courses or institutions due to their subjective nature. There has been little research on the impact of tutor perspectives on developing, representing and sharing learning designs. This study examines the creation of the e-Design Assessment Tool (eDAT) that represents and quantifies interaction and feedback activities so they can be compared to retention and other learning data for a course. A mixed methods approach was used to: a) test the effectiveness of existing terminology for categorising learning activities using a content analysis methodology by trialling sets of terms with a sample set of ODL course activities. b) identify tutor perspectives about learning and teaching using repertory grids based on personal construct psychology, and exploring the impact of these perspectives on the different meanings and uses of learning activity terminology. The content analysis testing of learning activity terminology was challenging. The pilot studies had low inter-rater reliability, suggesting difficulties in independent rating of existing learning design terminology. However, the final eDAT tool created through data collected for this study used terms that did lead to a greater level of inter-rater reliability. A significant contribution of this study is the use of repertory grids to gain insight into the issues relating to the development of a quantitative tool. The repertory grid interviews indicated that there were significant differences in the ways tutors and raters understood and used key educational concepts including interaction and feedback, and that there was a variety of vocabulary used when describing learning activities. This study argues that tutor perspectives impact on designing, representing and evaluating ODL courses in a way that has implications for learning design and for professional development of tutors. The final eDAT builds on and develops existing learning design representation and evaluation tools, but utilises more consistent terminology. Thus it offers a simplified approach to pedagogic guidance in the form of quantification of interaction and feedback activities in a course, and embeds reflection on tutor perspectives underpinning the design to support sharing and reuse. This combination will lead to better ways to represent learning designs, as well as providing a method for gathering learning analytics data useful for the comparison of learning designs to student retention data and thereby improve practice in ODL

ARTiVIS Arts, real-time video and interactivity for sustainability

Dissertação para obtenção do Grau de Doutor em Media DigitaisPortuguese Foundation for Science and Technology (SFRH/BD/42555/2007

ICT and its assessment at 16: an enquiry into the perceptions of year 11 students

This study, conducted between 2006 and 2011, enquired into student perceptions of Information and Communications Technology (ICT) and its assessment at aged 16. The prevailing orthodoxies amongst writers, commentators and educationalists are that the subject does not reflect the learning and use made by young people of technology. The voice of the learner, so often lauded in aspects of school democracy and in formative assessment, has not been heard in respect of the high-stakes assessment at the end of Key Stage (KS) 4 in schools in England. This research was a step in filling that void. Taking an interpretive phenomenological approach three phases of empirical data collection were used each building on the previous ones. To bring the student perception and voice to the fore a repertory grid analysis was initially used to elicit constructs of learning and assessment directly from the students. This was followed by a questionnaire and semi-structured interviews across a sample of state-funded schools in England. The use of a multiple-phase data collection allowed phenomena to be distilled with successively more depth at each phase

Learning mobility in professional practice: transforming workplace learning in higher education

This study investigates how educators as adult learners learn within the higher education sector and how to design for effective professional learning, from the perspective of the educator. The researcher had observed, and the educational literature suggests, that professional development activities are often perceived by educators as frustrating, irrelevant and time consuming, resulting in resistance to taking part in such events and perpetuating the status quo of professional learning practice in higher education. To add new thinking and new evidence to create a shift in the perception of professional learning, the researcher targeted higher education teachers who may have found professional learning frustrating but had navigated a pathway through the complexities to grow and develop their professional practice in ways that are personally meaningful to them. Such educators demonstrated a natural motivation to engage in professional learning. The central argument of this thesis is that designing for effective professional learning needs to take a bottom-up, inside-out approach. This approach recognises that personally meaningful professional learning that challenges and changes how educators learn needs to start from the inside by exploring the educator’s inner belief system, ever-changing identity and developing sense of self. However, when investigating how educators learn, attention is also given to the complex, strongly connected relationship between the individual and the institution. Within this study, a way of making sense of the relational nature of the educator and the institution is by using the metaphor of the higher education ecosystem to represent the inextricably linked system of humans and their environment. Four key concepts are introduced and developed as the thesis progresses and matures to the point that the concepts themselves evolve in an inter-connected, inter-related manner. First, the researcher introduces, builds and applies the concept of learning mobility to challenge the status quo of professional learning in higher education. The researcher’s concept of learning mobility is the educator’s choice to learn, work, communicate, collaborate and connect in any configuration, across learning contexts and boundaries for continuous professional learning and personal growth. Building on the idea of the educator’s learning mobility is the second concept of the wholeness of professional learning, which is concerned with how educators come to the learning, how educators learn, and what educators do with the learning, to bring about personally meaningful change in professional practice. The concepts of learning mobility and the wholeness of professional learning led to theorising the third concept of professional learning mobility. This concept provides an alternative approach to the design of effective professional learning as it shifts the focus towards understanding how individuals experience learning continuously across and within their inner (internal, personal) and outer (external, professional) worlds. Exploring and maturing a deeper understanding of these three concepts throughout the thesis contributed to theory building about how educators learn. Finally, this evidence-based theory building introduces the abstract concept of the third space, which was identified after considering the rationalities of the head space and the irrationalities of the heart space, and their powerful influences on the learning process. The third space of professional learning mobility represents the educator’s own growth and development that transcends the complexities of institutional structures, conditions and policies that are outside the educator’s control. The third space represents the educator’s emotional and mental resilience to respond to the disruptive nature of being human as we become conscious of who we are on the inside. This space is conceived as a transformative space that offers a sense of wholeness, giving individuals the inner motivation and courage to connect to themselves and others. A united revelation emerges across the four concepts to discern that it is the mobility of the learner and the learning which becomes significant to address the educator as adult learner’s natural human desire for growth, development and freedom.From the study, the “7Cs” design principles were derived in order to foster the educator’s professional learning mobility: context, control, connection, complexity, courage, continuity, and creativity. The '7Cs of professional learning mobility' are used to design dynamic learning environments that take into account the educator’s inner and outer worlds and their need for choice, autonomy and freedom to authentically engage in their learning. The 7Cs, framed within a conceptual model that encompasses the head space, heart space, and third space provide an opportunity to theorise the educator’s learning mobility in professional practice that could be used to transform professional learning in the higher education workplace. Overall, this study represents an evidence-based approach to contribute to theory in adult learning to support a shift in the practice status quo of professional learning in higher education

Reconstructing the Present Through Kinesthetic History: An Investigation into Modes of Preserving, Transmitting, and Restaging Contemporary Dance

Methods of dance preservation have evolved alongside conceptual themes that have framed dance’s historical narrative. The tradition of written dance notation developed in accordance with notions that prioritized logocentricity, and placed historical legitimacy on tangible artifacts and irrefutable archives; whereas the technical revolution of the late twentieth century saw dance preservation practices shift to embrace film and video documentation because they provided more accessible, and more convenient records. Since the 1970s video recordings have generally been considered to provide authentic visual representations of dance works, and the tradition of score writing has begun to wane. However, scholarly criticism has unveiled both philosophical and practical challenges posed by these two modes of documentation, thus illuminating a gap between theories of embodiment and the practice of dance preservation. In alignment with contemporary discourse, which legitimizes the body as a site of generating and storing knowledge, this dissertation suggests ‘kinesthetic history’ as a valid mode of dance preservation. Operating as a counterpart to oral history, and borrowing theoretical concepts from contemporary historiography, existential phenomenology and ethnography, the term ‘kinesthetic history’ suggests a mode of corporeal inscription and transmission that relies on the reciprocal interaction of bodies in space. The use of ‘kinesthetic history’ as a methodological approach to the preservation, translation, and reconstruction of movement material reflects the elements of fluidity, plurality and subjectivity that are often characteristic of contemporary choreographic practices. This theory is interrogated through a case study, which explores the ways in which both a written and digitized score, video recordings, and the ‘kinesthetic history’ of an original cast member operated as modes of transmission in a 2013 restaging of William Forsythe’s One Flat Thing, reproduced (2000) at The Juilliard School. Conclusions drawn from the case study challenge the traditional notions of reconstruction and restaging and suggest ‘regeneration’ as an alternative term to describe the process of preserving and transmitting contemporary dance works

Emancipating and developing learning activity: Systemic intervention and re-instrumentation in higher education.

The central theme of this thesis is the emancipation and further development of learning activity in higher education in the context of the ongoing digital transformation of our societies. It was developed in response to the highly problematic mainstream approach to digital re-instrumentation of teaching and studying practises in contemporary higher education. The mainstream approach is largely based on centralisation, standardisation, commoditisation, and commercialisation, while re-producing the general patterns of control, responsibility, and dependence that are characteristic for activity systems of schooling. Whereas much of educational research and development focuses on the optimisation and fine-tuning of schooling, the overall inquiry that is underlying this thesis has been carried out from an explicitly critical position and within a framework of action science. It thus conceptualises learning activity in higher education not only as an object of inquiry but also as an object to engage with and to intervene into from a perspective of intentional change. The knowledge-constituting interest of this type of inquiry can be tentatively described as a combination of heuristic-instrumental (guidelines for contextualised action and intervention), practical-phronetic (deliberation of value-rational aspects of means and ends), and developmental-emancipatory (deliberation of issues of power, self-determination, and growth) aspects. Its goal is the production of orientation knowledge for educational practise. The thesis provides an analysis, argumentation, and normative claim on why the development of learning activity should be turned into an object of individual|collective inquiry and intentional change in higher education, and why the current state of affairs in higher education actually impedes such a development. It argues for a decisive shift of attention to the intentional emancipation and further development of learning activity as an important cultural instrument for human (self-)production within the digital transformation. The thesis also attempts an in-depth exploration of what type of methodological rationale can actually be applied to an object of inquiry (developing learning activity) that is at the same time conceptualised as an object of intentional change within the ongoing digital transformation. The result of this retrospective reflection is the formulation of “optimally incomplete” guidelines for educational R&D practise that shares the practicalphronetic (value related) and developmental-emancipatory (power related) orientations that had been driving the overall inquiry. In addition, the thesis formulates the instrumental-heuristic knowledge claim that the conceptual instruments that were adapted and validated in the context of a series of intervention studies provide means to effectively intervene into existing practise in higher education to support the necessary development of (increasingly emancipated) networked learning activity. It suggests that digital networked instruments (tools and services) generally should be considered and treated as transient elements within critical systemic intervention research in higher education. It further argues for the predominant use of loosely-coupled, digital networked instruments that allow for individual|collective ownership, control, (co-)production, and re-use in other contexts and for other purposes. Since the range of digital instrumentation options is continuously expanding and currently shows no signs of an imminent slow-down or consolidation, individual and collective exploration and experimentation of this realm needs to be systematically incorporated into higher education practise.Siirretty Doriast