Sensemaking with learning analytics visualizations: Investigating dashboard comprehension and effects on learning strategy

In the provision of just-in-time feedback, student-facing learning analytics dashboards (LADs) are meant to aid decision-making during the process of learning. Unlike summative feedback received at its conclusion, this formative feedback may help learners pivot their learning strategies while still engaged in the learning activity. To turn this feedback into actionable insights however, learners must understand LADs well enough to make accurate judgements of learning with them. For these learners, LADs could become an integral part of their self-regulatory learning strategy. This dissertation presents a multifaceted examination of learners’ sensemaking processes with LADs designed to support self-regulatory learning. The in-situ studies detailed therein examine learners’ understanding of the data visualized in LADs and the effects of this understanding on their performance-related mental models. Trace data, surveys, semi-structured in-depth qualitative interviews, and retrospective cued recall methods were used to identify why, when, and how learners used LADs to guide their learning. Learners’ qualitative accounts of their experience explained and contextualized the quantitative data collected from the observed activities. Learners preferred less complex LADs, finding them more useful and aesthetically appealing, despite lower gist recall with simpler visualizations. During an early investigation of how LADs were used to make learning judgments in situ, we observed learners’ tendency to act upon brief LAD interactions. This inspired us to operationalize gist as a form of measurement, describing learners’ ability to make sense of a LAD after a brief visual interrogation. Subsequent comparisons of the accuracy and descriptiveness of learners’ gist estimates to those of laypeople repeatedly showed that laypeople were more apt than learners to produce accurate and complete gist descriptions. This dissertation culminates in a final study examining the evolution of learners’ mental models of their performance due to repeated LAD interaction, followed by a discussion of the contextual factors that contributed to what was observed. Trends observed across this work suggest that learners were more apt to “get the gist” with LAD after repeated interaction. This dissertation contributes a novel method for evaluating learners’ interpretation of LADs, while our findings offer insight into how LADs shape learners’ sensemaking processes

Using the BioDatamation strategy to learn introductory college biology: value-added effects on selected students\u27 conceptual understanding and conceptual integration of the processes of photosynthesis and celluar respiration

The purpose of this exploratory research was to study how students learn photosynthesis and cellular respiration and to determine the value added to the student\u27s learning by each of the three technology-scaffolded learning strategy components (animated concept presentations and WebQuest-style activities, data collection, and student-constructed animations) of the BioDatamation™ (BDM) Program. BDM learning strategies utilized the Theory of Interacting Visual Fields™ (TIVF) (Reuter & Wandersee, 2002a, 2002b; 2003a, 2003b) which holds that meaningful knowledge is hierarchically constructed using the past, present, and future visual fields, with visual metacognitive components that are derived from the principles of Visual Behavior (Jones,1995), Human Constructivist Theory (Mintzes & Wandersee, 1998a), and Visual Information Design Theory (Tufte, 1990, 1997, 2001). Student alternative conceptions of photosynthesis and cellular respiration were determined by the item analysis of 263,267 Biology Advanced Placement Examinations and were used to develop the BDM instructional strategy and interview questions. The subjects were 24 undergraduate students of high and low biology prior knowledge enrolled in an introductory-level General Biology course at a major research university in the Deep South. Fifteen participants received BDM instruction which included original and innovative learning materials and laboratories in 6 phases; 8 of the 15 participants were the subject of in depth, extended individual analysis. The other 9 participants received traditional, non-BDM instruction. Interviews which included participants’ creation of concept maps and visual field diagrams were conducted after each phase. Various content analyses, including Chi\u27s Verbal Analysis and quantitizing/qualitizing were used for data analysis. The total value added to integrative knowledge during BDM instruction with the three visual fields was an average increase of 56% for cellular respiration and 62% increase for photosynthesis knowledge, improved long-term memory of concepts, and enhanced biological literacy to the multidimensional level, as determined by the BSCS literacy model. WebQuest-style activities and data collection provided for animated prior knowledge in the past visual field, and detailed content knowledge construction in the present visual field. During student construction of animated presentations, layering required participants to think by rearranging words and images for improved hierarchical organization of knowledge with real-life applications

Experimental approaches for understanding conceptual design activities

Designers play an important and critical role in developing innovative product design, which is the key for a company to survive in the highly competitive market with ever-increasing demands from customers. However, how designers think, reason, judge, and make decisions has not yet been studied well. This thesis aims to develop experimental approaches to qualitatively and quantitatively understand designers' cognitive activities in order to explore product innovation in the early design stages. To study designers' cognitive activities, a new protocol analysis method is developed. In the protocol analysis experiment, designers' activities were recorded when solving a design problem. Designers were interviewed to recall their design process immediately after finishing the design. Then, the recorded verbal data were analyzed by transcribing, segmentation and encoding for further analysis. In general, protocol analysis is used to transform the unstructured data collected from designers into structured data. Existing protocol analysis methods used to study designers' cognitive activities heavily depend on specific design problems, specific domains and the persons who analyze the protocol data. The new protocol analysis method presented in this thesis is based on the concept of the state of design and recursive object model. It can be easily applied to any design problem and any domain. Using this method, the changes of designers' cognitive activities during the design process can be quantified. Some guidelines and recommendations for assisting designers to deliver an innovative design are summarized at the end of the present thesis based on the experimental results. The protocol analysis results have also been used to evaluate Environment-Based Design (EBD) as a descriptive design model; hence, EBD can be used as the theoretical foundation to illustrate, describe, and explain how designers conduct a design task. Another part of my thesis is to simulate the design process by developing a virtual experiment. In the virtual experiment, the selected design problem: finite element mesh design, can be automatically solved using a program developed in the computer environment of Visual C++. Net 2003. Then, different settings of the parameters, different strategies, different formulations of design requirements, and different sequences for solving the design problem are simulated to compare the differences of meshes. Three routes leading designers to innovative designs are proposed and examined through this virtual experiment. Based on the three routes, an ANN-based element extraction method for finite element mesh design is developed to illustrate the feasibility of the three routes for changing design solutions. Therefore, the two experimental approaches proposed in this thesis can be used for understanding design activities in a more systematic manne

Examining the Design and Usability of Telemedicine Communications: A Mixed-methods Study

This dissertation describes a mixed-methods study that examines the usability of telemedicine provider interfaces. This study consisted of content analysis, survey, and think aloud methodologies, which afford a multifaceted corpus of data for which to draw inferences and identify design features and functions that negatively impact usability. Usability is a critical component of the user experience with a telemedicine provider interface and can suede or impede the acceptance and adoption of telemedicine. Telemedicine has the potential to increase quality healthcare access and positive health outcomes for individuals who use it, and usability is a key component of technology acceptance and effective use. Empirical testing of health information technology (HIT) and telemedicine is advocated for as it is the most valuable method of research to understand humans\u27 cognitive processing of information as they interact with technology. In addition, using activity theory and mobile interface theory as a lens in which to understand human activities and interaction with telemedicine provider interfaces, including the telemedicine provider websites and their mobile-responsive websites in this study, is an effective tool for drawing reasonable inferences regarding the usability of telemedicine communications. Considering the rate at which an unprecedented amount of health information becomes available online and HIT facilitates the delivery of healthcare, usability testing and user-centered, iterative design practices become increasingly essential in order to design effective—and safe—health information and technology that enhance the patient-experience, the affordability and accessibility of healthcare, health literacy and patient empowerment, and positive health outcomes. Usability testing plays an increasingly important role in characterizing obstacles to achieving these initiatives of the modern patient-centered health paradigm and telemedicine. The mixed-methods usability testing performed in this study offers a principled approach to usability testing and is ecologically valid because it involves real human subjects. This study fulfills a void in research on the usability of telemedicine communications and reveals usability problems that may not be anticipated by designers of HIT and health information providers. Drawing from the insight gained from this mixed-method study, design features and functions that enhance the usability of health communications are offered. This study draws insight from the human factors, technical communication, and health and medical fields to develop systematic, practical usability testing methods that can be replicated and applied in many fields. The design recommendations resulting from this study will be valuable to programmers; systems analysts; clinicians and nurses; technical communicators; information architects; visual designers; and others in similar roles

The few touch digital diabetes diary : user-involved design of mobile self-help tools for peoplewith diabetes

Paper number 2, 4, 5 and 7 are not available in Munin, due to publishers' restrictions: 2. Årsand E, and Demiris G.: "User-Centered Methods for Designing Patient-Centric Self-Help Tools", Informatics for Health and Social Care, 2008 Vol. 33, No. 3, Pages 158-169 (Informa Healthcare). Available at http://dx.doi.org/10.1080/17538150802457562 4. Årsand E, Olsen OA, Varmedal R, Mortensen W, and Hartvigsen G.: "A System for Monitoring Physical Activity Data Among People with Type 2 Diabetes", pages 173-178 in S.K. Andersen, et.al. (eds.) "eHealth Beyond the Horizon - Get IT There", Proceedings of MIE2008, Studies in Health Technology and Informatics, Volume 136, May 2008, ISBN: 978-1-58603-864-9 5. Årsand E, Tufano JT, Ralston J, and Hjortdahl P.: "Designing Mobile Dietary Management Support Technologies for People with Diabetes", Journal of Telemedicine and Telecare, 2008 Volume 14, Number 7, Pp. 329-332 (Royal Society of Medicine Press). Available at http://dx.doi.org/10.1258/jtt.2008.007001 7. Årsand E, Walseth OA, Andersson N, Fernando R, Granberg O, Bellika JG, and Hartvigsen G.: "Using Blood Glucose Data as an Indicator for Epidemic Disease Outbreaks", pages 199-204 in R. Engelbrecht et.al. (eds.): "Connecting Medical Informatics and Bio-Informatics", Proceedings of MIE2005, Studies in Health Technology and Informatics, Volume 116, August 2005, ISBN: 978-1-58603-549-5. Check availabilityParadoxically, the technological revolution that has created a vast health problem due to a drastic change in lifestyle also holds great potential for individuals to take better care of their own health. The first consequence is not addressed in this dissertation, but the second represents the focus of the work presented, namely utilizing ICT to support self-management of individual health challenges. As long as only 35% of the patients in Norway achieve the International Diabetes Federation‟s goal for blood glucose (HbA1c), actions and activities to improve blood glucose control and related factors are needed. The presented work focuses on the development and integration of alternative sensor systems for blood glucose and physical activity, and a fast and effortless method for recording food habits. Various user-interface concepts running on a mobile terminal constitute a digital diabetes diary, and the total concept is referred to as the “Few Touch application”. The overall aim of this PhD project is to generate knowledge about how a mobile tool can be designed for supporting lifestyle changes among people with diabetes. Applying technologies and methods from the informatics field has contributed to improved insight into this issue. Conversely, addressing the concrete use cases for people with diabetes has resulted in the achievement of ICT designs that have been appreciated by the cohorts involved. Cooperation with three different groups of patients with diabetes over several years and various methods and theories founded in computer science, medical informatics, and telemedicine have been combined in design and research on patient-oriented aids. The blood glucose Bluetooth adapter, the step counter, and the nutrition habit registration system that have been developed were all novel and to my knowledge unique designs at the time they were first tested, and this still applies to the latter two. Whether it can be claimed that the total concept presented, the Few Touch application, will increase quality of life, is up to future research and large-scale tests of the system to answer. However, results from the Type 2 diabetes half-year study showed that several of the participants did adjust their medication, food habits and/or physical activity due to use of the application

Facilitating immersion, engagement and flow in multi-user virtual environments

Virtual worlds are providing myriad opportunities for the development of innovative curricula for tertiary educators. They provide a virtual meeting space for those students and lecturers who are geographically remote from one another, rendering distance irrelevant and facilitating the formation of community. This paper will look at those factors - physical, social, virtual and those related to pedagogy - which facilitate immersion in virtual worlds; that suspension of disbelief which generates the feeling of presence or 'being there', crucial to promoting student engagement and ultimately, flow

Taking an Effective Authorial Stance in Academic Writing: Inductive Learning for Second Language Writers using a Stance Corpus.

The study focuses on a pedagogical proposal using a specialized corpus to assist advanced second language (L2) writers to tackle a critical aspect in academic writing, authorial stance-taking. Novice L2 writers are often found to project an unauthoritative persona and voice, which compromises their research potential. Premised on linguistic (Systemic Functional and Corpus Linguistics) and learning theories, the study tested the hypotheses that (1) rendering explicit the complex linguistic stance resources is beneficial to inform ways to argue with a persuasive authorial stance, and (2) corpora tools encourage inductive pattern-finding, which brings about deep learning and better performance. Seven Mandarin-speaking learners of English studying at post-graduate levels were recruited to engage in five-session training and interaction with the tool as they wrote introductions to their research. Multiple methods were employed to analyze (1) overall writing performance (2) stance understanding, (3) the development of stance understanding, (4) cognitive patterns in using the tool and in overall learning and (5) tool use patterns. Results show a positive relationship between stance learning and writing performance. The semantically-oriented corpus also engaged the learners’ to manage the discursive flow of their stance deployment, critical to greater effectiveness in their writing. Their cognitive learning patterns reflected frequent “sense-making”, “exploring” and “reasoning” about the complex new knowledge while inductive pattern-finding is infrequent. The findings suggest training and more support are need to optimize an inductive learning approach. In addition, individual learning styles and preconception are both critical in dictating learning outcome. Learners’ cognitive burden in learning complex linguistic concept using a second language also need to be considered. Finally, to encourage use of the corpus, more examples need to be included so that the users find convincing patterns of stance deployment which is relevant to their argumentative style and disciplinary background. To conclude, to better support academic writers, corpora tools show great potential in hosting examples at the discursive level to complement a lexico-grammatical approach in writing. Such tools, with annotation and textual enhancement, stimulate the writers’ attention to the structural and prosodic aspects of professional writings, an awareness needed for succeeding in a writing task.Ph.D.Education StudiesUniversity of Michigan, Horace H. Rackham School of Graduate Studieshttp://deepblue.lib.umich.edu/bitstream/2027.42/77860/1/peichin_1.pd

Supporting Exploratory Search Tasks Through Alternative Representations of Information

Information seeking is a fundamental component of many of the complex tasks presented to us, and is often conducted through interactions with automated search systems such as Web search engines. Indeed, the ubiquity of Web search engines makes information so readily available that people now often turn to the Web for all manners of information seeking needs. Furthermore, as the range of online information seeking tasks grows, more complex and open-ended search activities have been identified. One type of complex search activities that is of increasing interest to researchers is exploratory search, where the goal involves "learning" or "investigating", rather than simply "looking-up". Given the massive increase in information availability and the use of online search for tasks beyond simply looking-up, researchers have noted that it becomes increasingly challenging for users to effectively leverage the available online information for complex and open-ended search activities. One of the main limitations of the current document retrieval paradigm offered by modern search engines is that it provides a ranked list of documents as a response to the searcher’s query with no further support for locating and synthesizing relevant information. Therefore, the searcher is left to find and make sense of useful information in a massive information space that lacks any overview or conceptual organization. This thesis explores the impact of alternative representations of search results on user behaviors and outcomes during exploratory search tasks. Our inquiry is inspired by the premise that exploratory search tasks require sensemaking, and that sensemaking involves constructing and interacting with representations of knowledge. As such, in order to provide the searchers with more support in performing exploratory activities, there is a need to move beyond the current document retrieval paradigm by extending the support for locating and externalizing semantic information from textual documents and by providing richer representations of the extracted information coupled with mechanisms for accessing and interacting with the information in ways that support exploration and sensemaking. This dissertation presents a series of discrete research endeavour to explore different aspects of providing information and presenting this information in ways that both extraction and assimilation of relevant information is supported. We first address the problem of extracting information – that is more granular than documents – as a response to a user's query by developing a novel information extraction system to represent documents as a series of entity-relationship tuples. Next, through a series of designing and evaluating alternative representations of search results, we examine how this extracted information can be represented such that it extends the document-based search framework's support for exploratory search tasks. Finally, we assess the ecological validity of this research by exploring error-prone representations of search results and how they impact a searcher's ability to leverage our representations to perform exploratory search tasks. Overall, this research contributes towards designing future search systems by providing insights into the efficacy of alternative representations of search results for supporting exploratory search activities, culminating in a novel hybrid representation called Hierarchical Knowledge Graphs (HKG). To this end we propose and develop a framework that enables a reliable investigation of the impact of different representations and how they are perceived and utilized by information seekers

An Investigation of the Impact of Task-Types on the Reactivity of the Concurrent Think-Aloud in Usability Testing

The Concurrent think-aloud (CTA) is primarily used to understand users’ task based cognitive processes. However, is not without limitations. CTA procedures varies widely among practitioners. Also, it has been known to cause reactivity: an artificial change in task performance. This is problematic because it may alter the accuracy of task performance. Also, research on reactivity within usability testing have shown mixed findings. Thus, conclusions cannot be drawn to attest to whether reactivity occurs due to varying administration procedures and therefore we must now consider its relationship to other test-based factors. This research will be the first to systematically investigate the impact of task-type on reactivity of the CTA and the first to systematically investigates practitioners working habit in terms of their views on reactivity when using CTA in practice. Three studies were conducted, the first study investigates the Impact of task-types on the Reactivity of CTA and uses a mixed design. The results suggest that, thinking aloud during usability testing does not cause reactivity, and task type does not impact concurrent think-aloud. However, sensemaking tasks increase mental demand. The second study investigates the impact of task-type on two different think-aloud protocols and uses a mixed design. The result indicates that, the classic think-aloud method led to more successful task completion and no reactivity, while the explicit instruction produced fewer successful task completions and a higher mental workload. The explicit instruction produced less verbalisation, resulting in fewer relevant explanatory utterances, contradicting expectations. The third study uses an interview method to explores practitioners' experiences, views on reactivity and challenges when using the think-aloud method within usability testing. These studies demonstrates unequivocally that CTA should not be abandoned in usability studies as it provided valuable think-aloud data and helped identify usability issues. Additionally, practitioners should not replace the traditional think- aloud approach with explicit instruction, as explicit instruction had a greater influence on participants' behaviour. Ericsson and Simon's recommendations should be used for concurrent data collection, as it ensures data validity and generates the same type of data as explicit instruction while reducing reactivity