Un modèle pour la gestion et la capitalisation d'analyses de traces d'activités en interaction collaborative

We present our three main results in adressing the problem of assisting the socio-cognitive analysis of human interaction. First, we propose a description of the process of analysis of such data, as well as a generic artefact which covers a large number of the analytic artefacts we have observed and which we call a replayable. Second, we present a study and a modelling of replayables, and describe the four fundamental operations which can be applied to them: synchronisation, visualisation, transformation and enrichment. Finally, we describe the implementation of this model in an environment that assists analysis through the manipulation of replayables, which we evaluate in real-life research situations. Tatiana (http://code.google.com/p/tatiana), the resulting software environment, is based on these four operations and integrates numerous possibilities for extending these operations to adapt to new kinds of analysis while staying within the analytic framework afforded by replayables.Nous présentons nos trois résultats principaux face à la difficulté d'assister l'analyse socio-cognitive d'interactions humaines. D'une part, nous proposons une description du processus d'analyse de ce genre données ainsi qu'un artefact générique permettant de recouvrir un grand nombre d'artefacts analytiques que nous avons pu observer et que nous nommons rejouable. D'autre part, nous présentons une étude et modélisation informatique des rejouables, et décrivons quatre opérations fondamentales qui peuvent s'y appliquer : synchronisation, visualisation, transformation et enrichissement. Enfin, nous décrivons l'implémentation de cette modélisation dans un environnement d'aide à l'analyse par manipulation de rejouables que nous évaluons dans des situations de recherche réelles. Tatiana (http://code.google.com/p/tatiana), l'environnement logiciel résultant, est basé sur ces quatre opérations et permet l'extension de ces opérations pour s'adapter à de nouvelles formes d'analyse

Augmented analyses: supporting the study of ubiquitous computing systems

Ubiquitous computing is becoming an increasingly prevalent part of our everyday lives. The reliance of society upon such devices as mobile phones, coupled with the increasing complexity of those devices is an example of how our everyday human-human interaction is affected by this phenomenon. Social scientists studying human-human interaction must now take into account the effects of these technologies not just on the interaction itself, but also on the approach required to study it. User evaluation is a challenging topic in ubiquitous computing. It is generally considered to be difficult, certainly more so than in previous computational settings. Heterogeneity in design, distributed and mobile users, invisible sensing systems and so on, all add up to render traditional methods of observation and evaluation insufficient to construct a complete view of interactional activity. These challenges necessitate the development of new observational technologies. This thesis explores some of those challenges and demonstrates that system logs, with suitable methods of synchronising, filtering and visualising them for use in conjunction with more traditional observational approaches such as video, can be used to overcome many of these issues. Through a review of both the literature of the field, and the state of the art of computer aided qualitative data analysis software (CAQDAS), a series of guidelines are constructed showing what would be required of a software toolkit to meet the challenges of studying ubiquitous computing systems. It outlines the design and implementation of two such software packages, \textit{Replayer} and \textit{Digital Replay System}, which approach the problem from different angles, the former being focussed on visualising and exploring the data in system logs and the latter focussing on supporting the methods used by social scientists to perform qualitative analyses. The thesis shows through case studies how this technique can be applied to add significant value to the qualitative analysis of ubiquitous computing systems: how the coordination of system logs and other media can help us find information in the data that would otherwise be inaccessible; an ability to perform studies in locations/settings that would otherwise be impossible, or at least very difficult; and how creating accessible qualitative data analysis tools allows people to study particular settings or technologies who could not have studied them before. This software aims to demonstrate the direction in which other CAQDAS packages may have to move in order to support the study of the characteristics of human-computer and human-human interaction in a world increasingly reliant upon ubiquitous computing technology

How online small groups co-construct mathematical artifacts to do collaborative problem solving

Developing pedagogies and instructional tools to support learning math with understanding is a major goal in math education. A common theme among various characterizations of mathematical understanding involves constructing relations among mathematical facts, procedures, and ideas encapsulated in graphical and symbolic artifacts. Discourse is key for enabling students to realize such connections among seemingly unrelated mathematical artifacts. Analysis of mathematical discourse on a moment-to-moment basis is needed to understand the potential of small-group collaboration and online communication tools to support learning math with understanding.This dissertation investigates interactional practices enacted by virtual teams of secondary students as they co-construct mathematical artifacts in an online environment with multiple interaction spaces including text-chat, whiteboard, and wiki components. The findings of the dissertation arrived at through ethnomethodologically-informed case studies of online sessions are organized along three dimensions: (a) Mathematical Affordances: Whiteboard and chat spaces allow teams to co-construct multiple realizations of relevant mathematical artifacts. Contributions remain persistentlyavailable for subsequent manipulation and reference in the shared visual field. The persistence of contributions facilitates the management of multiple threads of activities across dual media. The sequence of actions that lead to the construction and modification of shared inscriptions makes the visual reasoning process visible.(b) Coordination Methods: Team members achieve a sense of sequential organization across dual media through temporal coordination of their chat postings and drawings. Groups enact referential uses of available features to allocate their attention to specific objects in the shared visual field and to associate them with locally defined terminology. Drawings and text-messages are used together as semiotic resources in mutually elaborating ways.(c) Group Understanding: Teams develop shared mathematical understanding through joint recognition of connections among narrative, graphical and symbolic realizations of the mathematical artifacts that they have co-constructed to address their shared task. The interactional organization of the co-construction work establishes an indexical ground as support for the creation and maintenance of a shared problem space for the group. Each new contribution is made sense of in relation to this persistently available and shared indexical ground, which evolves sequentially as new contributions modify the sense of previous contributions.Ph.D., Information Science and Technology -- Drexel University, 200

Automating Software Development for Mobile Computing Platforms

Mobile devices such as smartphones and tablets have become ubiquitous in today\u27s computing landscape. These devices have ushered in entirely new populations of users, and mobile operating systems are now outpacing more traditional desktop systems in terms of market share. The applications that run on these mobile devices (often referred to as apps ) have become a primary means of computing for millions of users and, as such, have garnered immense developer interest. These apps allow for unique, personal software experiences through touch-based UIs and a complex assortment of sensors. However, designing and implementing high quality mobile apps can be a difficult process. This is primarily due to challenges unique to mobile development including change-prone APIs and platform fragmentation, just to name a few. in this dissertation we develop techniques that aid developers in overcoming these challenges by automating and improving current software design and testing practices for mobile apps. More specifically, we first introduce a technique, called Gvt, that improves the quality of graphical user interfaces (GUIs) for mobile apps by automatically detecting instances where a GUI was not implemented to its intended specifications. Gvt does this by constructing hierarchal models of mobile GUIs from metadata associated with both graphical mock-ups (i.e., created by designers using photo-editing software) and running instances of the GUI from the corresponding implementation. Second, we develop an approach that completely automates prototyping of GUIs for mobile apps. This approach, called ReDraw, is able to transform an image of a mobile app GUI into runnable code by detecting discrete GUI-components using computer vision techniques, classifying these components into proper functional categories (e.g., button, dropdown menu) using a Convolutional Neural Network (CNN), and assembling these components into realistic code. Finally, we design a novel approach for automated testing of mobile apps, called CrashScope, that explores a given android app using systematic input generation with the intrinsic goal of triggering crashes. The GUI-based input generation engine is driven by a combination of static and dynamic analyses that create a model of an app\u27s GUI and targets common, empirically derived root causes of crashes in android apps. We illustrate that the techniques presented in this dissertation represent significant advancements in mobile development processes through a series of empirical investigations, user studies, and industrial case studies that demonstrate the effectiveness of these approaches and the benefit they provide developers

Fundamental Approaches to Software Engineering

computer software maintenance; computer software selection and evaluation; formal logic; formal methods; formal specification; programming languages; semantics; software engineering; specifications; verificatio

Multi-objective Search-based Mobile Testing

Despite the tremendous popularity of mobile applications, mobile testing still relies heavily on manual testing. This thesis presents mobile test automation approaches based on multi-objective search. We introduce three approaches: Sapienz (for native Android app testing), Octopuz (for hybrid/web JavaScript app testing) and Polariz (for using crowdsourcing to support search-based mobile testing). These three approaches represent the primary scientific and technical contributions of the thesis. Since crowdsourcing is, itself, an emerging research area, and less well understood than search-based software engineering, the thesis also provides the first comprehensive survey on the use of crowdsourcing in software testing (in particular) and in software engineering (more generally). This survey represents a secondary contribution. Sapienz is an approach to Android testing that uses multi-objective search-based testing to automatically explore and optimise test sequences, minimising their length, while simultaneously maximising their coverage and fault revelation. The results of empirical studies demonstrate that Sapienz significantly outperforms both the state-of-the-art technique Dynodroid and the widely-used tool, Android Monkey, on all three objectives. When applied to the top 1,000 Google Play apps, Sapienz found 558 unique, previously unknown crashes. Octopuz reuses the Sapienz multi-objective search approach for automated JavaScript testing, aiming to investigate whether it replicates the Sapienz’ success on JavaScript testing. Experimental results on 10 real-world JavaScript apps provide evidence that Octopuz significantly outperforms the state of the art (and current state of practice) in automated JavaScript testing. Polariz is an approach that combines human (crowd) intelligence with machine (computational search) intelligence for mobile testing. It uses a platform that enables crowdsourced mobile testing from any source of app, via any terminal client, and by any crowd of workers. It generates replicable test scripts based on manual test traces produced by the crowd workforce, and automatically extracts from these test traces, motif events that can be used to improve search-based mobile testing approaches such as Sapienz

Designing to support impression management

This work investigates impression management and in particular impression management using ubiquitous technology. Generally impression management is the process through which people try to influence the impressions that others have about them. In particular, impression management focuses on the flow of information between a performer and his/her audience, with control over what is presented to whom being of the utmost importance when trying to create the appropriate impression. Ubiquitous technology has provided opportunities for individuals to present themselves to others. However, the disconnection between presenter and audience over both time and space can result in individuals being misrepresented. This thesis outlines two important areas when trying to control the impression one gives namely, hiding and revealing, and accountability. By exploring these two themes the continuous evolution and dynamic nature of controlling the impression one gives is explored. While this ongoing adaptation is recognised by designers they do not always create technology that is sufficiently dynamic to support this process. As a result, this work attempts to answer three research questions: RQ1: How do users of ubicomp systems appropriate recorded data from their everyday activity and make it into a resource for expressing themselves to others in ways that are dynamically tailored to their ongoing social context and audience? RQ2: What technology can be built to support ubicomp system developers to design and develop systems to support appropriation as a central part of a useful or enjoyable user experience? RQ3: What software architectures best suit this type of appropriated interaction and developers’ designing to support such interaction? Through a thorough review of existing literature, and the extensive study of several large ubicomp systems, the issues when presenting oneself through technology are identified. The main issues identified are hiding and revealing, and accountability. These are built into a framework that acts as a reference for designers wishing to support impression management. An architecture for supporting impression management has also been developed that conforms to this framework and its evolution is documented later in the thesis. A demonstration of this architecture in a multi-player mobile experience is subsequently presented

Scripts in a Frame: A Framework for Archiving Deferred Representations

Web archives provide a view of the Web as seen by Web crawlers. Because of rapid advancements and adoption of client-side technologies like JavaScript and Ajax, coupled with the inability of crawlers to execute these technologies effectively, Web resources become harder to archive as they become more interactive. At Web scale, we cannot capture client-side representations using the current state-of-the art toolsets because of the migration from Web pages to Web applications. Web applications increasingly rely on JavaScript and other client-side programming languages to load embedded resources and change client-side state. We demonstrate that Web crawlers and other automatic archival tools are unable to archive the resulting JavaScript-dependent representations (what we term deferred representations), resulting in missing or incorrect content in the archives and the general inability to replay the archived resource as it existed at the time of capture. Building on prior studies on Web archiving, client-side monitoring of events and embedded resources, and studies of the Web, we establish an understanding of the trends contributing to the increasing unarchivability of deferred representations. We show that JavaScript leads to lower-quality mementos (archived Web resources) due to the archival difficulties it introduces. We measure the historical impact of JavaScript on mementos, demonstrating that the increased adoption of JavaScript and Ajax correlates with the increase in missing embedded resources. To measure memento and archive quality, we propose and evaluate a metric to assess memento quality closer to Web users’ perception. We propose a two-tiered crawling approach that enables crawlers to capture embedded resources dependent upon JavaScript. Measuring the performance benefits between crawl approaches, we propose a classification method that mitigates the performance impacts of the two-tiered crawling approach, and we measure the frontier size improvements observed with the two-tiered approach. Using the two-tiered crawling approach, we measure the number of client-side states associated with each URI-R and propose a mechanism for storing the mementos of deferred representations. In short, this dissertation details a body of work that explores the following: why JavaScript and deferred representations are difficult to archive (establishing the term deferred representation to describe JavaScript dependent representations); the extent to which JavaScript impacts archivability along with its impact on current archival tools; a metric for measuring the quality of mementos, which we use to describe the impact of JavaScript on archival quality; the performance trade-offs between traditional archival tools and technologies that better archive JavaScript; and a two-tiered crawling approach for discovering and archiving currently unarchivable descendants (representations generated by client-side user events) of deferred representations to mitigate the impact of JavaScript on our archives. In summary, what we archive is increasingly different from what we as interactive users experience. Using the approaches detailed in this dissertation, archives can create mementos closer to what users experience rather than archiving the crawlers’ experiences on the Web

Promoting Andean children's learning of science through cultural and digital tools

Conference Theme: To see the world and a grain of sand: Learning across levels of space, time, and scaleIn Peru, there is a large achievement gap in rural schools. In order to overcome this problem, the study aims to design environments that enhance science learning through the integration of ICT with cultural artifacts, respecting the Andean culture and empower rural children to pursue lifelong learning. This investigation employs the Cultural-Historical Activity Theory (CHAT) framework, and the Design-Based Research (DBR) methodology using an iterative process of design, implementation and evaluation of the innovative practice.published_or_final_versio