Interplayable surface: an exploration on augmented GUI that co-exists with physical environments

The main goal of this experiment-driven thesis is to envision and design an interactive GUI1(graphic user interface) that coexists with physical surfaces. Based on an understanding of user behavioral patterns for getting access to information in these types of situations, experimentations and prototypes are implemented and tested with participants. In particular, to observe the user behavioral pattern for augmented GUI within certain environments and circumstances, this thesis presents several types of participatory experimentations with physical GUIs. The experiment participants were encouraged to participate in re-creates and reorganizes physical GUI, relating to their own situational specificity or informational tendencies they have. Based on extracted insights from research and experiments, in the last phase, I propose two thesis models about how interactive GUI applies to a physical environment: simulation mock-ups for user scenarios of augmented GUI and interactive GUI surface combined with projection mapping. Related to people’s behavioral patterns on augmented GUI, the thesis models will show several types of information structures and interactions. Also, in framing the overall data structure and wireframe for the thesis product model, informative affordance corresponding with users’ situational specificity2 is considered as a crucial direction point, actualized on an artifact in a perceptible way. Through experimentally prototyping a thesis model, consequently, I would like to expand the speculative usability interactive GUI will feature in the near future

Infants, interfaces, and intermediation: digital parenting and the production of 'iPad baby' videos on YouTube

We investigate the ways young children’s use of mobile touchscreen interfaces is both understood and shaped by parents through the production of YouTube videos and discussions in associated comment threads. This analysis expands on, and departs from, theories of parental mediation, which have traditionally been framed through a media effects approach in analyzing how parents regulate their children’s use of broadcast media, such as television, within family life. We move beyond the limitations of an effects framing through more culturally and materially oriented theoretical lenses of mediation, considering the role mobile interfaces now play in the lives of infants through analysis of the ways parents intermediate between domestic spaces and networked publics. We propose the concept of intermediation, which builds on insights from critical interface studies as well as cultural industries literature to help account for these expanded aspects of digital parenting. Here, parents are not simply moderating children’s media use within the home, but instead operating as an intermediary in contributing to online representations and discourses of children’s digital culture. This intermediary role of parents engages with ideological tensions in locating notions of “naturalness:” the iPad’s gestural interface or the child’s digital dexterity

Social shaping of digital publishing: exploring the interplay between culture and technology

The processes and forms of electronic publishing have been changing since the advent of the Web. In recent years, the open access movement has been a major driver of scholarly communication, and change is also evident in other fields such as e-government and e-learning. Whilst many changes are driven by technological advances, an altered social reality is also pushing the boundaries of digital publishing. With 23 articles and 10 posters, Elpub 2012 focuses on the social shaping of digital publishing and explores the interplay between culture and technology. This book contains the proceedings of the conference, consisting of 11 accepted full articles and 12 articles accepted as extended abstracts. The articles are presented in groups, and cover the topics: digital scholarship and publishing; special archives; libraries and repositories; digital texts and readings; and future solutions and innovations. Offering an overview of the current situation and exploring the trends of the future, this book will be of interest to all those whose work involves digital publishing

Enactivism and ethnomethodological conversation analysis as tools for expanding Universal Design for Learning: the case of visually impaired mathematics students

Blind and visually impaired mathematics students must rely on accessible materials such as tactile diagrams to learn mathematics. However, these compensatory materials are frequently found to offer students inferior opportunities for engaging in mathematical practice and do not allow sensorily heterogenous students to collaborate. Such prevailing problems of access and interaction are central concerns of Universal Design for Learning (UDL), an engineering paradigm for inclusive participation in cultural praxis like mathematics. Rather than directly adapt existing artifacts for broader usage, UDL process begins by interrogating the praxis these artifacts serve and then radically re-imagining tools and ecologies to optimize usability for all learners. We argue for the utility of two additional frameworks to enhance UDL efforts: (a) enactivism, a cognitive-sciences view of learning, knowing, and reasoning as modal activity; and (b) ethnomethodological conversation analysis (EMCA), which investigates participants’ multimodal methods for coordinating action and meaning. Combined, these approaches help frame the design and evaluation of opportunities for heterogeneous students to learn mathematics collaboratively in inclusive classrooms by coordinating perceptuo-motor solutions to joint manipulation problems. We contextualize the thesis with a proposal for a pluralist design for proportions, in which a pair of students jointly operate an interactive technological device

Embodied interaction with guitars: instruments, embodied practices and ecologies

In this thesis I investigate the embodied performance preparation practices of guitarists to design and develop tools to support them. To do so, I employ a series of human-centred design methodologies such as design ethnography, participatory design, and soma design. The initial ethnographic study I conducted involved observing guitarists preparing to perform individually and with their bands in their habitual places of practice. I also interviewed these musicians on their preparation activities. Findings of this study allowed me to chart an ecology of tools and resources employed in the process, as well as pinpoint a series of design opportunities for augmenting guitars, namely supporting (1) encumbered interactions, (2) contextual interactions, and (3) connected interactions. Going forward with the design process I focused on remediating encumbered interactions that emerge during performance preparation with multimedia devices, particularly during instrumental transcription. I then prepared and ran a series of hands-on co-design workshops with guitarists to discuss five media controller prototypes, namely, instrument-mounted controls, pedal-based controls, voice-based controls, gesture-based controls, and “music-based” controls. This study highlighted the value that guitarists give to their guitars and to their existing practice spaces, tools, and resources by critically reflecting on how these interaction modalities would support or disturb their existing embodied preparation practices with the instrument. In parallel with this study, I had the opportunity to participate in a soma design workshop (and then prepare my own) in which I harnessed my first-person perspective of guitar playing to guide the design process. By exploring a series of embodied ideation and somatic methods, as well as materials and sensors across several points of contact between our bodies and the guitar, we collaboratively ideated a series of design concepts for guitar across both workshops, such as a series of breathing guitars, stretchy straps, and soft pedals. I then continued to develop and refine the Stretchy Strap concept into a guitar strap augmented with electronic textile stretch sensors to harness it as an embodied media controller to remediate encumbered interaction during musical transcription with guitar when using secondary multimedia resources. The device was subsequently evaluated by guitarists at a home practicing space, providing insights on nuanced aspects of its embodied use, such as how certain media control actions like play and pause are better supported by the bodily gestures enacted with the strap, whilst other actions, like rewinding the play back or setting in and out points for a loop are better supported by existing peripherals like keyboards and mice, as these activities do not necessarily happen in the flow of the embodied practice of musical transcription. Reflecting on the overall design process, a series of considerations are extracted for designing embodied interactions with guitars, namely, (1) considering the instrument and its potential for augmentation, i.e., considering the shape of the guitar, its material and its cultural identity, (2) considering the embodied practices with the instrument, i.e., the body and the subjective felt experience of the guitarist during their skilled embodied practices with the instrument and how these determine its expert use according to a particular instrumental tradition and/or musical practice, and (3) considering the practice ecology of the guitarist, i.e., the tools, resources, and spaces they use according to their practice

Creative Haptic Interface Design for the Aging Population

Audiovisual human-computer-interfaces still make up the majority of content to the public; however, haptic interfaces offer unique advantage over the dominant information infrastructure, particularly for users with a disability or diminishing cognitive and physical skills like the elderly. The tactile sense allows users to integrate new, unobstructive channels for digital information into their sensorium, one that is less likely to be overwhelmed compared to vision and audition. Haptics research focus on the development of hardware, improving resolution, modality, and fidelity of the actuators. Despite the technological limitations, haptic interfaces are shown to reinforce physical skill acquisition, therapy, and communication. This chapter will present key characteristics intuitive tactile interfaces should capture for elderly end-users; sample projects will showcase unique applications and designs that identify the limitations of the UI

Investigando Natural User Interfaces (NUIs) : tecnologias e interação em contexto de acessibilidade

Orientador: Maria Cecília Calani BaranauskasTese (doutorado) - Universidade Estadual de Campinas, Instituto de ComputaçãoResumo: Natural User Interfaces (NUIs) representam um novo paradigma de interação, com a promessa de ser mais intuitivo e fácil de usar do que seu antecessor, que utiliza mouse e teclado. Em um contexto no qual as tecnologias estão cada vez mais invisíveis e pervasivas, não só a quantidade mas também a diversidade de pessoas que participam deste contexto é crescente. Nesse caso, é preciso estudar como esse novo paradigma de interação de fato consegue ser acessível a todas as pessoas que podem utilizá-lo no dia-a-dia. Ademais, é preciso também caracterizar o paradigma em si, para entender o que o torna, de fato, natural. Portanto, nesta tese apresentamos o caminho que percorremos em busca dessas duas respostas: como caracterizar NUIs, no atual contexto tecnológico, e como tornar NUIs acessíveis para todos. Para tanto, primeiro apresentamos uma revisão sistemática de literatura com o estado da arte. Depois, mostramos um conjunto de heurísticas para o design e a avaliação de NUIs, que foram aplicadas em estudos de caso práticos. Em seguida, estruturamos as ideias desta pesquisa dentro dos artefatos da Semiótica Organizacional, e obtivemos esclarecimentos sobre como fazer o design de NUIs com Acessibilidade, seja por meio de Design Universal, seja para propor Tecnologias Assistivas. Depois, apresentamos três estudos de caso com sistemas NUI cujo design foi feito por nós. A partir desses estudos de caso, expandimos nosso referencial teórico e conseguimos, por fim, encontrar três elementos que resumem a nossa caracterização de NUI: diferenças, affordances e enaçãoAbstract: Natural User Interfaces (NUIs) represent a new interaction paradigm, with the promise of being more intuitive and easy to use than its predecessor, that utilizes mouse and keyboard. In a context where technology is becoming each time more invisible and pervasive, not only the amount but also the diversity of people who participate in this context is increasing. In this case, it must be studied how this new interaction paradigm can, in fact, be accessible to all the people who may use it on their daily routine. Furthermore, it is also necessary to characterize the paradigm itself, to understand what makes it, in fact, natural. Therefore, in this thesis we present the path we took in search of these two answers: how to characterize NUIs in the current technological context, and how to make NUIs accessible to all. To do so, first we present a systematic literature review with the state of the art. Then, we show a set of heuristics for the design and evaluation of NUIs, which were applied in practical study cases. Afterwards, we structure the ideas of this research into the Organizational Semiotics artifacts, and we obtain insights into how to design NUIs with Accessibility, be it through Universal Design, be it to propose Assistive Technologies. Then, we present three case studies with NUI systems which we designed. From these case studies, we expanded our theoretical references were able to, finally, find three elements that sum up our characterization of NUI: differences, affordances and enactionDoutoradoCiência da ComputaçãoDoutora em Ciência da Computação160911/2015-0CAPESCNP

Readiness for digital learning: examining self-reported and observed mobile competencies as steps toward more effective learner readiness assessment

This mixed-methods study explores a digital-competencies survey tool for probing the readiness of higher-education students for digital learning, in three steps. First, it develops a methodology for capturing and analysing authentic, digital-learning activity, and relating this activity to reported digital competencies. Second, it explores situational factors shaping human-computer interactions through video-based, case-study analyses. Finally, it proposes a threshold approach to self-reported digital competencies, which may help identify students requiring greater preparedness for optimal functioning as digital learners. Quantitative findings include generally strong, positive correlations between self-reported competencies and performance quality. Qualitative case-study analyses highlight four sets of situational factors influencing performance: (a) task/scenario difficulty; (b) comfort with device; (c) engagement, frustration and fatigue; and (d) persistence and activity-completion strategy. In the end, this study recommends the General Technology Competency and Use instrument as a promising tool for probing participant readiness for digital learning, and offers methodologies for future performance studies