Understanding Multi-Device Usage Patterns: Physical Device Configurations and Fragmented Workflows

To better ground technical (systems) investigation and interaction design of cross-device experiences, we contribute an in-depth survey of existing multi-device practices, including fragmented workflows across devices and the way people physically organize and configure their workspaces to support such activity. Further, this survey documents a historically significant moment of transition to a new future of remote work, an existing trend dramatically accelerated by the abrupt switch to work-from-home (and having to contend with the demands of home-at-work) during the COVID-19 pandemic. We surveyed 97 participants, and collected photographs of home setups and open-ended answers to 50 questions categorized in 5 themes. We characterize the wide range of multi-device physical configurations and identify five usage patterns, including: partitioning tasks, integrating multi-device usage, cloning tasks to other devices, expanding tasks and inputs to multiple devices, and migrating between devices. Our analysis also sheds light on the benefits and challenges people face when their workflow is fragmented across multiple devices. These insights have implications for the design of multi-device experiences that support people's fragmented workflows

Interaction techniques for mobile collocation

Research on mobile collocated interactions has been exploring situations where collocated users engage in collaborative activities using their personal mobile devices (e.g., smartphones and tablets), thus going from personal/individual toward shared/multiuser experiences and interactions. The proliferation of ever- smaller computers that can be worn on our wrists (e.g., Apple Watch) and other parts of the body (e.g., Google Glass), have expanded the possibilities and increased the complexity of interaction in what we term “mobile collocated” situations. The focus of this workshop is to bring together a community of researchers, designers and practitioners to explore novel interaction techniques for mobile collocated interactions

Accessibility and tangible interaction in distributed workspaces based on multi-touch surfaces

[EN] Traditional interaction mechanisms in distributed digital spaces often fail to consider the intrinsic properties of action, perception, and communication among workgroups, which may affect access to the common resources used to mutually organize information. By developing suitable spatial geometries and natural interaction mechanisms, distributed spaces can become blended where the physical and virtual boundaries of local and remote spaces merge together to provide the illusion of a single unified space. In this paper, we discuss the importance of blended interaction in distributed spaces and the particular challenges faced when designing accessible technology. We illustrate this discussion through a new tangible interaction mechanism for collaborative spaces based on tabletop system technology implemented with optical frames. Our tangible elements facilitate the exchange of digital information in distributed collaborative settings by providing a physical manifestation of common digital operations. The tangibles are designed as passive elements that do not require the use of any additional hardware or external power while maintaining a high degree of accuracy.This work was supported by the Spanish Ministry of Economy and Competitiveness and the European Regional Development Fund, through the ANNOTA Project (Ref. TIN2013-46036-C3-1-R).Salvador-Herranz, G.; Camba, J.; Contero, M.; Naya Sanchis, F. (2018). Accessibility and tangible interaction in distributed workspaces based on multi-touch surfaces. Universal Access in the Information Society. 17(2):247-256. https://doi.org/10.1007/s10209-017-0563-7S247256172Arkin, E.M., Chew, L.P., Huttenlocher, D.P., Kedem, K., Mitchell, J.S.B.: An efficiently computable metric for comparing polygonal shapes. IEEE Trans. Acoust. Speech Signal Process. 13(3), 209–216 (1991)Benyon, D.: Presence in blended spaces. Interact. Comput. 24(4), 219–226 (2012)Bhalla, M.R., Bhalla, A.V.: Comparative study of various touchscreen technologies. Int. J. Comput. Appl. 6(8), 12–18 (2010)Bradski, G., Kaehler, A.: Learning OpenCV: Computer Vision with the OpenCV Library. O’Reilly Media Inc., Newton (2008)Candela, E.S., Pérez, M.O., Romero, C.M., López, D.C.P., Herranz, G.S., Contero, M., Raya, M.A.: Humantop: a multi-object tracking tabletop. Multimed. Tools Appl. 70(3), 1837–1868 (2014)Cohen, J., Withgott, M., Piernot, P.: Logjam: a tangible multi-person interface for video logging. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 128–135. ACM (1999)Couture, N., Rivière, G., Reuter, P.: Geotui: a tangible user interface for geoscience. In: Proceedings of the 2nd International Conference on Tangible and Embedded Interaction, pp. 89–96. ACM (2008)de la Guía, E., Lozano, M.D., Penichet, V.R.: Cognitive rehabilitation based on collaborative and tangible computer games. In: 2013 7th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth), pp. 389–392. IEEE (2013)Dietz, P., Leigh, D.: Diamondtouch: a multi-user touch technology. In: Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology, pp. 219–226. ACM (2001)Falcão, T.P., Price, S.: What have you done! the role of ‘interference’ in tangible environments for supporting collaborative learning. In: Proceedings of the 9th International Conference on Computer Supported Collaborative Learning-Volume 1, pp. 325–334. International Society of the Learning Sciences (2009)Fallman, D.: Wear, point and tilt. In: Proceedings of the Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques, pp. 293–302. ACM Press (2002)Fishkin, K.P., Gujar, A., Harrison, B.L., Moran, T.P., Want, R.: Embodied user interfaces for really direct manipulation. Commun. ACM 43(9), 74–80 (2000)Fitzmaurice, G.W., Buxton, W.: An empirical evaluation of graspable user interfaces: towards specialized, space-multiplexed input. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 43–50. ACM (1997)Fitzmaurice, G.W., Ishii, H., Buxton, W.A.: Bricks: laying the foundations for graspable user interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 442–449. ACM Press (1995)Graham, R.L., Yao, F.F.: Finding the convex hull of a simple polygon. J. Algorithms 4(4), 324–331 (1983)Hartigan, J.A., Wong, M.A.: Algorithm as 136: a k-means clustering algorithm. J. R. Stat. Soc.: Ser. C (Appl. Stat.) 28(1), 100–108 (1979)Higgins, S.E., Mercier, E., Burd, E., Hatch, A.: Multi-touch tables and the relationship with collaborative classroom pedagogies: a synthetic review. Int. J. Comput. Support. Collab. Learn. 6(4), 515–538 (2011)Hinckley, K., Pausch, R., Goble, J.C., Kassell, N.F.: Passive real-world interface props for neurosurgical visualization. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 452–458. ACM (1994)Hinske, S.: Determining the position and orientation of multi-tagged objects using RFID technology. In: 5th Annual IEEE International Conference on Pervasive Computing and Communications Workshops, 2007. PerCom Workshops’07, pp. 377–381. IEEE (2007)Hornecker, E.: A design theme for tangible interaction: embodied facilitation. In: ECSCW 2005, pp. 23–43. Springer (2005)Hoshi, K., Öhberg, F., Nyberg, A.: Designing blended reality space: conceptual foundations and applications. In: Proceedings of the 25th BCS Conference on Human–Computer Interaction, pp. 217–226. British Computer Society (2011)Ishii, H.: Tangible User Interfaces. CRC Press, Boca Raton (2007)Ishii, H., Ullmer, B.: Tangible bits: towards seamless interfaces between people, bits and atoms. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 234–241. ACM (1997)Jacob, R.J., Girouard, A., Hirshfield, L.M., Horn, M.S., Shaer, O., Solovey, E.T., Zigelbaum, J.: Reality-based interaction: a framework for post-wimp interfaces. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 201–210. ACM (2008)Jetter, H.C., Dachselt, R., Reiterer, H., Quigley, A., Benyon, D., Haller, M.: Blended Interaction: Envisioning Future Collaborative Interactive Spaces. ACM, New York (2013)Jin, X., Han, J.: Quality threshold clustering. In: Sammut, C., Webb, G.I. (eds.) Encyclopedia of Machine Learning, pp. 820–820. Springer, Boston, MA (2011)Jordà, S., Geiger, G., Alonso, M., Kaltenbrunner, M.: The reactable: exploring the synergy between live music performance and tabletop tangible interfaces. In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction, pp. 139–146. ACM (2007)Kaltenbrunner, M., Bovermann, T., Bencina, R., Costanza, E.: Tuio: a protocol for table-top tangible user interfaces. In: Proceedings of the 6th International Workshop on Gesture in Human–Computer Interaction and Simulation, pp. 1–5 (2005)Kirk, D., Sellen, A., Taylor, S., Villar, N., Izadi, S.: Putting the physical into the digital: issues in designing hybrid interactive surfaces. In: Proceedings of the 23rd British HCI Group Annual Conference on People and Computers: Celebrating People and Technology, pp. 35–44. British Computer Society (2009)Marques, T., Nunes, F., Silva, P., Rodrigues, R.: Tangible interaction on tabletops for elderly people. In: International Conference on Entertainment Computing, pp. 440–443. Springer (2011)Müller, D.: Mixed reality systems. iJOE 5(S2), 10–11 (2009)Newton-Dunn, H., Nakano, H., Gibson, J.: Block jam: a tangible interface for interactive music. In: Proceedings of the 2003 Conference on New Interfaces for Musical Expression, pp. 170–177. National University of Singapore (2003)Patten, J., Recht, B., Ishii, H.: Audiopad: a tag-based interface for musical performance. In: Proceedings of the 2002 Conference on New Interfaces for Musical Expression, pp. 1–6. National University of Singapore (2002)Patten, J., Recht, B., Ishii, H.: Interaction techniques for musical performance with tabletop tangible interfaces. In: Proceedings of the 2006 ACM SIGCHI International Conference on Advances in Computer Entertainment Technology, p. 27. ACM (2006)PQLabs: Inc. http://multitouch.com/ . Retrieved on 16 October 2016Ryokai, K., Marti, S., Ishii, H.: I/o brush: drawing with everyday objects as ink. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, CHI’04, pp. 303–310. ACM, New York (2004). doi: 10.1145/985692.985731Salvador, G., Bañó, M., Contero, M., Camba, J.: Evaluation of a distributed collaborative workspace as a creativity tool in the context of design education. In: 2014 IEEE Frontiers in Education Conference (FIE) Proceedings, pp. 1–7. IEEE (2014)Salvador-Herranz, G., Contero, M., Camba, J.: Use of tangible marks with optical frame interactive surfaces in collaborative design scenarios based on blended spaces. In: International Conference on Cooperative Design, Visualization and Engineering, pp. 253–260. Springer (2014)Salvador-Herranz, G., Camba, J.D., Naya, F., Contero, M.: On the integration of tangible elements with multi-touch surfaces for the collaborative creation of concept maps. In: International Conference on Learning and Collaboration Technologies, pp. 177–186. Springer (2016)Schöning, J., Hook, J., Bartindale, T., Schmidt, D., Oliver, P., Echtler, F., Motamedi, N., Brandl, P., von Zadow, U.: Building interactive multi-touch surfaces. In: Müller-Tomfelde, C. (ed.) Tabletops-Horizontal Interactive Displays, pp. 27–49. Springer, London, UK (2010)Shaer, O., Hornecker, E.: Tangible user interfaces: past, present, and future directions. Found. Trends Hum. Comput. Interact. 3(1–2), 1–137 (2010)Shen, C., Everitt, K., Ryall, K.: Ubitable: Impromptu face-to-face collaboration on horizontal interactive surfaces. In: International Conference on Ubiquitous Computing, pp. 281–288. Springer (2003)Suzuki, H., Kato, H.: Algoblock: a tangible programming language, a tool for collaborative learning. In: Proceedings of 4th European Logo Conference, pp. 297–303 (1993)Suzuki, H., Kato, H.: Interaction-level support for collaborative learning: Algoblockan open programming language. In: The 1st International Conference on Computer Support for Collaborative Learning, pp. 349–355. L. Erlbaum Associates Inc. (1995)Terrenghi, L., Kirk, D., Richter, H., Krämer, S., Hilliges, O., Butz, A.: Physical handles at the interactive surface: exploring tangibility and its benefits. In: Proceedings of the Working Conference on Advanced Visual Interfaces, pp. 138–145. ACM (2008)Veltkamp, R.C.: Shape matching: similarity measures and algorithms. In: SMI 2001 International Conference on Shape Modeling and Applications, pp. 188–197. IEEE (2001)Weinberg, G., Gan, S.L.: The squeezables: Toward an expressive and interdependent multi-player musical instrument. Comput. Music J. 25(2), 37–45 (2001)Weiser, M.: Some computer science issues in ubiquitous computing. Commun. ACM 36(7), 75–84 (1993)Wilson, F.: The hand: how its use shapes the brain, language, and human culture. Vintage Series. Vintage Books (1998). https://books.google.es/books?id=l_Boy_-NkwUCZuckerman, O., Arida, S., Resnick, M.: Extending tangible interfaces for education: digital montessori-inspired manipulatives. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 859–868. ACM (2005

Designing for Cross-Device Interactions

Driven by technological advancements, we now own and operate an ever-growing number of digital devices, leading to an increased amount of digital data we produce, use, and maintain. However, while there is a substantial increase in computing power and availability of devices and data, many tasks we conduct with our devices are not well connected across multiple devices. We conduct our tasks sequentially instead of in parallel, while collaborative work across multiple devices is cumbersome to set up or simply not possible. To address these limitations, this thesis is concerned with cross-device computing. In particular it aims to conceptualise, prototype, and study interactions in cross-device computing. This thesis contributes to the field of Human-Computer Interaction (HCI)—and more specifically to the area of cross-device computing—in three ways: first, this work conceptualises previous work through a taxonomy of cross-device computing resulting in an in-depth understanding of the field, that identifies underexplored research areas, enabling the transfer of key insights into the design of interaction techniques. Second, three case studies were conducted that show how cross-device interactions can support curation work as well as augment users’ existing devices for individual and collaborative work. These case studies incorporate novel interaction techniques for supporting cross-device work. Third, through studying cross-device interactions and group collaboration, this thesis provides insights into how researchers can understand and evaluate multi- and cross-device interactions for individual and collaborative work. We provide a visualization and querying tool that facilitates interaction analysis of spatial measures and video recordings to facilitate such evaluations of cross-device work. Overall, the work in this thesis advances the field of cross-device computing with its taxonomy guiding research directions, novel interaction techniques and case studies demonstrating cross-device interactions for curation, and insights into and tools for effective evaluation of cross-device systems

"It's cleaner, definitely": Collaborative Process in Audio Production.

Working from vague client instructions, how do audio producers collaborate to diagnose what specifically is wrong with a piece of music, where the problem is and what to do about it? This paper presents a design ethnography that uncovers some of the ways in which two music producers co-ordinate their understanding of complex representations of pieces of music while working together in a studio. Our analysis shows that audio producers constantly make judgements based on audio and visual evidence while working with complex digital tools, which can lead to ambiguity in assessments of issues. We show how multimodal conduct guides the process of work and that complex media objects are integrated as elements of interaction by the music producers. The findings provide an understanding how people currently collaborate when producing audio, to support the design of better tools and systems for collaborative audio production in the future

A Comparison of Neuroelectrophysiology Databases

As data sharing has become more prevalent, three pillars - archives, standards, and analysis tools - have emerged as critical components in facilitating effective data sharing and collaboration. This paper compares four freely available intracranial neuroelectrophysiology data repositories: Data Archive for the BRAIN Initiative (DABI), Distributed Archives for Neurophysiology Data Integration (DANDI), OpenNeuro, and Brain-CODE. These archives provide researchers with tools to store, share, and reanalyze neurophysiology data though the means of accomplishing these objectives differ. The Brain Imaging Data Structure (BIDS) and Neurodata Without Borders (NWB) are utilized by these archives to make data more accessible to researchers by implementing a common standard. While many tools are available to reanalyze data on and off the archives' platforms, this article features Reproducible Analysis and Visualization of Intracranial EEG (RAVE) toolkit, developed specifically for the analysis of intracranial signal data and integrated with the discussed standards and archives. Neuroelectrophysiology data archives improve how researchers can aggregate, analyze, distribute, and parse these data, which can lead to more significant findings in neuroscience research.Comment: 25 pages, 8 figures, 1 tabl

Grid Portal Application

This project concerns EMMIL (E-Marketplace Model Integrated with Logistics) and its viability as a grid application. The model was evaluated on a single processor and on the SEEGRID network using MTA-SZTAKI\u27s P-GRADE Portal. A granularity heuristic was developed to guide the mapping of EMMIL datasets to processes. A portlet for P-GRADE Portal was also created to aid in data entry. Finally, pre-processing filters were added. These were designed to discard useless combinations and reduce overall computing time

Division of labour and sharing of knowledge for synchronous collaborative information retrieval

Synchronous collaborative information retrieval (SCIR) is concerned with supporting two or more users who search together at the same time in order to satisfy a shared information need. SCIR systems represent a paradigmatic shift in the way we view information retrieval, moving from an individual to a group process and as such the development of novel IR techniques is needed to support this. In this article we present what we believe are two key concepts for the development of effective SCIR namely division of labour (DoL) and sharing of knowledge (SoK). Together these concepts enable coordinated SCIR such that redundancy across group members is reduced whilst enabling each group member to benefit from the discoveries of their collaborators. In this article we outline techniques from state-of-the-art SCIR systems which support these two concepts, primarily through the provision of awareness widgets. We then outline some of our own work into system-mediated techniques for division of labour and sharing of knowledge in SCIR. Finally we conclude with a discussion on some possible future trends for these two coordination techniques