Natural user interfaces for interdisciplinary design review using the Microsoft Kinect

As markets demand engineered products faster, waiting on the cyclical design processes of the past is not an option. Instead, industry is turning to concurrent design and interdisciplinary teams. When these teams collaborate, engineering CAD tools play a vital role in conceptualizing and validating designs. These tools require significant user investment to master, due to challenging interfaces and an overabundance of features. These challenges often prohibit team members from using these tools for exploring designs. This work presents a method allowing users to interact with a design using intuitive gestures and head tracking, all while keeping the model in a CAD format. Specifically, Siemens\u27 Teamcenter® Lifecycle Visualization Mockup (Mockup) was used to display design geometry while modifications were made through a set of gestures captured by a Microsoft KinectTM in real time. This proof of concept program allowed a user to rotate the scene, activate Mockup\u27s immersive menu, move the immersive wand, and manipulate the view based on head position. This work also evaluates gesture usability and task completion time for this proof of concept system. A cognitive model evaluation method was used to evaluate the premise that gesture-based user interfaces are easier to use and learn with regards to time than a traditional mouse and keyboard interface. Using a cognitive model analysis tool allowed the rapid testing of interaction concepts without the significant overhead of user studies and full development cycles. The analysis demonstrated that using the KinectTM is a feasible interaction mode for CAD/CAE programs. In addition, the analysis pointed out limitations in the gesture interfaces ability to compete time wise with easily accessible customizable menu options

Assessing the effectiveness of direct gesture interaction for a safety critical maritime application

Multi-touch interaction, in particular multi-touch gesture interaction, is widely believed to give a more natural interaction style. We investigated the utility of multi-touch interaction in the safety critical domain of maritime dynamic positioning (DP) vessels. We conducted initial paper prototyping with domain experts to gain an insight into natural gestures; we then conducted observational studies aboard a DP vessel during operational duties and two rounds of formal evaluation of prototypes - the second on a motion platform ship simulator. Despite following a careful user-centred design process, the final results show that traditional touch-screen button and menu interaction was quicker and less erroneous than gestures. Furthermore, the moving environment accentuated this difference and we observed initial use problems and handedness asymmetries on some multi-touch gestures. On the positive side, our results showed that users were able to suspend gestural interaction more naturally, thus improving situational awareness

Vision systems with the human in the loop

The emerging cognitive vision paradigm deals with vision systems that apply machine learning and automatic reasoning in order to learn from what they perceive. Cognitive vision systems can rate the relevance and consistency of newly acquired knowledge, they can adapt to their environment and thus will exhibit high robustness. This contribution presents vision systems that aim at flexibility and robustness. One is tailored for content-based image retrieval, the others are cognitive vision systems that constitute prototypes of visual active memories which evaluate, gather, and integrate contextual knowledge for visual analysis. All three systems are designed to interact with human users. After we will have discussed adaptive content-based image retrieval and object and action recognition in an office environment, the issue of assessing cognitive systems will be raised. Experiences from psychologically evaluated human-machine interactions will be reported and the promising potential of psychologically-based usability experiments will be stressed

A Comparison of Gesture and Controller-based User Interfaces for 3D Design Reviews in Virtual Reality

We present a custom virtual reality (VR) hand-tracking user interface developed in Unity, and compare it with a similar industry level application (CTRL Elements) utilising controllers. Both applications afford spawning, relocating, reorienting, and rescaling 3D-models within a virtual environment in addition to horizontal, vertical, and rotational user movement options. The interfaces are tested with Oculus Quest 2 VR headset, which is also responsible for the hand-tracking. Our main motivation is to gain an insight into whether hand-tracking UIs could provide added value to industrial design review solutions. We also pinpoint the major advantages and flaws in the controllerless approach and report which of our gesture-based controls are the most intuitive and usable

Usability of vision-based interfaces

Vision-based interfaces can employ gestures to interact with an interactive system without touching it. Gestures are frequently modelled in laboratories, and usability testing should be carried out. However, often these interfaces present usability issues, and the great diversity of uses of these interfaces and the applications where they are used, makes it difficult to decide which factors to take into account in a usability test. In this paper, we review the literature to compile and analyze the usability factors and metrics used for vision-based interfaces.Postprint (published version

Evaluating body tracking interaction in floor projection displays with an elderly population

The recent development of affordable full body tracking sensors has made this technology accessible to millions of users and gives the opportunity to develop new natural user interfaces. In this paper we focused on developing 2 natural user interfaces that could easily be used by an elderly population for interaction with a floor projection display. One interface uses feet positions to control a cursor and feet distance to activate interaction. In the second interface, the cursor is controlled by ray casting the forearm into the projection and interaction is activated by hand pose. The interfaces were tested by 19 elderly participants in a point-and-click and a drag-and-drop task using a between-subjects experimental design. The usability and perceived workload for each interface was assessed as well as performance indicators. Results show a clear preference by the participants for the feet controlled interface and also marginal better performance for this method.info:eu-repo/semantics/publishedVersio

Gesture based persuasive interfaces for public ambient displays

Dissertação de Mestrado em Engenharia Informática 2º Semestre, 2011/2012This Master thesis studies how Public Ambient Displays (PAD) can be used as a tool to achieve behaviour change, through persuasive technology. In order to reach the goals of the thesis, an interactive public ambient display system called Motion-based Ambient Interactive Display (MAID) was developed. MAID is driven to motivate behaviour changes regarding domestic energy consumption, through a persuasive game interface based on gesture recognition technology. The developed prototype guides players through the different rooms of a house, where they have to find out what is wrong and practice the correct actions to save energy, using similar gestures to the ones they would use in real life to achieve the same goals. The system provides feedback regarding the consequences of each action, in order to make users aware of the consequences of their actions. The implementation of MAID is based on a purpose built, highly configurable and modular framework. It allows the administrator to fine tune and tweak the application to the necessities of the setup location constraints, by adjusting basic display properties, change image content or even modify the scripted gameplay itself. The scripted game system is flexible enough to allow the repurposing of the framework, beyond the previously defined theme, for future studies. The MAID was subjected to user testing, in order to show that it is possible to create a persuasive PAD interface, using seamless interaction methods, with the currently available technology, and use it to spread awareness of a cause, leading to behaviour change.Fundação para a Ciência e Tecnologia - project DEAP (PTDC/AAC-AMB/104834/2008); CITI/DI/FCT/UNL (PEst-OE/EEI/UI0527/201

GEMINI: A Generic Multi-Modal Natural Interface Framework for Videogames

In recent years videogame companies have recognized the role of player engagement as a major factor in user experience and enjoyment. This encouraged a greater investment in new types of game controllers such as the WiiMote, Rock Band instruments and the Kinect. However, the native software of these controllers was not originally designed to be used in other game applications. This work addresses this issue by building a middleware framework, which maps body poses or voice commands to actions in any game. This not only warrants a more natural and customized user-experience but it also defines an interoperable virtual controller. In this version of the framework, body poses and voice commands are respectively recognized through the Kinect's built-in cameras and microphones. The acquired data is then translated into the native interaction scheme in real time using a lightweight method based on spatial restrictions. The system is also prepared to use Nintendo's Wiimote as an auxiliary and unobtrusive gamepad for physically or verbally impractical commands. System validation was performed by analyzing the performance of certain tasks and examining user reports. Both confirmed this approach as a practical and alluring alternative to the game's native interaction scheme. In sum, this framework provides a game-controlling tool that is totally customizable and very flexible, thus expanding the market of game consumers.Comment: WorldCIST'13 Internacional Conferenc