Automation of CAD model based assembly simulations using motion capture

The manufacturing industry often uses text and video as a supplement for worker training procedures. Unfortunately, text is often difficult to follow and video lends itself to occlusion issues. CAD based animations of assembly operations can overcome these shortcomings while offering greater potential for operations analysis and simulation flexibility. However, creating such CAD based training simulations manually is a time intensive task and many a times fail to reveal the practical assembly issues faced in the real world. Thus, it is highly beneficial to be able to automate these simulations using motion capture from a physical environment. This thesis research summary includes the development and demonstration of a low-cost versatile motion tracking system and its application for generating CAD based assembly simulations. The motion tracking system is practically attractive because it is inexpensive, wireless, and easily portable. The system development focus herein is on two important aspects. One is generation of model based simulation, and the other is motion capture. Multiple Wii Remotes (Wiimotes) are used to form vision systems to perform 3D motion tracking. All the 6 DOF of a part can be tracked with the help of four Infra-red (IR) LEDs mounted on the part to be tracked. The obtained data is fed in realtime to automatically generate an assembly simulation of object models represented by Siemens NX5 CAD software. A Wiimote Vision System Setup Toolkit has been developed to help users in setting up a new vision system using Wiimotes given the required volume and floor area to be tracked. Implementation examples have been developed with different physical assemblies to demonstrate the capabilities of the system --Abstract, page iii

Integration of multiple vision systems and toolbox development

Depending on the required coverage, multiple cameras with different fields of view, positions and orientations can be employed to form a motion tracking system. Correctly and efficiently designing and setting up a multi-camera vision system presents a technical challenge. This thesis describes the development and application of a toolbox that can help the user to design a multi-camera vision system. Using the parameters of cameras, including their positions and orientations, the toolbox can calculate the volume covered by the system and generate its visualization for a given tracking area. The cameras can be repositioned and reoriented using toolbox to generate the visualization of the volume covered. Finally, this thesis describes how to practically implement and achieve a proper multi-camera setup. This thesis describes the integration of multiple cameras for vision system development based on Svoboda\u27s and Horn\u27s algorithms. Also, Dijkstra\u27s algorithm is implemented to estimate the tracking error between the master vision system and any of the slave vision systems. The toolbox is evaluated by comparing the calculated and actual covered volumes of a multi-camera system. The toolbox also is evaluated for its error estimation. The multi-camera vision system design is implemented using the developed toolbox for a virtual fastening operation of an aircraft fuselage in a computer-automated virtual environment (CAVE) --Abstract, page iii

Sonic interaction with a virtual orchestra of factory machinery

This paper presents an immersive application where users receive sound and visual feedbacks on their interactions with a virtual environment. In this application, the users play the part of conductors of an orchestra of factory machines since each of their actions on interaction devices triggers a pair of visual and audio responses. Audio stimuli were spatialized around the listener. The application was exhibited during the 2013 Science and Music day and designed to be used in a large immersive system with head tracking, shutter glasses and a 10.2 loudspeaker configuration.Comment: Sonic Interaction for Virtual Environments, Minneapolis : United States (2014

The development and applications of serious games in the public services: defence and health

This thesis was submitted for the award of Doctor of Philosophy and was awarded by Brunel University London.The latest advances of Virtual Reality technologies and three-dimensional graphics, as well as the developments in Gaming Technologies in the recent years, have stemmed the proliferation of Serious Games in a broader spectrum of research applications. Among the most popular areas of application are public services such as Defence and Health, where digital technologies realise new challenges and opportunities for research and development of Serious Games and for a variety of contexts. As with all games, the user engagement is elevated and apart from the entertaining aspect, Serious Games serve as a novel and promising alternative experience to knowledge transfer. Furthermore, Serious Games bring to the end user and the overall society a series of attractive benefits. These benefits include safety, cost-effectiveness, increased motivation and personalisation. Hence, this Thesis aims to investigate novel approaches of developing Serious Games that utilise the recent advances of Virtual Reality and Gaming Technology and facilitate the aforementioned benefits. The process of design and development of the novel tools and applications follow an iterative manner and are driven by the review of the available literature as well as end-user feedbackEPSRC (Engineering and Physical Sciences Research Council ) , MOD (UK), NHS (UK

The use of the Nintendo Wii in motor rehabilitation for virtual reality interventions:a literature review

Several review articles have been published on the use of Virtual Reality (VR) in motor rehabilitation. The majority of these focus on the effectiveness of VR on improving motor function using relatively expensive commercial tools and technologies including robotics, cybergloves, cybergrasps, joysticks, force sensors and motion capture systems. However, we present the case in this chapter that game sensors and VR technologies which can be customized and reconfigured, such as the Nintendo Wii, provide an alternative and affordable VR intervention for rehabilitation. While the performance of many of the Wii based interventions in motor rehabilitation are currently the focus of investigation by researchers, an extensive and holistic discussion on this subject does not yet exist. As such, the purpose of this chapter is to provide readers with an understanding of the advantages and limitations of the Nintendo Wii game sensor device (and its associated accessories) for motor rehabilitation and in addition, to outline the potential for incorporating these into clinical interventions for the benefit of patients and therapists

Transforming musical performance: activating the audience as digital collaborators

Digital technologies have transformed the performance practice, recording and distribution technologies, economy and sonic landscape of music in a process of change that began in the early 1980s. Recent technological developments have opened up the possibility of embodied interaction between audiences and performers, reframing music performance as a collaborative improvisatory space that affords Interactive Musical Participation. The research in this practice-based thesis looks at the relationship and experience of audience members and musicians exploring Interactive Musical Participation within the wide stylistic framework of contemporary jazz. It also studies the potential for the creation of compositional, technological and performance protocols to enable successful Interactive Musical Participation. This has been achieved through a process of mapping the methodology behind the composition, technical infrastructure, performances and post-performance analysis of a series of musical artefacts. Cook (2001 and 2009) suggests that researchers in this field should “Make a piece, not an instrument or controller” and this dictum has influenced the development of the technical infrastructure for this research. Easily accessible and low-cost digital audio workstations Ableton Live (2017) and Logic Pro X (Apple, 2019) as well as the digital protocols Open Sound Control (OSC) (Opensoundcontrol.org) have been utilised to deliver the programming and networking requirements. A major innovation stemming from this project has been the development of the Deeper Love Soundpad App, a sample playback app for Apple smartphones and iPads, in collaboration with Dr. Rob Toulson. The theoretical background to this research has been informed by actornetwork theory, the sociological approach developed by Bruno Latour (2005), Michel Callon (1986) and John Law (1992). Actor-network theory (ANT) provides a framework for understanding the mechanics of power and organisation within heterogeneous non-hierarchical networks. Mapping and analysing the ANT networks and connections created by the research performances has provided valuable data in the Interactive Musical Participatio

Bridging The Gap Between Fun And Fitness: Instructional Techniques And Real-world Applications For Full-body Dance Games

Full-body controlled games offer the opportunity for not only entertainment, but education and exercise as well. Refined gameplay mechanics and content can boost intrinsic motivation and keep people playing over a long period of time, which is desirable for individuals who struggle with maintaining a regular exercise program. Within this gameplay genre, dance rhythm games have proven to be popular with game console owners. Yet, while other types of games utilize story mechanics that keep players engaged for dozens of hours, motion-controlled dance games are just beginning to incorporate these elements. In addition, this control scheme is still young, only becoming commercially available in the last few years. Instructional displays and clear real-time feedback remain difficult challenges. This thesis investigates the potential for full-body dance games to be used as tools for entertainment, education, and fitness. We built several game prototypes to investigate visual, aural, and tactile methods for instruction and feedback. We also evaluated the fitness potential of the game Dance Central 2 both by itself and with extra game content which unlocked based on performance. Significant contributions include a framework for running a longitudinal video game study, results indicating high engagement with some fitness potential, and informed discussion of how dance games could make exertion a more enjoyable experience

Development of Kinectᵀᴿ applications for assembly simulation and ergonomic analysis

Marker-less motion capture technology has been harnessed for several years to track human movements for developing various applications. Recently, with the launch of Microsoft Kinect, researchers have been keenly interested in developing applications using this device. Since Kinect is very inexpensive (only $110 at the time of writing this thesis), it is a low-cost and a promising substitute for the comparatively expensive marker-based motion capture systems. Though it is principally designed for home entertainment, numerous applications can be developed with the capabilities of Kinect. The skeleton data of a human being tracked by a single Kinect device is enough to simulate the human movements, in some cases. However, it is highly desirable to develop a multiple Kinect system to enhance the tracking volume and to address an issue of occlusions. This thesis presents a novel approach for addressing the issue of interference of infrared light patterns while using multiple Kinect devices for human motion capture without lowering the frame rate. This research also presents a software solution to obtain skeleton data from multiple Kinect devices using Kinect for Windows SDK. It also discusses the development of an application involving auto scaling of a human model in digital human modeling software by Siemens Jack and human motion simulation using skeleton tracking data from Kinect to assist the industries with a flexible tool for ergonomic analysis. Further, the capability of this application for obtaining assembly simulations of fastening operations on an aircraft fuselage is also presented. --Abstract, page iii

Geometric Surface Processing and Virtual Modeling

In this work we focus on two main topics "Geometric Surface Processing" and "Virtual Modeling". The inspiration and coordination for most of the research work contained in the thesis has been driven by the project New Interactive and Innovative Technologies for CAD (NIIT4CAD), funded by the European Eurostars Programme. NIIT4CAD has the ambitious aim of overcoming the limitations of the traditional approach to surface modeling of current 3D CAD systems by introducing new methodologies and technologies based on subdivision surfaces in a new virtual modeling framework. These innovations will allow designers and engineers to transform quickly and intuitively an idea of shape in a high-quality geometrical model suited for engineering and manufacturing purposes. One of the objective of the thesis is indeed the reconstruction and modeling of surfaces, representing arbitrary topology objects, starting from 3D irregular curve networks acquired through an ad-hoc smart-pen device. The thesis is organized in two main parts: "Geometric Surface Processing" and "Virtual Modeling". During the development of the geometric pipeline in our Virtual Modeling system, we faced many challenges that captured our interest and opened new areas of research and experimentation. In the first part, we present these theories and some applications to Geometric Surface Processing. This allowed us to better formalize and give a broader understanding on some of the techniques used in our latest advancements on virtual modeling and surface reconstruction. The research on both topics led to important results that have been published and presented in articles and conferences of international relevance