Continuous Surface Rendering, Passing from CAD to Physical Representation

This paper describes a desktop‐mechatronic interface that has been conceived to support designers in the evaluation of aesthetic virtual shapes. This device allows a continuous and smooth free hand contact interaction on a real and developable plastic tape actuated by a servo‐controlled mechanism. The objective in designing this device is to reproduce a virtual surface with a consistent physical rendering well adapted to designers' needs. The desktop‐mechatronic interface consists in a servo‐actuated plastic strip that has been devised and implemented using seven interpolation points. In fact, by using the MEC (Minimal Energy Curve) Spline approach, a developable real surface is rendered taking into account the CAD geometry of the virtual shapes. In this paper, we describe the working principles of the interface by using both absolute and relative approaches to control the position on each single control point on the MEC spline. Then, we describe the methodology that has been implemented, passing from the CAD geometry, linked to VisualNastran in order to maintain the parametric properties of the virtual shape. Then, we present the co‐ simulation between VisualNastran and MATLAB/Simulink used for achieving this goal and controlling the system and finally, we present the results of the subsequent testing session specifically carried out to evaluate the accuracy and the effectiveness of the mechatronic device

Evaluating Industrial Products in an Innovative Visual-Olfactory Environment

The sense of smell has a great importance in our daily life. Recently, smells have been used for marketing purposes for improving the people's mood and for communicating information about products as household cleaners and food. However, the scent design discipline can be used for creating a “scent identity” of these products not traditionally associated to a specific smell, in order to communicate their features to customers. In the area of virtual reality (VR), several researches concerned the integration of smells in virtual environments. The research questions addressed in this paper concern if virtual prototypes (VP), including smell simulation, can be used for evaluating products as effectively as studies performed in real environments, and also if smells can enhance the users' sense of presence in virtual environments. For this purpose, a VR experimental framework including a prototype of a wearable olfactory display (wOD) has been set up, and experimental tests have been carried out.</jats:p

Augmented interaction for custom-fit products by means of interaction devices at low costs

This Ph.D thesis refers to a research project that aims at developing an innovative platform to design lower limb prosthesis (both for below and above knee amputation) centered on the virtual model of the amputee and based on a computer-aided and knowledge-guided approach. The attention has been put on the modeling tool of the socket, which is the most critical component of the whole prosthesis. The main aim has been to redesign and develop a new prosthetic CAD tool, named SMA2 (Socket Modelling Assistant2) exploiting a low-cost IT technologies (e.g. hand/finger tracking devices) and making the user’s interaction as much as possible natural and similar to the hand-made manipulation. The research activities have been carried out in six phases as described in the following. First, limits and criticalities of the already available modeling tool (namely SMA) have been identified. To this end, the first version of SMA has been tested with Ortopedia Panini and the orthopedic research group of Salford University in Manchester with real case studies. Main criticalities were related to: (i) automatic reconstruction of the residuum geometric model starting from medical images, (ii) performance of virtual modeling tools to generate the socket shape, and (iii) interaction mainly based on traditional devices (e.g., mouse and keyboard). The second phase lead to the software reengineering of SMA according to the limits identified in the first phase. The software architecture has been re-designed adopting an object-oriented paradigm and its modularity permits to remove or add new features in a very simple way. The new modeling system, i.e. SMA2, has been totally implemented using open source Software Development Kit-SDK (e.g., Visualization ToolKit VTK, OpenCASCADE and Qt SDK) and based on low cost technology. It includes: • A new module to automatically reconstruct the 3D model of the residual limb from MRI images. In addition, a new procedure based on low-cost technology, such as Microsoft Kinect V2 sensor, has been identified to acquire the 3D external shape of the residuum. • An open source software library, named SimplyNURBS, for NURBS modeling and specifically used for the automatic reconstruction of the residuum 3D model from medical images. Even if, SimplyNURBS has been conceived for the prosthetic domain, it can be used to develop NURBS-based modeling tools for a range of applicative domains from health-care to clothing design. • A module for mesh editing to emulate the hand-made operations carried out by orthopedic technicians during traditional socket manufacturing process. In addition several virtual widgets have been implemented to make available virtual tools similar to the real ones used by the prosthetist, such as tape measure and pencil. • A Natural User Interface (NUI) to allow the interaction with the residuum and socket models using hand-tracking and haptic devices. • A module to generate the geometric models for additive manufacturing of the socket. The third phase concerned the study and design of augmented interaction with particular attention to the Natural User Interface (NUI) for the use of hand-tracking and haptic devices into SMA2. The NUI is based on the use of the Leap Motion device. A set of gestures, mainly iconic and suitable for the considered domain, has been identified taking into account ergonomic issues (e.g., arm posture) and ease of use. The modularity of SMA2 permits us to easily generate the software interface for each device for augmented interaction. To this end, a software module, named Tracking plug-in, has been developed to automatically generate the source code of software interfaces for managing the interaction with low cost hand-tracking devices (e.g., Leap Motion and Intel Gesture Camera) and replicate/emulate manual operations usually performed to design custom-fit products, such medical devices and garments. Regarding haptic rendering, two different devices have been considered, the Falcon Novint, and a haptic mouse developed in-house. In the fourth phase, additive manufacturing technologies have been investigated, in particular FDM one. 3D printing has been exploited in order to permit the creation of trial sockets in laboratory to evaluate the potentiality of SMA2. Furthermore, research activities have been done to study new ways to design the socket. An innovative way to build the socket has been developed based on multi-material 3D printing. Taking advantage of flexible material and multi-material print possibility, new 3D printers permit to create object with soft and hard parts. In this phase, issues about infill, materials and comfort have been faced and solved considering different compositions of materials to re-design the socket shape. In the fifth phase the implemented solution, integrated within the whole prosthesis design platform, has been tested with a transfemoral amputee. Following activities have been performed: • 3D acquisition of the residuum using MRI and commercial 3D scanning systems (low cost and professional). • Creation of the residual limb and socket geometry. • Multi-material 3D printing of the socket using FDM technology. • Gait analysis of the amputee wearing the socket using a markerless motion capture system. • Acquisition of contact pressure between residual limb and a trial socket by means of Teskan’s F-Socket System. Acquired data have been combined inside an ad-hoc developed application, which permits to simultaneously visualize pressure data on the 3D model of the residual lower limb and the animation of gait analysis. Results and feedback have been possible thanks to this application that permits to find correlation between several phases of the gait cycle and the pressure data at the same time. Reached results have been considered very interested and several tests have been planned in order to try the system in orthopedic laboratories in real cases. The reached results have been very useful to evaluate the quality of SMA2 as a future instruments that can be exploited for orthopedic technicians in order to create real socket for patients. The solution has the potentiality to begin a potential commercial product, which will be able to substitute the classic procedure for socket design. The sixth phase concerned the evolution of SMA2 as a Mixed Reality environment, named Virtual Orthopedic LABoratory (VOLAB). The proposed solution is based on low cost devices and open source libraries (e.g., OpenCL and VTK). In particular, the hardware architecture consists of three Microsoft Kinect v2 for human body tracking, the head mounted display Oculus Rift SDK 2 for 3D environment rendering, and the Leap Motion device for hand/fingers tracking. The software development has been based on the modular structure of SMA2 and dedicated modules have been developed to guarantee the communication among the devices. At present, two preliminary tests have been carried out: the first to verify real-time performance of the virtual environment and the second one to verify the augmented interaction with hands using SMA2 modeling tools. Achieved results are very promising but, highlighted some limitations of this first version of VOLAB and improvements are necessary. For example, the quality of the 3D real world reconstruction, especially as far as concern the residual limb, could be improved by using two HD-RGB cameras together the Oculus Rift. To conclude, the obtained results have been evaluated very interested and encouraging from the technical staff of orthopedic laboratory. SMA2 will made possible an important change of the process to design the socket of lower limb prosthesis, from a traditional hand-made manufacturing process to a totally virtual knowledge-guided process. The proposed solutions and results reached so far can be exploited in other industrial sectors where the final product heavily depends on the human body morphology. In fact, preliminary software development has been done to create a virtual environment for clothing design by starting from the basic modules exploited in SMA2

Perception and performance: an evaluation of multimodal feedback for the assessment of curve shape differences

The EU-funded SATIN project sought to provide a multimodal interface to aid product designers in judging the quality of curved shapes. This thesis outlines a research programme designed to assist in the exploration of fundamental issues related to this project, and provide a means to evaluate the success of such interfaces more generally. Therefore, three studies were undertaken with the aim of exploring the value of haptic and sound feedback in the perception of curve shape differences, and through the knowledge gained provide an evaluative framework for the assessment of such interfaces. The first study found that visual, haptic, and visual-haptic perception was insufficient to judge discontinuities in curvature without some further augmentation. This led to a second study which explored the use of sound for conveying curve shape information. It was found that sine waves or harmonic sounds were most suited to for this task. The third study combined visual-haptic and auditory information. It was found that sound improved the perception of curve shape differences, although this was dependent upon the type of sonification method used. Further to this, data from studies one and three were used to identify gradient as the active mechanism of curve shape differentiation and provided a model for the prediction of these differences. Similarly performance data (response time, accuracy, and confidence) were analysed to produce a model for the prediction of user performance at varying degrees of task difficulty. The research undertaken across these studies was used to develop a framework to evaluate multimodal interfaces for curve shape exploration. In particular a ‘discount’ psychophysical method was proposed, along with predictive tools for the creation of perceptual and performance metrics, plus guidelines to aid development. This research has added to fundamental knowledge and provided a useful framework through which future multimodal interfaces may be evaluated

Perception and performance: an evaluation of multimodal feedback for the assessment of curve shape differences

The EU-funded SATIN project sought to provide a multimodal interface to aid product designers in judging the quality of curved shapes. This thesis outlines a research programme designed to assist in the exploration of fundamental issues related to this project, and provide a means to evaluate the success of such interfaces more generally. Therefore, three studies were undertaken with the aim of exploring the value of haptic and sound feedback in the perception of curve shape differences, and through the knowledge gained provide an evaluative framework for the assessment of such interfaces. The first study found that visual, haptic, and visual-haptic perception was insufficient to judge discontinuities in curvature without some further augmentation. This led to a second study which explored the use of sound for conveying curve shape information. It was found that sine waves or harmonic sounds were most suited to for this task. The third study combined visual-haptic and auditory information. It was found that sound improved the perception of curve shape differences, although this was dependent upon the type of sonification method used. Further to this, data from studies one and three were used to identify gradient as the active mechanism of curve shape differentiation and provided a model for the prediction of these differences. Similarly performance data (response time, accuracy, and confidence) were analysed to produce a model for the prediction of user performance at varying degrees of task difficulty. The research undertaken across these studies was used to develop a framework to evaluate multimodal interfaces for curve shape exploration. In particular a ‘discount’ psychophysical method was proposed, along with predictive tools for the creation of perceptual and performance metrics, plus guidelines to aid development. This research has added to fundamental knowledge and provided a useful framework through which future multimodal interfaces may be evaluated

Haptic and sound interface for shape rendering

This paper presents a system for the evaluation of the shape of aesthetic products. The evaluation of shapes is based on characteristic curves, which is a typical practice in the industrial design domain. The system, inspired by characteristic curves, is based on a haptic strip that conforms to a curve that the designer wishes to feel, explore, and analyze by physically touching it. The haptic strip is an innovative solution in the haptics domain, although it has some limitations concerning the domain of curves that can be actually represented. In order to extend this domain and make users feel the various curve features, for example curvature discontinuities, sound has been exploited as an additional information modality