Virtual reality for safe testing and development in collaborative robotics: challenges and perspectives

Collaborative robots (cobots) could help humans in tasks that are mundane, dangerous or where direct human contact carries risk. Yet, the collaboration between humans and robots is severely limited by the aspects of the safety and comfort of human operators. In this paper, we outline the use of extended reality (XR) as a way to test and develop collaboration with robots. We focus on virtual reality (VR) in simulating collaboration scenarios and the use of cobot digital twins. This is specifically useful in situations that are difficult or even impossible to safely test in real life, such as dangerous scenarios. We describe using XR simulations as a means to evaluate collaboration with robots without putting humans at harm. We show how an XR setting enables combining human behavioral data, subjective self-reports, and biosignals signifying human comfort, stress and cognitive load during collaboration. Several works demonstrate XR can be used to train human operators and provide them with augmented reality (AR) interfaces to enhance their performance with robots. We also provide a first attempt at what could become the basis for a human–robot collaboration testing framework, specifically for designing and testing factors affecting human–robot collaboration. The use of XR has the potential to change the way we design and test cobots, and train cobot operators, in a range of applications: from industry, through healthcare, to space operations.info:eu-repo/semantics/publishedVersio

Virtual reality retooling humanities courses: Finance and marketing experience at a Czech university

Virtual reality environments (VRE) allow users to visualize both real-life and imaginary activities. For this reason, they make appropriate training fields at universities, too. However, the positive or negative effects of VRE are still a subject of research. There is a need to verify methods of their deployment, student responses and the impact of VRE implementation. Science and medicine courses are frequently exploiting VRE, while their exploitation in humanities is much less frequent. In our paper, we describe and evaluate their application in finance and marketing courses. Both courses were designed and developed as part of a larger, potentially university-wide project. The courses were enriched by mazes including 3-D rooms with course content elements. Students could explore them and communicate with their lecturers and classmates. To allow anytime/anywhere access, the VRE does not require using any special interface. The finance course was organized as a pedagogical experiment with test and control groups. Due to organizational and scheduling reasons, the VRE in marketing served just as enrichment. At the end of the term, all students using VRE were given a questionnaire assessing their satisfaction. The majority expressed satisfaction. In the finance course, positive opinion was also supported by students' improved grades. In total, 87.5% of students agreed that the application of VRE contributed to gaining knowledge. Based on the positive experience and outcomes, the university plans to expand and to intensify its VRE-supported education.Web of Science1219art. no. 1017

Characterizing Computer Access Using a One-Channel EEG Wireless Sensor

This work studies the feasibility of using mental attention to access a computer. Brain activity was measured with an electrode placed at the Fp1 position and the reference on the left ear; seven normally developed people and three subjects with cerebral palsy (CP) took part in the experimentation. They were asked to keep their attention high and low for as long as possible during several trials. We recorded attention levels and power bands conveyed by the sensor, but only the first was used for feedback purposes. All of the information was statistically analyzed to find the most significant parameters and a classifier based on linear discriminant analysis (LDA) was also set up. In addition, 60% of the participants were potential users of this technology with an accuracy of over 70%. Including power bands in the classifier did not improve the accuracy in discriminating between the two attentional states. For most people, the best results were obtained by using only the attention indicator in classification. Tiredness was higher in the group with disabilities (2.7 in a scale of 3) than in the other (1.5 in the same scale); and modulating the attention to access a communication board requires that it does not contain many pictograms (between 4 and 7) on screen and has a scanning period of a relatively high tscan 10 s. The information transfer rate (ITR) is similar to the one obtained by other brain computer interfaces (BCI), like those based on sensorimotor rhythms (SMR) or slow cortical potentials (SCP), and makes it suitable as an eye-gaze independent BCI

Response time to a vibrotactile stimulus presented on the foot at rest and during walking on different surfaces

This study investigates the simple reaction time (SRT) and response time (RT) to a vibrotactile stimulus presented on two body locations at the lower extremity of the foot on different types of surface during walking. We determined RTs while walking on Concrete, Foam, Sand, and gravel surface. Also, for RT, we evaluated two vibrotactile stimulus (VS) locations on the lower extremity: the ankle (AL) and under the foot plantar (FP). A total of 21 young adult participants (n = 21), aged mean 24 ± 2.9 years, took part in a two-session experiment with two main conditions (at rest and while walking on four types of surface). The control session included 2016 repeated measures, with one-way and two-way ANOVA analyses. The findings have consistently revealed slowness of RT to VS, in particular on sand and gravel surface. In addition, we found that body location has a significant effect on RT in certain surfaces. These results showed that RTs increased with environment changes during the performance of dual tasks

Human-Robot Collaborations in Industrial Automation

Technology is changing the manufacturing world. For example, sensors are being used to track inventories from the manufacturing floor up to a retail shelf or a customer’s door. These types of interconnected systems have been called the fourth industrial revolution, also known as Industry 4.0, and are projected to lower manufacturing costs. As industry moves toward these integrated technologies and lower costs, engineers will need to connect these systems via the Internet of Things (IoT). These engineers will also need to design how these connected systems interact with humans. The focus of this Special Issue is the smart sensors used in these human–robot collaborations

Fine-grained Haptics: Sensing and Actuating Haptic Primary Colours (force, vibration, and temperature)

This thesis discusses the development of a multimodal, fine-grained visual-haptic system for teleoperation and robotic applications. This system is primarily composed of two complementary components: an input device known as the HaptiTemp sensor (combines “Haptics” and “Temperature”), which is a novel thermosensitive GelSight-like sensor, and an output device, an untethered multimodal finegrained haptic glove. The HaptiTemp sensor is a visuotactile sensor that can sense haptic primary colours known as force, vibration, and temperature. It has novel switchable UV markers that can be made visible using UV LEDs. The switchable markers feature is a real novelty of the HaptiTemp because it can be used in the analysis of tactile information from gel deformation without impairing the ability to classify or recognise images. The use of switchable markers in the HaptiTemp sensor is the solution to the trade-off between marker density and capturing high-resolution images using one sensor. The HaptiTemp sensor can measure vibrations by counting the number of blobs or pulses detected per unit time using a blob detection algorithm. For the first time, temperature detection was incorporated into a GelSight-like sensor, making the HaptiTemp sensor a haptic primary colours sensor. The HaptiTemp sensor can also do rapid temperature sensing with a 643 ms response time for the 31°C to 50°C temperature range. This fast temperature response of the HaptiTemp sensor is comparable to the withdrawal reflex response in humans. This is the first time a sensor can trigger a sensory impulse that can mimic a human reflex in the robotic community. The HaptiTemp sensor can also do simultaneous temperature sensing and image classification using a machine vision camera—the OpenMV Cam H7 Plus. This capability of simultaneous sensing and image classification has not been reported or demonstrated by any tactile sensor. The HaptiTemp sensor can be used in teleoperation because it can communicate or transmit tactile analysis and image classification results using wireless communication. The HaptiTemp sensor is the closest thing to the human skin in tactile sensing, tactile pattern recognition, and rapid temperature response. In order to feel what the HaptiTemp sensor is touching from a distance, a corresponding output device, an untethered multimodal haptic hand wearable, is developed to actuate the haptic primary colours sensed by the HaptiTemp sensor. This wearable can communicate wirelessly and has fine-grained cutaneous feedback to feel the edges or surfaces of the tactile images captured by the HaptiTemp sensor. This untethered multimodal haptic hand wearable has gradient kinesthetic force feedback that can restrict finger movements based on the force estimated by the HaptiTemp sensor. A retractable string from an ID badge holder equipped with miniservos that control the stiffness of the wire is attached to each fingertip to restrict finger movements. Vibrations detected by the HaptiTemp sensor can be actuated by the tapping motion of the tactile pins or by a buzzing minivibration motor. There is also a tiny annular Peltier device, or ThermoElectric Generator (TEG), with a mini-vibration motor, forming thermo-vibro feedback in the palm area that can be activated by a ‘hot’ or ‘cold’ signal from the HaptiTemp sensor. The haptic primary colours can also be embedded in a VR environment that can be actuated by the multimodal hand wearable. A VR application was developed to demonstrate rapid tactile actuation of edges, allowing the user to feel the contours of virtual objects. Collision detection scripts were embedded to activate the corresponding actuator in the multimodal haptic hand wearable whenever the tactile matrix simulator or hand avatar in VR collides with a virtual object. The TEG also gets warm or cold depending on the virtual object the participant has touched. Tests were conducted to explore virtual objects in 2D and 3D environments using Leap Motion control and a VR headset (Oculus Quest 2). Moreover, a fine-grained cutaneous feedback was developed to feel the edges or surfaces of a tactile image, such as the tactile images captured by the HaptiTemp sensor, or actuate tactile patterns in 2D or 3D virtual objects. The prototype is like an exoskeleton glove with 16 tactile actuators (tactors) on each fingertip, 80 tactile pins in total, made from commercially available P20 Braille cells. Each tactor can be controlled individually to enable the user to feel the edges or surfaces of images, such as the high-resolution tactile images captured by the HaptiTemp sensor. This hand wearable can be used to enhance the immersive experience in a virtual reality environment. The tactors can be actuated in a tapping manner, creating a distinct form of vibration feedback as compared to the buzzing vibration produced by a mini-vibration motor. The tactile pin height can also be varied, creating a gradient of pressure on the fingertip. Finally, the integration of the high-resolution HaptiTemp sensor, and the untethered multimodal, fine-grained haptic hand wearable is presented, forming a visuotactile system for sensing and actuating haptic primary colours. Force, vibration, and temperature sensing tests with corresponding force, vibration, and temperature actuating tests have demonstrated a unified visual-haptic system. Aside from sensing and actuating haptic primary colours, touching the edges or surfaces of the tactile images captured by the HaptiTemp sensor was carried out using the fine-grained cutaneous feedback of the haptic hand wearable

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility

Study of an onboard wired-wireless health monitoring system equipped with power save algorithm for freight railway wagons

Goods transport is an essential factor for the European market for its significant contribution to economic growth and thus to the creation of new employment. Nowadays, approximately 75% of goods are transported by road within the European Union. The use of more efficient and sustainable modes of transportation, such as rail transport and inland waterways, would reduce oil imports and pollution abatement. The growth of rail goods transport must be accompanied by an increasing introduction of tools and technologies that make possible to constantly monitor the European rolling stock. The introduction of monitoring technologies that allow to constantly know the status of the wagon would bring real and concrete benefits to the world of rail transport enabling to optimize the maintenance of rolling stock thus reducing costs but ensuring at the same time a maximization of the safety. Currently, the only information available are provided by the equipment installed along the railway network, separated by tens of kilometers. However, to identify and intervene on an incipient failure, it is necessary to have continuous monitoring and a communication system that can warn the train conductor and the maintenance staff of wagon’s owners. A good monitoring system has to be: cheap, energy autonomous, wireless and reliable. Currently monitoring systems can be divided into two large groups. The former are those developed by universities or research centres within projects financed by third parties, while the latter are monitoring systems developed individually by companies operating in the logistics sector. In light of the existing research projects and products already available on the market, the following thesis work aims to develop a monitoring system demonstrator dedicated to freight wagons that can demonstrate the effectiveness of these devices. The results of preliminary literature and market analyses served as the base for the realization of a first wired demonstrator. All the subsystems of the first demonstrator were long tested in laboratory in order to guarantee the maximum reliability of the device and maximum repeatability of the recorded data. The parameters monitored were the pressures of the pneumatic braking system, the temperature of the cast-iron brake blocks and the dynamics of the body frame. The second demonstrator developed was significantly more complex. In fact, it consists of two wireless units: a base station which represents the further development of the first demonstrator and a completely new axle box node monitoring system. From the analysis of the brake block temperature data two fundamental aspects emerge. The first is the need and importance of maintaining the braking system always in good conditions, doing maintenance in line with the regulations. The second is related to the adoption of new brake blocks in synthetic material. In fact, in addition to the complete review of the brake system as prescribed by the regulation, also the material of the wheelsets must be suitable for the use of new type of brake blocks. Another aspect subject to monitoring in this work is the vibration monitoring. Vibrations of particular interest for freight wagon monitoring are those along the vertical axis and the longitudinal axis. The accelerations along the vertical axis in fact describe the stability of the vehicle and its interaction with the rails. Vertical acceleration is a parameter that allows to determine if the wagon is traveling safely or not. In fact, this parameter makes it possible to identify a possible derailment, if the acceleration level recorded is anomalous. The longitudinal acceleration is a parameter monitored by all the railway monitoring devices present on the market. It is important to know the longitudinal accelerometric levels both in the phases of train composition and during the braking operations in order to identify possible incorrect behaviour. The second demonstrator allowed to monitor the external temperature of the axle box cover and verified the correct behaviour of bearings. The most important result of the second demonstrator was the creation of a wireless network that makes it possible to monitor any quantities without invasive wiring. The creation of a wireless network has also required the development of power saving algorithms for the reduction of energy consumption in order to obtain the maximum operating time. In both prototypes developed, the monitored parameters were very numerous and were sampled with a very high frequency, especially those related to temperatures and pressures. This is a typical feature of the demonstrators. Instead, in order to monitor and study the phenomena related to the dynamics of the wagon it is necessary a sampling frequency as the one adopted. The developed prototypes, even if marked by a strong manual activity, have shown a very high reliability. Monitoring all these parameters for such a long distance led to the creation of a large database. Generally, only large industrial groups can boast such prolonged tests. The prototypes made, thanks to their hardware and software effectiveness, were the basis for the most complex monitoring system that we have set ourselves to achieve with the SWAM Rail project. In conclusion, the project carried out in these three years has therefore obtained as results the realization of demonstrators of monitoring devices, the collection of data that would allow to understand and study the operation of a wagon in optimal maintenance conditions, the development of thermal models and the identification of threshold parameters for delimiting conditions of normal operation by fault conditions

Haptics: Science, Technology, Applications

This open access book constitutes the proceedings of the 12th International Conference on Human Haptic Sensing and Touch Enabled Computer Applications, EuroHaptics 2020, held in Leiden, The Netherlands, in September 2020. The 60 papers presented in this volume were carefully reviewed and selected from 111 submissions. The were organized in topical sections on haptic science, haptic technology, and haptic applications. This year's focus is on accessibility

Proceedings of the 9th Arab Society for Computer Aided Architectural Design (ASCAAD) international conference 2021 (ASCAAD 2021): architecture in the age of disruptive technologies: transformation and challenges.

The ASCAAD 2021 conference theme is Architecture in the age of disruptive technologies: transformation and challenges. The theme addresses the gradual shift in computational design from prototypical morphogenetic-centered associations in the architectural discourse. This imminent shift of focus is increasingly stirring a debate in the architectural community and is provoking a much needed critical questioning of the role of computation in architecture as a sole embodiment and enactment of technical dimensions, into one that rather deliberately pursues and embraces the humanities as an ultimate aspiration