Mulsemedia Communication Research Challenges for Metaverse in 6G Wireless Systems

Although humans have five basic senses, sight, hearing, touch, smell, and taste, most multimedia systems in current systems only capture two of them, namely, sight and hearing. With the development of the metaverse and related technologies, there is a growing need for a more immersive media format that leverages all human senses. Multisensory media(Mulsemedia) that can stimulate multiple senses will play a critical role in the near future. This paper provides an overview of the history, background, use cases, existing research, devices, and standards of mulsemedia. Emerging mulsemedia technologies such as Extended Reality (XR) and Holographic-Type Communication (HTC) are introduced. Additionally, the challenges in mulsemedia research from the perspective of wireless communication and networking are discussed. The potential of 6G wireless systems to address these challenges is highlighted, and several research directions that can advance mulsemedia communications are identified

From plain visualisation to vibration sensing: using a camera to control the flexibilities in the ITER remote handling equipment

Thermonuclear fusion is expected to play a key role in the energy market during the second half of this century, reaching 20% of the electricity generation by 2100. For many years, fusion scientists and engineers have been developing the various technologies required to build nuclear power stations allowing a sustained fusion reaction. To the maximum possible extent, maintenance operations in fusion reactors are performed manually by qualified workers in full accordance with the "as low as reasonably achievable" (ALARA) principle. However, the option of hands-on maintenance becomes impractical, difficult or simply impossible in many circumstances, such as high biological dose rates. In this case, maintenance tasks will be performed with remote handling (RH) techniques. The International Thermonuclear Experimental Reactor ITER, to be commissioned in southern France around 2025, will be the first fusion experiment producing more power from fusion than energy necessary to heat the plasma. Its main objective is “to demonstrate the scientific and technological feasibility of fusion power for peaceful purposes”. However ITER represents an unequalled challenge in terms of RH system design, since it will be much more demanding and complex than any other remote maintenance system previously designed. The introduction of man-in-the-loop capabilities in the robotic systems designed for ITER maintenance would provide useful assistance during inspection, i.e. by providing the operator the ability and flexibility to locate and examine unplanned targets, or during handling operations, i.e. by making peg-in-hole tasks easier. Unfortunately, most transmission technologies able to withstand the very specific and extreme environmental conditions existing inside a fusion reactor are based on gears, screws, cables and chains, which make the whole system very flexible and subject to vibrations. This effect is further increased as structural parts of the maintenance equipment are generally lightweight and slender structures due to the size and the arduous accessibility to the reactor. Several methodologies aiming at avoiding or limiting the effects of vibrations on RH system performance have been investigated over the past decade. These methods often rely on the use of vibration sensors such as accelerometers. However, reviewing market shows that there is no commercial off-the-shelf (COTS) accelerometer that meets the very specific requirements for vibration sensing in the ITER in-vessel RH equipment (resilience to high total integrated dose, high sensitivity). The customisation and qualification of existing products or investigation of new concepts might be considered. However, these options would inevitably involve high development costs. While an extensive amount of work has been published on the modelling and control of flexible manipulators in the 1980s and 1990s, the possibility to use vision devices to stabilise an oscillating robotic arm has only been considered very recently and this promising solution has not been discussed at length. In parallel, recent developments on machine vision systems in nuclear environment have been very encouraging. Although they do not deal directly with vibration sensing, they open up new prospects in the use of radiation tolerant cameras. This thesis aims to demonstrate that vibration control of remote maintenance equipment operating in harsh environments such as ITER can be achieved without considering any extra sensor besides the embarked rad-hardened cameras that will inevitably be used to provide real-time visual feedback to the operators. In other words it is proposed to consider the radiation-tolerant vision devices as full sensors providing quantitative data that can be processed by the control scheme and not only as plain video feedback providing qualitative information. The work conducted within the present thesis has confirmed that methods based on the tracking of visual features from an unknown environment are effective candidates for the real-time control of vibrations. Oscillations induced at the end effector are estimated by exploiting a simple physical model of the manipulator. Using a camera mounted in an eye-in-hand configuration, this model is adjusted using direct measurement of the tip oscillations with respect to the static environment. The primary contribution of this thesis consists of implementing a markerless tracker to determine the velocity of a tip-mounted camera in an untrimmed environment in order to stabilise an oscillating long-reach robotic arm. In particular, this method implies modifying an existing online interaction matrix estimator to make it self-adjustable and deriving a multimode dynamic model of a flexible rotating beam. An innovative vision-based method using sinusoidal regression to sense low-frequency oscillations is also proposed and tested. Finally, the problem of online estimation of the image capture delay for visual servoing applications with high dynamics is addressed and an original approach based on the concept of cross-correlation is presented and experimentally validated

Autonomous landing of fixed-wing aircraft on mobile platforms

E n esta tesis se propone un nuevo sistema que permite la operación de aeronaves autónomas sin tren de aterrizaje. El trabajo está motivado por el interés industrial en aeronaves con la capacidad de volar a gran altitud, con más capacidad de carga útil y capaces de aterrizar con viento cruzado. El enfoque seguido en este trabajo consiste en eliminar el sistema de aterrizaje de una aeronave de ala fija empleando una plataforma móvil de aterrizaje en tierra. La aeronave y la plataforma deben sincronizar su movimiento antes del aterrizaje, lo que se logra mediante la estimación del estado relativo entre ambas y el control cooperativo del movimiento. El objetivo principal de esta Tesis es el desarrollo de una solución práctica para el aterrizaje autónomo de una aeronave de ala fija en una plataforma móvil. En la tesis se combinan nuevos métodos con experimentos prácticos para los cuales se ha desarrollado un sistema de pruebas específico. Se desarrollan dos variantes diferentes del sistema de aterrizaje. El primero presta atención especial a la seguridad, es robusto ante retrasos en la comunicación entre vehículos y cumple procedimientos habituales de aterrizaje, al tiempo que reduce la complejidad del sistema. En el segundo se utilizan trayectorias optimizadas del vehículo y sincronización bilateral de posición para maximizar el rendimiento del aterrizaje en términos de requerimientos de longitud necesaria de pista, pero la estabilidad es dependiente del retraso de tiempo, con lo cual es necesario desarrollar un controlador estabilizador ampliado, basado en pasividad, que permite resolver este problema. Ambas estrategias imponen requisitos funcionales a los controladores de cada uno de los vehículos, lo que implica la capacidad de controlar el movimiento longitudinal sin afectar el control lateral o vertical, y viceversa. El control de vuelo basado en energía se utiliza para proporcionar dicha funcionalidad a la aeronave. Los sistemas de aterrizaje desarrollados se han analizado en simulación estableciéndose los límites de rendimiento mediante múltiples repeticiones aleatorias. Se llegó a la conclusión de que el controlador basado en seguridad proporciona un rendimiento de aterrizaje satisfactorio al tiempo que suministra una mayor seguridad operativa y un menor esfuerzo de implementación y certificación. El controlador basado en el rendimiento es prometedor para aplicaciones con una longitud de pista limitada. Se descubrió que los beneficios del controlador basado en el rendimiento son menos pronunciados para una dinámica de vehículos terrestres más lenta. Teniendo en cuenta la dinámica lenta de la configuración del demostrador, se eligió el enfoque basado en la seguridad para los primeros experimentos de aterrizaje. El sistema de aterrizaje se validó en diversas pruebas de aterrizaje exitosas, que, a juicio del autor, son las primeras en el mundo realizadas con aeronaves reales. En última instancia, el concepto propuesto ofrece importantes beneficios y constituye una estrategia prometedora para futuras soluciones de aterrizaje de aeronaves.In this thesis a new landing system is proposed, which allows for the operation of autonomous aircraft without landing gear. The work was motivated by the industrial need for more capable high altitude aircraft systems, which typically suffer from low payload capacity and high crosswind landing sensitivity. The approach followed in this work consists in removing the landing gear system from the aircraft and introducing a mobile ground-based landing platform. The vehicles must synchronize their motion prior to landing, which is achieved through relative state estimation and cooperative motion control. The development of a practical solution for the autonomous landing of an aircraft on a moving platform thus constitutes the main goal of this thesis. Therefore, theoretical investigations are combined with real experiments for which a special setup is developed and implemented. Two different landing system variants are developed — the safety-based landing system is robust to inter-vehicle communication delays and adheres to established landing procedures, while reducing system complexity. The performance-based landing system uses optimized vehicle trajectories and bilateral position synchronization to maximize landing performance in terms of used runway, but suffers from time delay-dependent stability. An extended passivity-based stabilizing controller was implemented to cope with this issue. Both strategies impose functional requirements on the individual vehicle controllers, which imply independent controllability of the translational degrees of freedom. Energy-based flight control is utilized to provide such functionality for the aircraft. The developed landing systems are analyzed in simulation and performance bounds are determined by means of repeated random sampling. The safety-based controller was found to provide satisfactory landing performance while providing higher operational safety, and lower implementation and certification effort. The performance-based controller is promising for applications with limited runway length. The performance benefits were found to be less pronounced for slower ground vehicle dynamics. Given the slow dynamics of the demonstrator setup, the safety-based approach was chosen for first landing experiments. The landing system was validated in a number of successful landing trials, which to the author’s best knowledge was the first time such technology was demonstrated on the given scale, worldwide. Ultimately, the proposed concept offers decisive benefits and constitutes a promising strategy for future aircraft landing solutions

Intelligent and Efficient Transport Systems

The aim of this book is to present a number of digital and technology solutions to real-world problems across transportation sectors and infrastructures. Nine chapters have been well prepared and organized with the core topics as follows: -A guideline to evaluate the energy efficiency of a vehicle -A guideline to design and evaluate an electric propulsion system -Potential opportunities for intelligent transportation systems and smart cities -The importance of system control and energy-power management in transportation systems and infrastructures -Bespoke modeling tools and real-time simulation platforms for transportation system development This book will be useful to a wide range of audiences: university staff and students, engineers, and business people working in relevant fields

Modeling Off-the-Shelf Pan/Tilt Cameras for Active Vision Systems

There are many existing multicamera systems that perform object identification and track ing. Some applications include but are not limited to security surveillance and smart rooms. Yet there is still much work to be done in improving such systems to achieve a high level of automation while obtaining reasonable performance. Thus far design and implementation of these systems has been done using heuristic methods, primarily due to the complexity of the problem. Most importantiy, the performance of these systems is assessed by evaluating subjective quantities. The goal of this work is to take the first step in structured analysis and design of multicamera systems, that is, to introduce a model of a single camera with asso ciated image processing algorithms capable of tracking a target. A single camera model is developed such that it could be easily used as a building block for a multicamera system