A continuum robotic platform for endoscopic non-contact laser surgery: design, control, and preclinical evaluation

The application of laser technologies in surgical interventions has been accepted in the clinical domain due to their atraumatic properties. In addition to manual application of fibre-guided lasers with tissue contact, non-contact transoral laser microsurgery (TLM) of laryngeal tumours has been prevailed in ENT surgery. However, TLM requires many years of surgical training for tumour resection in order to preserve the function of adjacent organs and thus preserve the patient’s quality of life. The positioning of the microscopic laser applicator outside the patient can also impede a direct line-of-sight to the target area due to anatomical variability and limit the working space. Further clinical challenges include positioning the laser focus on the tissue surface, imaging, planning and performing laser ablation, and motion of the target area during surgery. This dissertation aims to address the limitations of TLM through robotic approaches and intraoperative assistance. Although a trend towards minimally invasive surgery is apparent, no highly integrated platform for endoscopic delivery of focused laser radiation is available to date. Likewise, there are no known devices that incorporate scene information from endoscopic imaging into ablation planning and execution. For focusing of the laser beam close to the target tissue, this work first presents miniaturised focusing optics that can be integrated into endoscopic systems. Experimental trials characterise the optical properties and the ablation performance. A robotic platform is realised for manipulation of the focusing optics. This is based on a variable-length continuum manipulator. The latter enables movements of the endoscopic end effector in five degrees of freedom with a mechatronic actuation unit. The kinematic modelling and control of the robot are integrated into a modular framework that is evaluated experimentally. The manipulation of focused laser radiation also requires precise adjustment of the focal position on the tissue. For this purpose, visual, haptic and visual-haptic assistance functions are presented. These support the operator during teleoperation to set an optimal working distance. Advantages of visual-haptic assistance are demonstrated in a user study. The system performance and usability of the overall robotic system are assessed in an additional user study. Analogous to a clinical scenario, the subjects follow predefined target patterns with a laser spot. The mean positioning accuracy of the spot is 0.5 mm. Finally, methods of image-guided robot control are introduced to automate laser ablation. Experiments confirm a positive effect of proposed automation concepts on non-contact laser surgery.Die Anwendung von Lasertechnologien in chirurgischen Interventionen hat sich aufgrund der atraumatischen Eigenschaften in der Klinik etabliert. Neben manueller Applikation von fasergeführten Lasern mit Gewebekontakt hat sich die kontaktfreie transorale Lasermikrochirurgie (TLM) von Tumoren des Larynx in der HNO-Chirurgie durchgesetzt. Die TLM erfordert zur Tumorresektion jedoch ein langjähriges chirurgisches Training, um die Funktion der angrenzenden Organe zu sichern und damit die Lebensqualität der Patienten zu erhalten. Die Positionierung des mikroskopis chen Laserapplikators außerhalb des Patienten kann zudem die direkte Sicht auf das Zielgebiet durch anatomische Variabilität erschweren und den Arbeitsraum einschränken. Weitere klinische Herausforderungen betreffen die Positionierung des Laserfokus auf der Gewebeoberfläche, die Bildgebung, die Planung und Ausführung der Laserablation sowie intraoperative Bewegungen des Zielgebietes. Die vorliegende Dissertation zielt darauf ab, die Limitierungen der TLM durch robotische Ansätze und intraoperative Assistenz zu adressieren. Obwohl ein Trend zur minimal invasiven Chirurgie besteht, sind bislang keine hochintegrierten Plattformen für die endoskopische Applikation fokussierter Laserstrahlung verfügbar. Ebenfalls sind keine Systeme bekannt, die Szeneninformationen aus der endoskopischen Bildgebung in die Ablationsplanung und -ausführung einbeziehen. Für eine situsnahe Fokussierung des Laserstrahls wird in dieser Arbeit zunächst eine miniaturisierte Fokussieroptik zur Integration in endoskopische Systeme vorgestellt. Experimentelle Versuche charakterisieren die optischen Eigenschaften und das Ablationsverhalten. Zur Manipulation der Fokussieroptik wird eine robotische Plattform realisiert. Diese basiert auf einem längenveränderlichen Kontinuumsmanipulator. Letzterer ermöglicht in Kombination mit einer mechatronischen Aktuierungseinheit Bewegungen des Endoskopkopfes in fünf Freiheitsgraden. Die kinematische Modellierung und Regelung des Systems werden in ein modulares Framework eingebunden und evaluiert. Die Manipulation fokussierter Laserstrahlung erfordert zudem eine präzise Anpassung der Fokuslage auf das Gewebe. Dafür werden visuelle, haptische und visuell haptische Assistenzfunktionen eingeführt. Diese unterstützen den Anwender bei Teleoperation zur Einstellung eines optimalen Arbeitsabstandes. In einer Anwenderstudie werden Vorteile der visuell-haptischen Assistenz nachgewiesen. Die Systemperformanz und Gebrauchstauglichkeit des robotischen Gesamtsystems werden in einer weiteren Anwenderstudie untersucht. Analog zu einem klinischen Einsatz verfolgen die Probanden mit einem Laserspot vorgegebene Sollpfade. Die mittlere Positioniergenauigkeit des Spots beträgt dabei 0,5 mm. Zur Automatisierung der Ablation werden abschließend Methoden der bildgestützten Regelung vorgestellt. Experimente bestätigen einen positiven Effekt der Automationskonzepte für die kontaktfreie Laserchirurgie

An approach to open virtual commissioning for component-based automation

Increasing market demands for highly customised products with shorter time-to-market and at lower prices are forcing manufacturing systems to be built and operated in a more efficient ways. In order to overcome some of the limitations in traditional methods of automation system engineering, this thesis focuses on the creation of a new approach to Virtual Commissioning (VC). In current VC approaches, virtual models are driven by pre-programmed PLC control software. These approaches are still time-consuming and heavily control expertise-reliant as the required programming and debugging activities are mainly performed by control engineers. Another current limitation is that virtual models validated during VC are difficult to reuse due to a lack of tool-independent data models. Therefore, in order to maximise the potential of VC, there is a need for new VC approaches and tools to address these limitations. The main contributions of this research are: (1) to develop a new approach and the related engineering tool functionality for directly deploying PLC control software based on component-based VC models and reusable components; and (2) to build tool-independent common data models for describing component-based virtual automation systems in order to enable data reusability. [Continues.

On the Mitigation of Late Stage Redesign in Mechatronics Using Integrated Approaches

RÉSUMÉ Les systèmes mécatronique combinent des éléments issus du génie mécanique, électrique, contrôle et logiciel. Due à la nature multi-domaine de ces systèmes, il est nécessaire de s’assurer d’un processus de conception optimal afin de réduire le temps et le cout de développement. De ce fait, cette thèse s’intéresse aux boucles de re-conception tard durant le processus de développement. Ces boucles peuvent être causé entre autres par des interactions négatives qui affectent la performance et l’intégration des composantes et sous-systèmes et l’incertitude dans les paramètres du système. Premièrement, cette thèse propose une nouvelle méthode de modélisation qui permet d’identifier et d’évaluer les dépendances durant les phases initiales de conception. Cette méthode est ensuite utilisée dans la création d’un index qui permet de représenter le niveau total de dépendances négative du système. L’index est ensuite utilisé dans l’évaluation multicritère, ce qui permet de choisir des systèmes étant plus faciles à concevoir. Finalement, une méthode de modélisation qui permet de considérer de façon concurrente les dépendance positive et négative est présenté. Par la suite, cette thèse propose d’utiliser les nombres flous afin de traiter l’incertitude des paramètres. En premier lieu, la thèse montre que les nombres flous peuvent être utilisé afin de simuler le comportement d’un système mécatronique sujet à de l’incertitude. De plus, une méthode de conception utilisant la simulation floue est proposée afin de concevoir les systèmes mécatronique de façon robuste. De plus, les nombres floues permettent de déterminer la stabilité du système, ce qui permet le développement d’une méthodologie de conception robuste totalement intégré, qui considère à la fois l’aspect physique et contrôle du système.----------ABSTRACT Mechatronic systems are highly integrated devices, with elements from mechanical, electrical, software and control engineering. It is thus necessary to ensure a streamlined design process to reduce development time and cost. Consequently, this thesis researches on the issue of late stages redesigns in mechatronics. The late stages redesigns may occur due to problems while integrating the different components and subsystems. Two causes of these redesigns are unpredicted negative interactions between the elements of the system, and inadequate performance due to uncertainties. To deal with the issue of negative interactions, this thesis first suggests a modeling method that enables to identify and assess negative dependencies early during the design process. It is shown that the modeling method can be efficiently used to detect dependencies that would be detrimental to the system’s performance and which may require more design effort. Then, based on this modeling method, an index representing the total level of negative dependencies present within the system is proposed. The index is shown to be able to predict decrease of performance due to the negative dependencies and can thus be used as a valuable criterion during decision making. Finally, a modeling method to handle concurrently positive and negative dependencies is suggested. This modeling method is shown to have an impact on the currently existing complexity metrics and should thus allow to better represent the reality of the design. Furthermore, to deal with the issue of uncertainties affecting the performance of the system, this thesis proposes a design methodology using fuzzy numbers. First, it is shown that fuzzy numbers can be used to model and simulate the uncertain behavior of mechatronic systems while being computationally efficient. Then a robust design methodology is presented and shown to be effective in optimizing a mechatronic system while reducing the uncertainties in the performance. Furthermore, based on the use of fuzzy numbers in the modeling of the mechatronic system, it is shown that it is possible to determine the stability of the device under uncertainties. Finally, a fully integrated robust design methodology is presented, which consider both control and design parameters selection, and which can be used to mitigate late stages redesigns due to improper performance. In sum, this thesis investigates and suggests multiple integrated design solutions to mitigate late stages redesigns in the mechatronic design process

The Internet of Things: the future or the end of mechatronics.

The advent and increasing implementation of user configured and user oriented systems structured around the use of cloud configured information and the Internet of Things is presenting a new range and class of challenges to the underlying concepts of integration and transfer of functionality around which mechatronics is structured. It is suggested that the ways in which system designers and educators in particular respond to and manage these changes and challenges is going to have a significant impact on the way in which both the Internet of Things and mechatronics develop over time. The paper places the relationship between the Internet of Things and mechatronics into perspective and considers the issues and challenges facing systems designers and implementers in relation to managing the dynamics of the changes required

A Smart Products Lifecycle Management (sPLM) Framework - Modeling for Conceptualization, Interoperability, and Modularity

Autonomy and intelligence have been built into many of today’s mechatronic products, taking advantage of low-cost sensors and advanced data analytics technologies. Design of product intelligence (enabled by analytics capabilities) is no longer a trivial or additional option for the product development. The objective of this research is aimed at addressing the challenges raised by the new data-driven design paradigm for smart products development, in which the product itself and the smartness require to be carefully co-constructed. A smart product can be seen as specific compositions and configurations of its physical components to form the body, its analytics models to implement the intelligence, evolving along its lifecycle stages. Based on this view, the contribution of this research is to expand the “Product Lifecycle Management (PLM)” concept traditionally for physical products to data-based products. As a result, a Smart Products Lifecycle Management (sPLM) framework is conceptualized based on a high-dimensional Smart Product Hypercube (sPH) representation and decomposition. First, the sPLM addresses the interoperability issues by developing a Smart Component data model to uniformly represent and compose physical component models created by engineers and analytics models created by data scientists. Second, the sPLM implements an NPD3 process model that incorporates formal data analytics process into the new product development (NPD) process model, in order to support the transdisciplinary information flows and team interactions between engineers and data scientists. Third, the sPLM addresses the issues related to product definition, modular design, product configuration, and lifecycle management of analytics models, by adapting the theoretical frameworks and methods for traditional product design and development. An sPLM proof-of-concept platform had been implemented for validation of the concepts and methodologies developed throughout the research work. The sPLM platform provides a shared data repository to manage the product-, process-, and configuration-related knowledge for smart products development. It also provides a collaborative environment to facilitate transdisciplinary collaboration between product engineers and data scientists

ThinkingSkins:

New technologies and automation concepts emerge in the digitalization of our environment. This is, for example, reflected by intelligent production systems in Industry 4.0. A core aspect of such systems is their cyber-physical implementation, which aims to increase productivity and flexibility through embedded computing capacities and the cooperation of decentrally networked production plants. This development stage of automation has not yet been achieved in the current state-of-the-art of façades. Being responsible for the execution of adaptive measures, façade automation is part of hierarchically and centrally organised Building Automation Systems (BAS). The research project ThinkingSkins is guided by the hypothesis that, aiming at an enhanced overall building performance, façades can be implemented as cyber-physical systems. Accordingly, it addresses the research question: How can cyber-physical systems be applied to façades, in order to enable coordinated adaptations of networked individual façade functions? The question is approached in four partial investigations. First, a comprehensive understanding of intelligent systems in both application fields, façades and Industry 4.0, is elaborated by a literature review. Subsequently, relevant façade functions are identified by a second literature review in a superposition matrix, which also incorporates characteristics for a detailed assessment of each function’s adaptive capacities. The third investigation focuses on existing conditions in building practice by means of a multiple case study analysis. Finally, the technical feasibility of façades implemented as cyber-physical systems is investigated by developing a prototype. The research project identifies the possibility and promising potential of cyberphysical façades. As result, the doctoral dissertation provides a conceptual framework for the implementation of such systems in building practice and for further research

Developing Methods of Obtaining Quality Failure Information from Complex Systems

The complexity in most engineering systems is constantly growing due to ever-increasing technological advancements. This result in a corresponding need for methods that adequately account for the reliability of such systems based on failure information from components that make up these systems. This dissertation presents an approach to validating qualitative function failure results from model abstraction details. The impact of the level of detail available to a system designer during conceptual stages of design is considered for failure space exploration in a complex system. Specifically, the study develops an efficient approach towards detailed function and behavior modeling required for complex system analyses. In addition, a comprehensive research and documentation of existing function failure analysis methodologies is also synthesized into identified structural groupings. Using simulations, known governing equations are evaluated for components and system models to study responses to faults by accounting for detailed failure scenarios, component behaviors, fault propagation paths, and overall system performance. The components were simulated at nominal states and varying degrees of fault representing actual modes of operation. Information on product design and provisions on expected working conditions of components were used in the simulations to address normally overlooked areas during installation. The results of system model simulations were investigated using clustering analysis to develop an efficient grouping method and measure of confidence for the obtained results. The intellectual merit of this work is the use of a simulation based approach in studying how generated failure scenarios reveal component fault interactions leading to a better understanding of fault propagation within design models. The information from using varying fidelity models for system analysis help in identifying models that are sufficient enough at the conceptual design stages to highlight potential faults. This will reduce resources such as cost, manpower and time spent during system design. A broader impact of the project is to help design engineers identifying critical components, quantifying risks associated with using particular components in their prototypes early in the design process and help improving fault tolerant system designs. This research looks to eventually establishing a baseline for validating and comparing theories of complex systems analysis

Thinking- Skins

Under the guiding concept of a thinking skin, the research project examines the transferability of cyber-physical systems to the application field of façades. It thereby opens up potential increases in the performance of automated and adaptive façade systems and provides a conceptual framework for further research and development of intelligent building envelopes in the current age of digital transformation. The project is characterized by the influence of digital architectural design methods and the associated computational processing of information in the design process. The possible establishment of relationships and dependencies in an architecture understood as a system, in particular, are the starting point for the conducted investigation. With the available automation technologies, the possibility of movable building constructions, and existing computer-based control systems, the technical preconditions for the realisation of complex and active buildings exist today. Against this background, dynamic and responsive constructions that allow adaptations in the operation of the building are a current topic in architecture. In the application field of the building envelope, the need for such designs is evident, particularly with regards to the concrete field of adaptive façades. In its mediating role, the façade is confronted with the dynamic influences of the external microclimate of a building and the changing comfort demands of the indoor climate. The objective in the application of adaptive façades is to increase building efficiency by balancing dynamic influencing factors and requirements. Façade features are diverse and with the increasing integration of building services, both the scope of fulfilled façade functions and the complexity of today’s façades increase. One challenge is the coordination of adaptive functions to ensure effective reactions of the façade as a complete system. The ThinkingSkins research project identifies cyber-physical systems as a possible solution to this challenge. This involves the close integration of physical systems with their digital control. Important features are the decentralized organization of individual system constituents and their cooperation via an exchange of information. Developments in recent decades, such as the miniaturisation of computer technology and the availability of the Internet, have established the technical basis required for these developments. Cyber-physical systems are already employed in many fields of application. Examples are decentralized energy supply, or transportation systems with autonomous vehicles. The influence is particularly evident in the transformation of the industrial sector to Industry 4.0, where formerly mechatronic production plants are networked into intelligent technical systems with the aim of achieving higher and more flexible productivity. In the ThinkingSkins research project it is assumed that the implementation of cyber-physical systems based on the role model of cooperating production plants in IIndustry 4.0 can contribute to an increase in the performance of façades. Accordingly, the research work investigates a possible transfer of cyber-physical systems to the application field of building envelopes along the research question: How can cyber-physical systems be applied to façades, in order to enable coordinated adaptations of networked individual façade functions? To answer this question, four partial studies are carried out, which build upon each other. The first study is based on a literature review, in which the understanding and the state-of-the-art development of intelligent façade systems is examined in comparison to the exemplary field of application of cyber-physical systems in the manufacturing industry. In the following partial study, a second literature search identifies façade functions that can be considered as components of a cyber-physical façade due to their adaptive feasibility and their effect on the façade performance. For the evaluation of the adaptive capabilities, characteristics of their automated and adaptive implementation are assigned to the identified façade functions. The resulting superposition matrix serves as an organizational tool for the third investigation of the actual conditions in construction practice. In a multiple case study, realized façade projects in Germany are examined with regard to their degree of automation and adaptivity. The investigation includes interviews with experts involved in the projects as well as field studies on site. Finally, an experimental examination of the technical feasibility of cyber-physical façade systems is carried out through the development of a prototype. In the sense of an internet of façade functions, the automated adaptive façade functions ventilation, sun protection as well as heating and cooling are implemented in decentrally organized modules. They are connected to a digital twin and can exchange data with each other via a communication protocol. The research project shows that the application field of façades has not yet been exploited for the implementation of cyber-physical systems. With the automation technologies used in building practice, however, many technical preconditions for the development of cyber-physical façade systems already exist. Many features of such a system are successfully implemented within the study by the development of a prototype. The research project therefore comes to the conclusion that the application of cyber-physical systems to the façade is possible and offers a promising potential for the effective use of automation technologies. Due to the lack of artificial intelligence and machine learning strategies, the project does not achieve the goal of developing a façade in the sense of a true ThinkingSkin as the title indicates. A milestone is achieved by the close integration of the physical façade system with a decentralized and integrated control system. In this sense, the researched cyber-physical implementation of façades represents a conceptual framework for the realisation of corresponding systems in building practice, and a pioneer for further research of ThinkingSkins

SOFTENG 2023: the ninth international conference on advances and trends in software engineering

The Ninth International Conference on Advances and Trends in Software Engineering (SOFTENG 2023), held between April 24th and April 28th, 2023, continued a series of events focusing on these challenging aspects for software development and deployment, across the whole life-cycle. Software engineering exhibits challenging dimensions in the light of new applications, devices, and services. Mobility, user-centric development, smart-devices, e-services, ambient environments, e-health and wearable/implantable devices pose specific challenges for specifying software requirements and developing reliable and safe software. Specific software interfaces, agile organization and software dependability require particular approaches for software security, maintainability, and sustainability. We take here the opportunity to warmly thank all the members of the SOFTENG 2023 technical program committee, as well as all the reviewers. The creation of such a high-quality conference program would not have been possible without their involvement. We also kindly thank all the authors who dedicated much of their time and effort to contribute to SOFTENG 2023. We truly believe that, thanks to all these efforts, the final conference program consisted of top-quality contributions. We also thank the members of the SOFTENG 2023 organizing committee for their help in handling the logistics of this event. We hope that SOFTENG 2023 was a successful international forum for the exchange of ideas and results between academia and industry and for the promotion of progress in the field of software engineering