Perception and Prediction in Multi-Agent Urban Traffic Scenarios for Autonomous Driving

In multi-agent urban scenarios, autonomous vehicles navigate an intricate network of interactions with a variety of agents, necessitating advanced perception modeling and trajectory prediction. Research to improve perception modeling and trajectory prediction in autonomous vehicles is fundamental to enhance safety and efficiency in complex driving scenarios. Better data association for 3D multi-object tracking ensures consistent identification and tracking of multiple objects over time, crucial in crowded urban environments to avoid mis-identifications that can lead to unsafe maneuvers or collisions. Effective context modeling for 3D object detection aids in interpreting complex scenes, effectively dealing with challenges like noisy or missing points in sensor data, and occlusions. It enables the system to infer properties of partially observed or obscured objects, enhancing the robustness of the autonomous system in varying conditions. Furthermore, improved trajectory prediction of surrounding vehicles allows an autonomous vehicle to anticipate future actions of other road agents and adapt accordingly, crucial in scenarios like merging lanes, making unprotected turns, or navigating intersections. In essence, these research directions are key to mitigating risks in autonomous driving, and facilitating seamless interaction with other road users. In Part I, we address the task of improving perception modeling for AV systems. Concretely our contributions are: (i) FANTrack introduces a novel application of Convolutional Neural Networks (CNNs) for real-time 3D Multi-object Tracking (MOT) in autonomous driving, addressing challenges such as varying number of targets, track fragmentation, and noisy detections, thereby enhancing the accuracy of perception capabilities for safe and efficient navigation. (ii) FANTrack proposes to leverage both visual and 3D bounding box data, utilizing Siamese networks and hard-mining, to enhance the similarity functions used in data associations for 3D Multi-object Tracking (MOT). (iii) SA-Det3D introduces a globally-adaptive Full Self-Attention (FSA) module for enhanced feature extraction in 3D object detection, overcoming the limitations of traditional convolution-based techniques by facilitating adaptive context aggregation from entire point-cloud data, thereby bolstering perception modeling in autonomous driving. (iv) SA-Det3D also introduces the Deformable Self-Attention (DSA) module, a scalable adaptation for global context assimilation in large-scale point-cloud datasets, designed to select and focus on most informative regions, thereby improving the quality of feature descriptors and perception modeling in autonomous driving. In Part II, we focus on the task of improving trajectory prediction of surrounding agents. Concretely, our contributions are: (i) SSL-Lanes introduces a self-supervised learning approach for motion forecasting in autonomous driving that enhances accuracy and generalizability without compromising inference speed or model simplicity, utilizing pseudo-labels from pretext tasks for learning transferable motion patterns. (ii) The second contribution in SSL-Lanes is the design of comprehensive experiments to demonstrate that SSL-Lanes can yield more generalizable and robust trajectory predictions than traditional supervised learning approaches. (iii) SSL-Interactions presents a new framework that utilizes pretext tasks to enhance interaction modeling for trajectory prediction in autonomous driving. (iv) SSL-Interactions advances the prediction of agent trajectories in interaction-centric scenarios by creating a curated dataset that explicitly labels meaningful interactions, thus enabling the effective training of a predictor utilizing pretext tasks and enhancing the modeling of agent-agent interactions in autonomous driving environments

Tip-Enhanced Nano-Spectroscopy, Imaging, and Control: From Single Molecules to Van Der Waals Materials

Photon-induced phenomena in molecules and other materials play a significant role in device applications as well as understanding their physical properties. While a range of device applications using organic and inorganic molecules and soft and hard materials have led striking developments in modern technologies, using bulk systems has reached the limit in their functions, performance, and regarding application range. Recently, low-dimensional systems have emerged as appealing resources for the advanced technologies based on their significantly improved functions and properties. Hence, understanding light-matter interactions at their natural length scale is of fundamental significance, in addition to the next generation device applications. This thesis demonstrates a range of new functions and behaviors of low-dimensional materials revealed and controlled by the advanced tip-enhanced near-field spectroscopy and imaging techniques exceeding the current instrumental limits. To understand the behaviors of zero-dimensional (0D) molecular systems in interacting environments, we explore new regimes in tip-enhanced Raman spectroscopy (TERS) and scanning near-field optical microscopy (SNOM), revealing the fundamental nature of single-molecule dynamics and nanoscale spatial heterogeneity of biomolecules on the cell membranes. To gain insight into intramolecular properties and dynamic processes of single molecules, we use TERS at cryogenic temperatures. From temperature-dependent line narrowing and splitting, we investigate and quantify ultrafast vibrational dephasing, intramolecular coupling, and conformational heterogeneity. Through correlation analysis of fluctuations of individual modes, we observe rotational motion and spectral fluctuations of single-molecule. We extend single-molecule spectroscopy study into in situ nano-biomolecular imaging of cancer cells by developing in-liquid SNOM. We use a new mechanical resonance control, achieving a high-Q force sensing of the near-field probe. We reveal nanoscale correlations between surface biomolecules and intracellular organelle structures through near-field imaging of the spatial distribution of EGFRs on the membrane of A431 cancer cells. In addition, to understand modified spontaneous emission properties of single quantum dots coupled strongly with localized plasmon, we perform tip-enhanced photoluminescence (TEPL) spectroscopy of the single CdSe/ZnS quantum dots on gold film. We probe and control nanoscale processes in van der Waals two-dimensional (2D) materials. To understand lattice and electronic structure as well as elastic and phonon scattering properties of grain boundaries (GBs) in large-area graphene, we perform TERS imaging. Through correlated analysis of multispectral TERS images with corresponding topography and near-field scattering image, we reveal bilayer structure of GBs in the form of twisted stacking. In addition, we determine the misorientation angles of the bilayer GBs from a detailed quantitative investigation of the Raman modes. In addition, we present a new hybrid nano-optomechanical tip-enhanced spectroscopy and imaging approach combining TERS, TEPL, and atomic force local strain manipulation to probe the heterogeneous PL responses at nanoscale defects and control the local bandgap in transition metal dichalcogenide (TMD) monolayer. We further extend this approach to probe and control the radiative emission of dark excitons and localized excitons. Based on nano-tip enhanced spectroscopy with &sim;6 &times; 105-fold PL enhancement induced by the plasmonic Purcell effect and few-fs radiative dynamics of the optical antenna tip, we can directly probe and actively modulate the dark exciton and localized exciton emissions in time (~ms) and space (&lt;15 nm) at room temperature. Lastly, to extend the range of tip-enhanced microscopy applications to nano-crystallography and nonlinear optics, we present a generalizable approach controlling the excitation polarizability for both in-plane and out-of-plane vector fields by breaking the axial symmetry of a conventional Au tip. This vector field control with the tip enables probing of nonlinear optical second harmonic generation (SHG) responses from a range of ferroic materials as well as van der Waals 2D materials. Specifically, we demonstrate SHG nano-crystallography results for MoS2 monolayer film, ferroelectric YMnO3, BaTiO3-BiFeO3 multiferroics, and PbTiO3/SrTiO3 superlattices.</p

High throughput surface mass spectrometry-based proteomics & metabolomics for biological applications

Surface-based mass spectrometry analysis benefits from the minimum sample preparation required and high throughput nature of the analysis (few minutes per sample) and shows therefore potential for tackling current issues in the field of proteomics and metabolomics. This thesis aims to develop a robust high throughput methodology for the quantitative analysis of surface-adsorbed proteins and for untargeted metabolomics One of the current problems in the field of biomaterials research is the limited understanding of the mechanistic behind cell attachment and behaviour on polymeric substrates. Fully synthetic substrates have been identified which support growth and survival of human pluripotent stem cells. Pluripotent stem cells are a valuable cell type for regenerative medicine due to their ability to differentiate into the three germ layers. To treat a single patient, more than a billion stem cells are required. Current cell systems use biological feeder layers for stem cell expansion. However, these animal-derived matrices are expensive, undefined, and show high batch-to-batch variation. In order to move towards reproducible, industrial culturing of stem cells a suitable growth substrate needs to be defined. Through high throughput biomaterials discovery, it was shown that some fully synthetic polymers can maintain stem cell cultures to a similar level as biological substrates. Current understanding of the response of cells on those synthetic polymers is relatively poor. Research has shown that coating of synthetic polymers with culture medium-derived proteins increase the cell attachment which is required for cell survival. This shows the potential role of culture medium proteins in the response mechanism of cells on synthetic polymers. However, current technology does not allow analysis of (combinatorial) polymer libraries which has limited the understanding of relation between cell response and physicochemical properties and molecular features of the polymers. A full understanding of the cell-polymer response mechanism would allow the development and rationalisation of synthetic polymers for culturing of pluripotent stem cells. It was shown that liquid extraction surface analysis-tandem mass spectrometry (LESA-MS/MS) is a suitable analytical technique for the analysis of in situ digested proteins. LESA is a commercial system which can automatically extract analytes from a given substrate and directly introduce the sample in to the MS. Here, this potential was further explored for polymer array screening as well as polymers taken forward for scale-up experiments. A suitable substrate was chosen (Droplet Microarray) which allowed control over the spreading of the monomer solutions, digestion solution, and organic extraction solvent for reproducible MS results. With carefully optimised LESA and MS parameters, difference in protein adsorption could be detected between different chemical surfaces. These difference in protein adsorption did not show a good correlation with the observed cell response (attachment and number of pluripotent stem cells). Through multivariate modelling was found that surface chemistry was found to play a role in protein adsorption. Whilst array screening did not reveal solid evidence of the importance of protein adsorption in relation to cellular response, experiments of protein adsorption on a larger surface area (6-well plates) revealed higher protein adsorption on polymers with higher numbers of pluripotent stem cells. Altogether, LESA-MS/MS shows to be an interesting tool to quantitatively assess protein adsorption on synthetic polymers. The developed methodology can not only be further used to study more complex growth media for human cell lines, but also extended study the relation between protein adsorption and response of different organisms. The addition of LESA-MS/MS to high throughput screening of material microarrays might reveal vital information and could assist in proper choice of polymers for biomedical purposes. Further interest of surface analysis comes from the field of oncometabolomics. In this thesis, paediatric ependymoma were analysed by Orbitrap secondary ion mass spectrometry (3D OrbiSIMS) and LESA-MS/MS. The main challenge here was to acquire data using only minimal tumour tissue which was available in the form of a tumour tissue microarray. By analysing the same tumour tissue with two complementary mass spectrometry techniques, a more complete metabolite profile could be obtained. Moreover, the combination of 3D OrbiSIMS and LESA-MS/MS data followed by partial-least squares discriminant analysis (PLS-DA) permitted the classification of tumour tissue based on eventual recurrence. This means that certain metabolite levels are indicative of tumour relapse. Understanding these changes in metabolite abundance along with the changes in corresponding metabolic pathways could open new insight into ependymoma relapse. Further, this analytical strategy would be suitable to study other types of (tumour) tissues

High throughput surface mass spectrometry-based proteomics & metabolomics for biological applications

Surface-based mass spectrometry analysis benefits from the minimum sample preparation required and high throughput nature of the analysis (few minutes per sample) and shows therefore potential for tackling current issues in the field of proteomics and metabolomics. This thesis aims to develop a robust high throughput methodology for the quantitative analysis of surface-adsorbed proteins and for untargeted metabolomics One of the current problems in the field of biomaterials research is the limited understanding of the mechanistic behind cell attachment and behaviour on polymeric substrates. Fully synthetic substrates have been identified which support growth and survival of human pluripotent stem cells. Pluripotent stem cells are a valuable cell type for regenerative medicine due to their ability to differentiate into the three germ layers. To treat a single patient, more than a billion stem cells are required. Current cell systems use biological feeder layers for stem cell expansion. However, these animal-derived matrices are expensive, undefined, and show high batch-to-batch variation. In order to move towards reproducible, industrial culturing of stem cells a suitable growth substrate needs to be defined. Through high throughput biomaterials discovery, it was shown that some fully synthetic polymers can maintain stem cell cultures to a similar level as biological substrates. Current understanding of the response of cells on those synthetic polymers is relatively poor. Research has shown that coating of synthetic polymers with culture medium-derived proteins increase the cell attachment which is required for cell survival. This shows the potential role of culture medium proteins in the response mechanism of cells on synthetic polymers. However, current technology does not allow analysis of (combinatorial) polymer libraries which has limited the understanding of relation between cell response and physicochemical properties and molecular features of the polymers. A full understanding of the cell-polymer response mechanism would allow the development and rationalisation of synthetic polymers for culturing of pluripotent stem cells. It was shown that liquid extraction surface analysis-tandem mass spectrometry (LESA-MS/MS) is a suitable analytical technique for the analysis of in situ digested proteins. LESA is a commercial system which can automatically extract analytes from a given substrate and directly introduce the sample in to the MS. Here, this potential was further explored for polymer array screening as well as polymers taken forward for scale-up experiments. A suitable substrate was chosen (Droplet Microarray) which allowed control over the spreading of the monomer solutions, digestion solution, and organic extraction solvent for reproducible MS results. With carefully optimised LESA and MS parameters, difference in protein adsorption could be detected between different chemical surfaces. These difference in protein adsorption did not show a good correlation with the observed cell response (attachment and number of pluripotent stem cells). Through multivariate modelling was found that surface chemistry was found to play a role in protein adsorption. Whilst array screening did not reveal solid evidence of the importance of protein adsorption in relation to cellular response, experiments of protein adsorption on a larger surface area (6-well plates) revealed higher protein adsorption on polymers with higher numbers of pluripotent stem cells. Altogether, LESA-MS/MS shows to be an interesting tool to quantitatively assess protein adsorption on synthetic polymers. The developed methodology can not only be further used to study more complex growth media for human cell lines, but also extended study the relation between protein adsorption and response of different organisms. The addition of LESA-MS/MS to high throughput screening of material microarrays might reveal vital information and could assist in proper choice of polymers for biomedical purposes. Further interest of surface analysis comes from the field of oncometabolomics. In this thesis, paediatric ependymoma were analysed by Orbitrap secondary ion mass spectrometry (3D OrbiSIMS) and LESA-MS/MS. The main challenge here was to acquire data using only minimal tumour tissue which was available in the form of a tumour tissue microarray. By analysing the same tumour tissue with two complementary mass spectrometry techniques, a more complete metabolite profile could be obtained. Moreover, the combination of 3D OrbiSIMS and LESA-MS/MS data followed by partial-least squares discriminant analysis (PLS-DA) permitted the classification of tumour tissue based on eventual recurrence. This means that certain metabolite levels are indicative of tumour relapse. Understanding these changes in metabolite abundance along with the changes in corresponding metabolic pathways could open new insight into ependymoma relapse. Further, this analytical strategy would be suitable to study other types of (tumour) tissues

Enhancing Federated Learning Robustness and Fairness in Non-IID Scenarios

Federated Learning is a distributed machine learning paradigm that allows multiple clients to collaboratively train a joint model without sharing the raw data. Despite its advantages, FL faces the security issues inherent to its decentralized nature, and FL clients often encounter unfair treatment from the design that prioritizes server interests. Today, many studies have been proposed to mitigate the research gap; nevertheless, in the absence of a non-IID setting, ensuring robustness and fairness in FL remains an open problem. Therefore, in this thesis, we study several topics on the robustness and fairness of FL in non-IID scenarios, including attack surface reduction, poisoning attack defense, and implicit class-level fair enhancement. We start by investigating FL's non-IID resource and propose the Mini FL framework. Based on a predefined grouping principle, Mini FL assigns similar clients to different groups and aggregates them respectively to achieve attack surface reduction. Then, we focus on defending against FL poisoning attacks. For the Label Flipping Attack, we introduce the HSCS FL method. It evaluates the accuracy of each class in both global and local models in each iteration. These accuracies are then translated into a score, and only clients with top scores are included in the current aggregation. For the Class Imbalance Attack, we introduce the Class-Balanced FL framework. This approach dynamically determines the aggregation weight for each client, considering their potential contribution to the current global model, thereby preventing the joint model biases toward specific data distributions. Lastly, we propose the ICB FL method to enhance FL fairness. This framework enables the server to identify implicit classes and dynamically distribute weights, ensuring a similar learning performance across these implicit classes. We provide mathematical proofs for each scheme and framework we proposed and conduct experiments to show their effectiveness

Homeobox genes in the development and regeneration of the cephalochordate Branchiostoma lanceolatum and the polychaete annelid Spirobranchus lamarcki

The development of complex animal morphology requires the extremely sophisticated spatiotemporal coordination of cell behaviour and communication. Homeobox genes encode transcription factors that are deployed in developmental processes to control the expression of other genes in particular locations and contexts. Many homeobox genes are highly conserved and act in similar roles between distantly-related animals that derive from the roles of their ancestral orthologues. The way that these genes have differentially evolved between taxa, and the effect that these changes have on the development and morphology of animals, is critical to our understanding of metazoan evolution. One particular developmental context, the regeneration of missing tissue, offers a unique perspective on evolutionary developmental biology because of its relationship to ontogenic development and its surprising diversity of retention and process between animal taxa. I examined the homeobox gene content of transcriptomes taken from the mature and regenerating tissue of the post-anal tail of Branchiostoma lanceolatum, a well-studied cephalochordate with a highly conserved genome, and the evolutionarily novel operculum of Spirobranchus lamarcki, a sedentarian annelid. In S. lamarcki regeneration, a diverse variety of homeobox genes is expressed, and the regenerative expression response is substantial. The discovery of several difficult-to-classify homeobox genes lead to the substantial expansion and improvement of the classification of a variety of homeobox genes undergoing unusual rapid and expansive evolution in the Spiralia, including dozens of TALE and PRD class genes, a new orthology group, and a strange S. lamarcki Hox gene. In B. lanceolatum, a similar diversity of expressed genes is observed but a milder regenerative response. One transcriptomic sequence in particular, identified as Pax3/7, led to the discovery that this well-studied gene has a previously unnoticed duplication in cephalochordates. This discovery has implications for ongoing study of vertebrate and cephalochordate neural plate border evolution

The HBP1 tumor suppressor is a negative epigenetic regulator of MYCN driven neuroblastoma through interaction with the PRC2 complex

C

Quantifying expression variability in single-cell RNA sequencing data

Transcriptional noise is an intrinsic feature of cell populations and plays a driving role in mammalian development, tissue homoeostasis and immune function. While expression heterogeneity, a phenotypic readout of transcriptional noise, has been broadly studied in prokaryotic model systems or by profiling individual genes, few whole-transcriptome studies in mammalian systems have been reported. The development of single-cell RNA sequencing technologies introduced powerful tools to investigate transcriptional differences between individual cells, therefore allowing the in-depth characterisation of expression variability. In this thesis, I computationally analysed single-cell RNA sequencing data to understand transcriptional variability and expanded a statistical model to avoid confounding effects when quantifying such variability. First, I profiled individual transcriptomes of CD4$^+$ T cells, identifying a global decrease in transcriptional variability upon immune activation. By extending this analysis across two sub-species of mice, I identified an evolutionarily conserved set of immune response genes for which transcriptional variability increases during ageing. I used a Bayesian modelling framework to quantify mean expression and transcriptional variability but due to a strong confounding effect between these two parameters, variability analysis was restricted to genes that are similarly expressed across the tested conditions. To address this problem, I extended the computational framework allowing the parallel assessment of changes in mean expression and variability. Within this Bayesian framework, I introduced a joint prior linking mean expression and variability parameters, which allowed a residual over-dispersion to be measured for each gene. This measure allowed me to statistically assess changes in variability even for genes with differences in mean expression between conditions. Finally, I applied the model to identify temporal changes in variability over the time-course of spermatogenesis. This unidirectional differentiation process involves several complex steps before mature sperm form from spermatogonial stem cells. When profiling changes in variability across this developmental time-course, peaks in variability are caused by rapid changes in gene expression along the differentiation trajectory. This thesis provides a deeper understanding of technical and biological factors that drive transcriptional variability and offers a basis for future research to characterise its role in health and disease.Funding was provided via the EMBL international PhD programm