Socially and Spatially Aware Motion Prediction of Dynamic Objects for Autonomous Driving

The primary goal of this thesis project is to develop a robust object motion prediction framework enabling safe decision making for autonomous vehicles in various driving scenarios. Given the comparatively higher importance and complexity of urban driving settings such as stop-sign controlled intersections or non-signalized/sign controlled roads are of primary interest; the approach, however, is not limited to these settings and is applicable to other driving settings. Specifically, motion prediction for all moving objects surrounding the autonomous vehicle such as pedestrians, cyclists, cars, and trucks is considered. In this thesis, estimation of the position and velocity of the objects surrounding the autonomous vehicle is performed using observed positions of the object in interest during a finite time window in the past, subsequent to which a socially and spatially informed model predicts the positions of these objects for a finite time window in the future through the use of the obtained position and velocity estimate as well as an artificial potential field (PF) modelling social interactions between surrounding objects and the scene. The necessary inputs for prediction are the class, position, and velocity of object of interest which can be obtained through 3D object detection approaches. However, often times, intermittent noise and/or loss in detections is observed pointing to the need for a robust estimation scheme. Traditional one-step lookback-based filtering and estimation approaches do not perform well due to a lack of sufficient prior information and simplistic model assumptions. On the other hand, most data-driven approaches do not offer any explicit embeddings of physical motion models or constraints leading to lack of generalizability in unseen scenarios. To this end, a constrained moving horizon state estimation (MHE) approach to estimate an object's states with respect to a global stationary frame including position, velocity, and acceleration that are robust to intermittently noisy or absent sensor measurements is proposed. Utilizing a computationally light-weight fusion of a Convolutional Neural Network (CNN)-based 2D object detection algorithm and projected LIDAR depth measurements, the approach produces the required measurements relative to the vehicle frame and combines them with the rotation and translation information obtained via a global positioning and inertial measurement system. The performance of the proposed approach is experimentally verified on an in-house dataset featuring urban crossings, with and without autonomous vehicle motion. Taking the position and velocity estimates as inputs, three key observations in microscopic agent-agent behaviour are incorporated for motion prediction namely – inclination to maintain direction of heading for pedestrians and follow lane centers for vehicles when free of surrounding agents, tendency to maintain heading and speed unless a collision is anticipated, and most importantly social interaction demonstrating collision avoidance. Traditionally, a fixed model or a model chosen from a fixed set of models is used for modelling future behaviour. These models are applicable to a variety of scenarios, however, they have an inherent bias and may lead to inaccurate predictions. On the other hand, purely data driven approaches suffer from a lack of holistic set of rules governing predictions and hence do not generalize well to a variety of scenarios. To address these issues, a novel potential field-based model predictive control (MPC) algorithm, MPC-PF, is proposed incorporating social interaction in a single cost function. Simulation results on a variety of scenarios including pedestrians and vehicles approaching directly head-on or otherwise show accurate predictions for a long future horizon. Furthermore, detailed qualitative and quantitative evaluation on a large public motion prediction dataset demonstrates state-of-the-art performance achieved by the proposed approach. Lastly, the potential field-based notion is integrated in a hybrid data driven Deep Deterministic Policy Gradient (DDPG) reinforcement learning (RL) agent, termed RL-PF, with a reward function governed by the potential field and is a valuable direction for further research and experimental validation

Fine-Tuning Language Models Using Formal Methods Feedback

Although pre-trained language models encode generic knowledge beneficial for planning and control, they may fail to generate appropriate control policies for domain-specific tasks. Existing fine-tuning methods use human feedback to address this limitation, however, sourcing human feedback is labor intensive and costly. We present a fully automated approach to fine-tune pre-trained language models for applications in autonomous systems, bridging the gap between generic knowledge and domain-specific requirements while reducing cost. The method synthesizes automaton-based controllers from pre-trained models guided by natural language task descriptions. These controllers are verifiable against independently provided specifications within a world model, which can be abstract or obtained from a high-fidelity simulator. Controllers with high compliance with the desired specifications receive higher ranks, guiding the iterative fine-tuning process. We provide quantitative evidences, primarily in autonomous driving, to demonstrate the method's effectiveness across multiple tasks. The results indicate an improvement in percentage of specifications satisfied by the controller from 60% to 90%

MM3DGS SLAM: Multi-modal 3D Gaussian Splatting for SLAM Using Vision, Depth, and Inertial Measurements

Simultaneous localization and mapping is essential for position tracking and scene understanding. 3D Gaussian-based map representations enable photorealistic reconstruction and real-time rendering of scenes using multiple posed cameras. We show for the first time that using 3D Gaussians for map representation with unposed camera images and inertial measurements can enable accurate SLAM. Our method, MM3DGS, addresses the limitations of prior neural radiance field-based representations by enabling faster rendering, scale awareness, and improved trajectory tracking. Our framework enables keyframe-based mapping and tracking utilizing loss functions that incorporate relative pose transformations from pre-integrated inertial measurements, depth estimates, and measures of photometric rendering quality. We also release a multi-modal dataset, UT-MM, collected from a mobile robot equipped with a camera and an inertial measurement unit. Experimental evaluation on several scenes from the dataset shows that MM3DGS achieves 3x improvement in tracking and 5% improvement in photometric rendering quality compared to the current 3DGS SLAM state-of-the-art, while allowing real-time rendering of a high-resolution dense 3D map. Project Webpage: https://vita-group.github.io/MM3DGS-SLAMComment: Project Webpage: https://vita-group.github.io/MM3DGS-SLA

Comparative outcomes of percutaneous coronary intervention for ST-segment-elevation myocardial infarction among medicare beneficiaries with multivessel coronary artery disease: An national cardiovascular data registry research to practice project

BACKGROUND: Prior studies on the use of multivessel percutaneous coronary intervention (MV PCI) for patients with STEMI and multivessel coronary artery disease have yielded heterogeneous results. The recent COMPLETE trial demonstrated that MV PCI was superior to culprit-only PCI among patients with STEMI. It is unclear how these trial results apply to clinical decisions encountered in routine practice. METHODS: We studied STEMI admissions among patients >65 years with multivessel disease and CMS-linked data in the NCDR CathPCI Registry® from 7/1/2009–12/31/2017. MV PCI was defined as PCI to a non-culprit lesion ≤45 days of the index procedure. The primary outcome was the composite of death, myocardial infarction, and revascularization from 45 days through 1 year. To account for unmeasured confounders, an instrumental variable analysis (IVA) was used to compare treatment strategies. The instrument was institutional rates of MV PCI. A falsification endpoint of post-discharge major bleeding was utilized to assess for residual confounding. RESULTS: Of 56,332 admissions from 1,102 institutions, 37.7% received MV PCI ≤45 days of index STEMI PCI. Of those undergoing MV PCI, 74.8% received complete revascularization. In unadjusted analysis, MV PCI was associated with a lower cumulative incidence of the composite outcome between 45 days and 1 year (13.9% vs 18.2% for non-MV PCI, p<0.01). In the IVA, there was no association between MV PCI and the composite outcome (adjusted risk difference [RD] −0.97%; 95%CI −3.52%, 1.59%; p=0.46). An association between MV PCI and the falsification endpoint of major bleeding was not observed (RD −2.54%; 95%CI −5.30%, 0.22%; p=0.07). CONCLUSIONS: In this large, nationwide analysis, we did not find benefit of MV PCI by 1 year among older STEMI patients. The clinical benefit of MV PCI may not extend equally outside of trials to include all patients, including those with more extreme ages and more complex decision making

Comparative Outcomes of Percutaneous Coronary Intervention for ST-Segment–Elevation Myocardial Infarction Among Medicare Beneficiaries With Multivessel Coronary Artery Disease: An National Cardiovascular Data Registry Research to Practice Project

[Figure: see text]

Association of Chronic Graft-versus-Host Disease with Late Effects following Allogeneic Hematopoietic Cell Transplantation for Children with Hematologic Malignancy

•Chronic graft-versus-host disease (cGVHD) is associated with nonmalignant late effects (NM-LEs) but not with subsequent neoplasms.•Severe cGVHD features are more closely associated with nonmalignant NM-LEs.•The duration of systemic immunosuppression was not predictive for LEs. Chronic graft-versus-host disease (cGVHD) occurs in up to 25% of children following allogeneic hematopoietic cell transplantation (HCT) and continues to be a major cause of late morbidity and poor quality of life among long-term survivors of pediatric HCT. Late effects (LEs) of HCT are well documented in this population, and cGVHD has been identified as a risk factor for subsequent neoplasms (SNs) and several nonmalignant LEs (NM-LEs); however, the reported correlation between cGVHD and LEs varies among studies. We compared LEs occurring ≥2 years following childhood HCT for a hematologic malignancy in 2-year disease-free survivors with and without cGVHD and further evaluated the association of cGVHD features on the development of LEs. This systematic retrospective analysis used data from the Center of International Blood and Marrow Transplant Research (CIBMTR) on a large, representative cohort of 1260 survivors of pediatric HCT for hematologic malignancy to compare first malignant LEs and NM-LEs in those with a diagnosis of cGVHD and those who never developed cGVHD. The cumulative incidences of any first LE, SN, and NM-LE were estimated at 10 years after HCT, with death as a competing risk for patients with cGVHD versus no cGVHD. Cox proportional hazards models were used to evaluate the impact of cGVHD and its related characteristics on the development of first LEs. The estimated 10-year cumulative incidence of any LE in patients with and without cGVHD was 43% (95% CI, 38% to 48.2%) versus 32% (95% confidence interval [CI], 28.5% to 36.3%) (P < .001), respectively. The development of cGVHD by 2 years post-HCT was independently associated with any LE (hazard ratio [HR], 1.38; 95% CI, 1.13 to 1.68; P = .001) and NM-LE (HR, 1.37; 95% CI, 1.10 to 1.70; P = .006), but not SN (HR, 1.30; 95% CI, .73 to 2.31; P = .38). cGVHD-related factors linked with the development of an NM-LE included having extensive grade cGVHD (HR, 1.60; 95% CI, 1.23 to 2.08; P = .0005), severe cGVHD (HR, 2.25; 95% CI, 1.60 to 3.17; P < .0001), interrupted onset type (HR, 1.57; 95% CI, 1.21 to 2.05; P = .0008), and both mucocutaneous and visceral organ involvement (HR, 1.59; 95% CI, 1.24 to 2.03; P = .0002). No significant association between cGVHD-specific variables and SN was identified. Finally, the duration of cGVHD treatment of cGVHD with systemic immunosuppression was not significantly associated with SNs or NM-LEs. cGVHD was more closely associated with NM-LEs than with SNs among survivors of pediatric HCT for hematologic malignancy. In this analysis, the development of SNs was strongly associated with the use of myeloablative total body irradiation. cGVHD-related characteristics consistent with a state of greater immune dysregulation were more closely linked to NM-LEs

Late hepatic toxicity surveillance for survivors of childhood, adolescent and young adult cancer: Recommendations from the international late effects of childhood cancer guideline harmonization group

BACKGROUND: Survivors of childhood, adolescent and young adult (CAYA) cancer may develop treatment-induced chronic liver disease. Surveillance guidelines can improve survivors' health outcomes. However, current recommendations vary, leading to uncertainty about optimal screening. The International Late Effects of Childhood Cancer Guideline Harmonization Group has developed recommendations for the surveillance of late hepatotoxicity after CAYA cancer. METHODS: Evidence-based methods based on the GRADE framework were used in guideline development. A multidisciplinary guideline panel performed systematic literature reviews, developed evidence summaries, appraised the evidence, and formulated recommendations on the basis of evidence, clinical judgement, and consideration of benefits versus the harms of the surveillance while allowing for flexibility in implementation across different health care systems. RESULTS: The guideline strongly recommends a physical examination and measurement of serum liver enzyme concentrations (ALT, AST, gGT, ALP) once at entry into long-term follow-up for survivors treated with radiotherapy potentially exposing the liver (moderate- to high-quality evidence). For survivors treated with busulfan, thioguanine, mercaptopurine, methotrexate, dactinomycin, hematopoietic stem cell transplantation (HSCT), or hepatic surgery, or with a history of chronic viral hepatitis or sinusoidal obstruction syndrome, similar surveillance for late hepatotoxicity once at entry into LTFU is reasonable (low-quality evidence/expert opinion, moderate recommendation). For survivors who have undergone HSCT and/or received multiple red blood cell transfusions, surveillance for iron overload with serum ferritin is strongly recommended once at long-term follow-up entry. CONCLUSIONS: These evidence-based, internationally-harmonized recommendations for the surveillance of late hepatic toxicity in cancer survivors can inform clinical care and guide future research of health outcomes for CAYA cancer survivors