Incremental Visual-Inertial 3D Mesh Generation with Structural Regularities

Visual-Inertial Odometry (VIO) algorithms typically rely on a point cloud representation of the scene that does not model the topology of the environment. A 3D mesh instead offers a richer, yet lightweight, model. Nevertheless, building a 3D mesh out of the sparse and noisy 3D landmarks triangulated by a VIO algorithm often results in a mesh that does not fit the real scene. In order to regularize the mesh, previous approaches decouple state estimation from the 3D mesh regularization step, and either limit the 3D mesh to the current frame or let the mesh grow indefinitely. We propose instead to tightly couple mesh regularization and state estimation by detecting and enforcing structural regularities in a novel factor-graph formulation. We also propose to incrementally build the mesh by restricting its extent to the time-horizon of the VIO optimization; the resulting 3D mesh covers a larger portion of the scene than a per-frame approach while its memory usage and computational complexity remain bounded. We show that our approach successfully regularizes the mesh, while improving localization accuracy, when structural regularities are present, and remains operational in scenes without regularities.Comment: 7 pages, 5 figures, ICRA accepte

Ultimate SLAM? Combining Events, Images, and IMU for Robust Visual SLAM in HDR and High Speed Scenarios

Event cameras are bio-inspired vision sensors that output pixel-level brightness changes instead of standard intensity frames. These cameras do not suffer from motion blur and have a very high dynamic range, which enables them to provide reliable visual information during high speed motions or in scenes characterized by high dynamic range. However, event cameras output only little information when the amount of motion is limited, such as in the case of almost still motion. Conversely, standard cameras provide instant and rich information about the environment most of the time (in low-speed and good lighting scenarios), but they fail severely in case of fast motions, or difficult lighting such as high dynamic range or low light scenes. In this paper, we present the first state estimation pipeline that leverages the complementary advantages of these two sensors by fusing in a tightly-coupled manner events, standard frames, and inertial measurements. We show on the publicly available Event Camera Dataset that our hybrid pipeline leads to an accuracy improvement of 130% over event-only pipelines, and 85% over standard-frames-only visual-inertial systems, while still being computationally tractable. Furthermore, we use our pipeline to demonstrate - to the best of our knowledge - the first autonomous quadrotor flight using an event camera for state estimation, unlocking flight scenarios that were not reachable with traditional visual-inertial odometry, such as low-light environments and high-dynamic range scenes.Comment: 8 pages, 9 figures, 2 table

Kimera: from SLAM to Spatial Perception with 3D Dynamic Scene Graphs

Humans are able to form a complex mental model of the environment they move in. This mental model captures geometric and semantic aspects of the scene, describes the environment at multiple levels of abstractions (e.g., objects, rooms, buildings), includes static and dynamic entities and their relations (e.g., a person is in a room at a given time). In contrast, current robots' internal representations still provide a partial and fragmented understanding of the environment, either in the form of a sparse or dense set of geometric primitives (e.g., points, lines, planes, voxels) or as a collection of objects. This paper attempts to reduce the gap between robot and human perception by introducing a novel representation, a 3D Dynamic Scene Graph(DSG), that seamlessly captures metric and semantic aspects of a dynamic environment. A DSG is a layered graph where nodes represent spatial concepts at different levels of abstraction, and edges represent spatio-temporal relations among nodes. Our second contribution is Kimera, the first fully automatic method to build a DSG from visual-inertial data. Kimera includes state-of-the-art techniques for visual-inertial SLAM, metric-semantic 3D reconstruction, object localization, human pose and shape estimation, and scene parsing. Our third contribution is a comprehensive evaluation of Kimera in real-life datasets and photo-realistic simulations, including a newly released dataset, uHumans2, which simulates a collection of crowded indoor and outdoor scenes. Our evaluation shows that Kimera achieves state-of-the-art performance in visual-inertial SLAM, estimates an accurate 3D metric-semantic mesh model in real-time, and builds a DSG of a complex indoor environment with tens of objects and humans in minutes. Our final contribution shows how to use a DSG for real-time hierarchical semantic path-planning. The core modules in Kimera are open-source.Comment: 34 pages, 25 figures, 9 tables. arXiv admin note: text overlap with arXiv:2002.0628

Extramedullary disease in multiple myeloma: a systematic literature review

Extramedullary involvement (or extramedullary disease, EMD) represents an aggressive form of multiple myeloma (MM), characterized by the ability of a clone and/or subclone to thrive and grow independent of the bone marrow microenvironment. Several different definitions of EMD have been used in the published literature. We advocate that true EMD is restricted to soft-tissue plasmacytomas that arise due to hematogenous spread and have no contact with bony structures. Typical sites of EMD vary according to the phase of MM. At diagnosis, EMD is typically found in skin and soft tissues; at relapse, typical sites involved include liver, kidneys, lymph nodes, central nervous system (CNS), breast, pleura, and pericardium. The reported incidence of EMD varies considerably, and differences in diagnostic approach between studies are likely to contribute to this variability. In patients with newly diagnosed MM, the reported incidence ranges from 0.5% to 4.8%, while in relapsed/refractory MM the reported incidence is 3.4 to 14%. Available data demonstrate that the prognosis is poor, and considerably worse than for MM without soft-tissue plasmacytomas. Among patients with plasmacytomas, those with EMD have poorer outcomes than those with paraskeletal involvement. CNS involvement is rare, but prognosis is even more dismal than for EMD in other locations, particularly if there is leptomeningeal involvement. Available data on treatment outcomes for EMD are derived almost entirely from retrospective studies. Some agents and combinations have shown a degree of efficacy but, as would be expected, this is less than in MM patients with no extramedullary involvement. The paucity of prospective studies makes it difficult to justify strong recommendations for any treatment approach. Prospective data from patients with clearly defined EMD are important for the optimal evaluation of treatment outcomes

Extramedullary disease in multiple myeloma: a systematic literature review

Extramedullary involvement (or extramedullary disease, EMD) represents an aggressive form of multiple myeloma (MM), characterized by the ability of a clone and/or subclone to thrive and grow independent of the bone marrow microenvironment. Several different definitions of EMD have been used in the published literature. We advocate that true EMD is restricted to soft-tissue plasmacytomas that arise due to hematogenous spread and have no contact with bony structures. Typical sites of EMD vary according to the phase of MM. At diagnosis, EMD is typically found in skin and soft tissues; at relapse, typical sites involved include liver, kidneys, lymph nodes, central nervous system (CNS), breast, pleura, and pericardium. The reported incidence of EMD varies considerably, and differences in diagnostic approach between studies are likely to contribute to this variability. In patients with newly diagnosed MM, the reported incidence ranges from 0.5% to 4.8%, while in relapsed/refractory MM the reported incidence is 3.4 to 14%. Available data demonstrate that the prognosis is poor, and considerably worse than for MM without soft-tissue plasmacytomas. Among patients with plasmacytomas, those with EMD have poorer outcomes than those with paraskeletal involvement. CNS involvement is rare, but prognosis is even more dismal than for EMD in other locations, particularly if there is leptomeningeal involvement. Available data on treatment outcomes for EMD are derived almost entirely from retrospective studies. Some agents and combinations have shown a degree of efficacy but, as would be expected, this is less than in MM patients with no extramedullary involvement. The paucity of prospective studies makes it difficult to justify strong recommendations for any treatment approach. Prospective data from patients with clearly defined EMD are important for the optimal evaluation of treatment outcomes

Asynchronous, Photometric Feature Tracking using Events and Frames

We present a method that leverages the complementarity of event cameras and standard cameras to track visual features with low-latency. Event cameras are novel sensors that output pixel-level brightness changes, called "events". They offer significant advantages over standard cameras, namely a very high dynamic range, no motion blur, and a latency in the order of microseconds. However, because the same scene pattern can produce different events depending on the motion direction, establishing event correspondences across time is challenging. By contrast, standard cameras provide intensity measurements (frames) that do not depend on motion direction. Our method extracts features on frames and subsequently tracks them asynchronously using events, thereby exploiting the best of both types of data: the frames provide a photometric representation that does not depend on motion direction and the events provide low-latency updates. In contrast to previous works, which are based on heuristics, this is the first principled method that uses raw intensity measurements directly, based on a generative event model within a maximum-likelihood framework. As a result, our method produces feature tracks that are both more accurate (subpixel accuracy) and longer than the state of the art, across a wide variety of scenes.Comment: 22 pages, 15 figures, Video: https://youtu.be/A7UfeUnG6c

Isatuximab plus carfilzomib and dexamethasone versus carfilzomib and dexamethasone in elderly patients with relapsed multiple myeloma: IKEMA subgroup analysis

In this subgroup analysis of the randomized, Phase 3 IKEMA study (NCT03275285), we evaluated efficacy and safety of the anti-CD38 monoclonal antibody isatuximab (Isa) in combination with carfilzomib-dexamethasone (Isa-Kd) versus Kd in older (≥70 years of age, n = 86) and younger (<70 years, n = 216) patients with relapsed multiple myeloma (MM). Patients received Isa 10 mg/kg intravenously weekly for 4 weeks, then every 2 weeks in the Isa-Kd arm, and approved schedule of carfilzomib (twice weekly) and dexamethasone in both study arms. Primary endpoint was progression-free survival (PFS); key secondary efficacy endpoints included rates of overall response (ORR), very good partial response or better (≥VGPR), minimal residual disease negativity (MRD–), and complete response (CR). Addition of Isa to Kd resulted in improved PFS in elderly patients (hazard ratio, 0.36 [95% CI, 0.18–0.75]) consistent with the significant PFS improvement observed in the overall IKEMA population. Treatment with Isa-Kd improved depth of response versus Kd, with higher rates of ≥VGPR (73.1% vs. 55.9%), MRD– (23.1% vs. 11.8%), and CR (38.5% vs. 23.5%). Although the incidence of grade ≥3 treatment-emergent adverse events (TEAEs) was higher in Isa-Kd, the incidence of serious TEAEs was similar between arms. Fewer elderly patients definitively discontinued treatment due to TEAEs in Isa-Kd than Kd: 11.8% versus 23.5%. In conclusion, Isa-Kd provides a consistent benefit versus Kd in elderly patients, with a manageable safety profile, and represents a new treatment option for patients with relapsed MM, independent of age

Isatuximab plus carfilzomib and dexamethasone in relapsed multiple myeloma patients with high-risk cytogenetics: IKEMA subgroup analysis

Introduction: The presence of high-risk chromosomal abnormalities [t(4;14), del(17p), and t(14;16)] has been linked with inferior outcomes in patients with multiple myeloma (MM). A prespecified interim analysis of the Phase 3 IKEMA study (NCT03275285) demonstrated that isatuximab (Isa) + carfilzomib (K) and dexamethasone (d; Isa-Kd) significantly improved progression-free survival (PFS) versus Kd in patients with relapsed MM. This prespecified subgroup analysis of IKEMA examined efficacy and safety in patients with high-risk cytogenetics. Methods: High-risk cytogenetics was assessed by central laboratory and patients were classified as high risk if abnormalities were present in ≥1 of the following: del(17p): 50% cutoff; t(4;14), and/or t(14;16): 30% cutoff. Results: Of the randomized patients, 23.5% (Isa-Kd) and 25.2% (Kd) had ≥1 high-risk chromosomal abnormality. A PFS benefit was seen in favor of Isa-Kd for patients with standard-risk (HR 0.440; 95% CI 0.266–0.728) and high-risk cytogenetics (HR 0.724; 95% CI 0.361–1.451). Grade ≥3 treatment-emergent adverse events (TEAEs) were more common with Isa-Kd (85.7%) versus Kd (63.3%) in patients with high-risk cytogenetics; however, the incidence of serious TEAEs (64.3% vs. 66.7%) was similar. Conclusions: Isa-Kd is a new treatment option for the difficult-to-treat subgroup of patients with relapsed MM and high-risk cytogenetics