Don't Look Back: Robustifying Place Categorization for Viewpoint- and Condition-Invariant Place Recognition

When a human drives a car along a road for the first time, they later recognize where they are on the return journey typically without needing to look in their rear-view mirror or turn around to look back, despite significant viewpoint and appearance change. Such navigation capabilities are typically attributed to our semantic visual understanding of the environment [1] beyond geometry to recognizing the types of places we are passing through such as "passing a shop on the left" or "moving through a forested area". Humans are in effect using place categorization [2] to perform specific place recognition even when the viewpoint is 180 degrees reversed. Recent advances in deep neural networks have enabled high-performance semantic understanding of visual places and scenes, opening up the possibility of emulating what humans do. In this work, we develop a novel methodology for using the semantics-aware higher-order layers of deep neural networks for recognizing specific places from within a reference database. To further improve the robustness to appearance change, we develop a descriptor normalization scheme that builds on the success of normalization schemes for pure appearance-based techniques such as SeqSLAM [3]. Using two different datasets - one road-based, one pedestrian-based, we evaluate the performance of the system in performing place recognition on reverse traversals of a route with a limited field of view camera and no turn-back-and-look behaviours, and compare to existing state-of-the-art techniques and vanilla off-the-shelf features. The results demonstrate significant improvements over the existing state of the art, especially for extreme perceptual challenges that involve both great viewpoint change and environmental appearance change. We also provide experimental analyses of the contributions of the various system components.Comment: 9 pages, 11 figures, ICRA 201

LoST? Appearance-Invariant Place Recognition for Opposite Viewpoints using Visual Semantics

Human visual scene understanding is so remarkable that we are able to recognize a revisited place when entering it from the opposite direction it was first visited, even in the presence of extreme variations in appearance. This capability is especially apparent during driving: a human driver can recognize where they are when travelling in the reverse direction along a route for the first time, without having to turn back and look. The difficulty of this problem exceeds any addressed in past appearance- and viewpoint-invariant visual place recognition (VPR) research, in part because large parts of the scene are not commonly observable from opposite directions. Consequently, as shown in this paper, the precision-recall performance of current state-of-the-art viewpoint- and appearance-invariant VPR techniques is orders of magnitude below what would be usable in a closed-loop system. Current engineered solutions predominantly rely on panoramic camera or LIDAR sensing setups; an eminently suitable engineering solution but one that is clearly very different to how humans navigate, which also has implications for how naturally humans could interact and communicate with the navigation system. In this paper we develop a suite of novel semantic- and appearance-based techniques to enable for the first time high performance place recognition in this challenging scenario. We first propose a novel Local Semantic Tensor (LoST) descriptor of images using the convolutional feature maps from a state-of-the-art dense semantic segmentation network. Then, to verify the spatial semantic arrangement of the top matching candidates, we develop a novel approach for mining semantically-salient keypoint correspondences.Comment: Accepted for Robotics: Science and Systems (RSS) 2018. Source code now available at https://github.com/oravus/lost

The Need for Inherently Privacy-Preserving Vision in Trustworthy Autonomous Systems

Vision is a popular and effective sensor for robotics from which we can derive rich information about the environment: the geometry and semantics of the scene, as well as the age, gender, identity, activity and even emotional state of humans within that scene. This raises important questions about the reach, lifespan, and potential misuse of this information. This paper is a call to action to consider privacy in the context of robotic vision. We propose a specific form privacy preservation in which no images are captured or could be reconstructed by an attacker even with full remote access. We present a set of principles by which such systems can be designed, and through a case study in localisation demonstrate in simulation a specific implementation that delivers an important robotic capability in an inherently privacy-preserving manner. This is a first step, and we hope to inspire future works that expand the range of applications open to sighted robotic systems.Comment: 7 pages, 6 figure

Episode-based active learning with Bayesian neural networks

We investigate different strategies for active learning with Bayesian deep neural networks. We focus our analysis on scenarios where new, unlabeled data is obtained episodically, such as commonly encountered in mobile robotics applications. An evaluation of different strategies for acquisition, updating, and final training on the CIFAR-10 dataset shows that incremental network updates with final training on the accumulated acquisition set are essential for best performance, while limiting the amount of required human labeling labor

Liver Cirrhosis Affects the Pharmacokinetics of the Six Substrates of the Basel Phenotyping Cocktail Differently.

BACKGROUND Activities of hepatic cytochrome P450 enzymes (CYPs) are relevant for hepatic clearance of drugs and known to be decreased in patients with liver cirrhosis. Several studies have reported the effect of liver cirrhosis on CYP activity, but the results are partially conflicting and for some CYPs lacking. OBJECTIVE In this study, we aimed to investigate the CYP activity in patients with liver cirrhosis with different Child stages (A-C) using the Basel phenotyping cocktail approach. METHODS We assessed the pharmacokinetics of the six compounds and their CYP-specific metabolites of the Basel phenotyping cocktail (CYP1A2: caffeine, CYP2B6: efavirenz, CYP2C9: flurbiprofen, CYP2C19: omeprazole, CYP2D6: metoprolol, CYP3A: midazolam) in patients with liver cirrhosis (n = 16 Child A cirrhosis, n = 15 Child B cirrhosis, n = 5 Child C cirrhosis) and matched control subjects (n = 12). RESULTS While liver cirrhosis only marginally affected the pharmacokinetics of the low to moderate extraction drugs efavirenz and flurbiprofen, the elimination rate of caffeine was reduced by 51% in patients with Child C cirrhosis. For the moderate to high extraction drugs omeprazole, metoprolol, and midazolam, liver cirrhosis decreased the elimination rate by 75%, 37%, and 60%, respectively, increased exposure, and decreased the apparent systemic clearance (clearance/bioavailability). In patients with Child C cirrhosis, the metabolic ratio (ratio of the area under the plasma concentration-time curve from 0 to 24 h of the metabolite to the parent compound), a marker for CYP activity, decreased by 66%, 47%, 92%, 73%, and 43% for paraxanthine/caffeine (CYP1A2), 8-hydroxyefavirenz/efavirenz (CYP2B6), 5-hydroxyomeprazole/omeprazole (CYP2C19), α-hydroxymetoprolol/metoprolol (CYP2D6), and 1'-hydroxymidazolam/midazolam (CYP3A), respectively. In comparison, the metabolic ratio 4-hydroxyflurbiprofen/flurbiprofen (CYP2C9) remained unchanged. CONCLUSIONS Liver cirrhosis affects the activity of CYP isoforms differently. This variability must be considered for dose adjustment of drugs in patients with liver cirrhosis. CLINICAL TRIAL REGISTRATION NCT03337945

Two-dimensional simulation of quantum reflection

A propagation method for the scattering of a quantum wave packet from a potential surface is presented. It is used to model the quantum reflection of single atoms from a corrugated (metallic) surface. Our numerical procedure works well in two spatial dimensions requiring only reasonable amounts of memory and computing time. The effects of the surface corrugation on the reflectivity are investigated via simulations with a paradigm potential. These indicate that our approach should allow for future tests of realistic, effective potentials obtained from theory in a quantitative comparison to experimental data

Pre-treatment comorbidities, C-reactive protein and eosinophil count, and immune-related adverse events as predictors of survival with checkpoint inhibition for multiple tumour entities

BACKGROUND The development of immune-related adverse events (irAEs) may be associated with clinical efficacy of checkpoint inhibitors (CPIs) in patients with cancer. We therefore investigated the effect of irAEs and pre-treatment parameters on outcome in a large, real-life patient cohort. METHODS We performed a single-centre, retrospective, observational study including patients who received CPIs from 2011 to 2018 and followed until 2021. The primary outcome was overall survival, and the secondary outcome was the development of irAEs. RESULTS In total, 229 patients with different tumour entities (41% non-small cell lung cancer [NSCLC], 29% melanoma) received a total of 282 CPI treatment courses (ipilimumab, nivolumab, pembrolizumab or atezolizumab). Thirty-four percent of patients developed irAEs (of these 17% had CTCAE Grade ≥3). Factors independently associated with mortality were pre-treatment CRP ≥10 mg/L (hazard ratio [HR] 2.064, p = 0.0003), comorbidity measured by Charlson comorbidity index (HR 1.149, p = 0.014) and irAEs (HR 0.644, p = 0.036) (age-adjusted, n = 216). Baseline eosinophil count ≤0.2 × 10$^{9}$ /L was a further independent predictor of mortality (age-, CRP-, CCI- and irAE-adjusted HR = 2.252, p = 0.002, n = 166). Anti-CTLA-4 use (p < 0.001), and pre-treatment CRP <10 mg/L were independently associated with irAE occurrence (p = 0.037). CONCLUSIONS We found an independent association between irAE occurrence and improved overall survival in a real-life cohort spanning multiple tumour entities and treatment regimens. Pre-treatment comorbidities, CRP and eosinophil count represent potential markers for predicting treatment response