Unsupervised RGB-to-Thermal Domain Adaptation via Multi-Domain Attention Network

This work presents a new method for unsupervised thermal image classification and semantic segmentation by transferring knowledge from the RGB domain using a multi-domain attention network. Our method does not require any thermal annotations or co-registered RGB-thermal pairs, enabling robots to perform visual tasks at night and in adverse weather conditions without incurring additional costs of data labeling and registration. Current unsupervised domain adaptation methods look to align global images or features across domains. However, when the domain shift is significantly larger for cross-modal data, not all features can be transferred. We solve this problem by using a shared backbone network that promotes generalization, and domain-specific attention that reduces negative transfer by attending to domain-invariant and easily-transferable features. Our approach outperforms the state-of-the-art RGB-to-thermal adaptation method in classification benchmarks, and is successfully applied to thermal river scene segmentation using only synthetic RGB images. Our code is made publicly available at https://github.com/ganlumomo/thermal-uda-attention

Fast Uncertainty Estimation for Deep Learning Based Optical Flow

We present a novel approach to reduce the processing time required to derive the estimation uncertainty map in deep learning-based optical flow determination methods. Without uncertainty aware reasoning, the optical flow model, especially when it is used for mission critical fields such as robotics and aerospace, can cause catastrophic failures. Although several approaches such as the ones based on Bayesian neural networks have been proposed to handle this issue, they are computationally expensive. Thus, to speed up the processing time, our approach applies a generative model, which is trained by input images and an uncertainty map derived through a Bayesian approach. By using synthetically generated images of spacecraft, we demonstrate that the trained generative model can produce the uncertainty map 100∼700 times faster than the conventional uncertainty estimation method used for training the generative model itself. We also show that the quality of uncertainty map derived by the generative model is close to that of the original uncertainty map. By applying the proposed approach, the deep learning model operated in real-time can avoid disastrous failures by considering the uncertainty as well as achieving better performance removing uncertain portions of the prediction result

Fast Uncertainty Estimation for Deep Learning Based Optical Flow

We present a novel approach to reduce the processing time required to derive the estimation uncertainty map in deep learning-based optical flow determination methods. Without uncertainty aware reasoning, the optical flow model, especially when it is used for mission critical fields such as robotics and aerospace, can cause catastrophic failures. Although several approaches such as the ones based on Bayesian neural networks have been proposed to handle this issue, they are computationally expensive. Thus, to speed up the processing time, our approach applies a generative model, which is trained by input images and an uncertainty map derived through a Bayesian approach. By using synthetically generated images of spacecraft, we demonstrate that the trained generative model can produce the uncertainty map 100∼700 times faster than the conventional uncertainty estimation method used for training the generative model itself. We also show that the quality of uncertainty map derived by the generative model is close to that of the original uncertainty map. By applying the proposed approach, the deep learning model operated in real-time can avoid disastrous failures by considering the uncertainty as well as achieving better performance removing uncertain portions of the prediction result

Photometric defocus observations of transiting extrasolar planets

We have carried out photometric follow-up observations of bright transiting extrasolar planets using the CbNUOJ 0.6m telescope. We have tested the possibility of obtaining high photometric precision by applying the telescope defocus technique allowing the use of several hundred seconds in exposure time for a single measurement. We demonstrate that this technique is capable of obtaining a root-mean-square scatter of order sub-millimagnitude over several hours for a V $\sim$ 10 host star typical for transiting planets detected from ground-based survey facilities. We compare our results with transit observations with the telescope operated in in-focus mode. High photometric precision is obtained due to the collection of a larger amount of photons resulting in a higher signal compared to other random and systematic noise sources. Accurate telescope tracking is likely to further contribute to lowering systematic noise by probing the same pixels on the CCD. Furthermore, a longer exposure time helps reducing the effect of scintillation noise which otherwise has a significant effect for small-aperture telescopes operated in in-focus mode. Finally we present the results of modelling four light-curves for which a root-mean-square scatter of 0.70 to 2.3 milli-magnitudes have been achieved.Comment: 12 pages, 11 figures, 5 tables. Submitted to Journal of Astronomy and Space Sciences (JASS

A Flight Mechanics-Centric Review of Bird-Scale Flapping Flight

This paper reviews the flight mechanics and control of birds and bird-size aircraft. It is intended to fill a niche in the current survey literature which focuses primarily on the aerodynamics, flight dynamics and control of insect scale flight. We review the flight mechanics from first principles and summarize some recent results on the stability and control of birds and bird-scale aircraft. Birds spend a considerable portion of their flight in the gliding (i.e., non-flapping) phase. Therefore, we also review the stability and control of gliding flight, and particularly those aspects which are derived from the unique control features of birds

Concomitant Laparoendoscopic Single-Site Surgery for Vesicolithotomy and Finger-Assisted Single-Port Transvesical Enucleation of the Prostate

Transurethral resection of the prostate is the most common surgery for benign prostatic hyperplasia. However, it doesn't work best for men with very large prostate and bladder stones. Herein we report our initial experience with concomitant laparoendoscopic single-site surgery and finger-assisted single-port transvesical enucleation of the prostate for the treatment of the condition

Online Self-Supervised Thermal Water Segmentation for Aerial Vehicles

We present a new method to adapt an RGB-trained water segmentation network to target-domain aerial thermal imagery using online self-supervision by leveraging texture and motion cues as supervisory signals. This new thermal capability enables current autonomous aerial robots operating in near-shore environments to perform tasks such as visual navigation, bathymetry, and flow tracking at night. Our method overcomes the problem of scarce and difficult-to-obtain near-shore thermal data that prevents the application of conventional supervised and unsupervised methods. In this work, we curate the first aerial thermal near-shore dataset, show that our approach outperforms fully-supervised segmentation models trained on limited target-domain thermal data, and demonstrate real-time capabilities onboard an Nvidia Jetson embedded computing platform. Code and datasets used in this work will be available at: https://github.com/connorlee77/uav-thermal-water-segmentation.Comment: 8 pages, 4 figures, 3 table

RGB-X Object Detection via Scene-Specific Fusion Modules

Multimodal deep sensor fusion has the potential to enable autonomous vehicles to visually understand their surrounding environments in all weather conditions. However, existing deep sensor fusion methods usually employ convoluted architectures with intermingled multimodal features, requiring large coregistered multimodal datasets for training. In this work, we present an efficient and modular RGB-X fusion network that can leverage and fuse pretrained single-modal models via scene-specific fusion modules, thereby enabling joint input-adaptive network architectures to be created using small, coregistered multimodal datasets. Our experiments demonstrate the superiority of our method compared to existing works on RGB-thermal and RGB-gated datasets, performing fusion using only a small amount of additional parameters. Our code is available at https://github.com/dsriaditya999/RGBXFusion.Comment: Accepted to 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2024