RGB-X Object Detection via Scene-Specific Fusion Modules

Multimodal deep sensor fusion has the potential to enable autonomous vehicles to visually understand their surrounding environments in all weather conditions. However, existing deep sensor fusion methods usually employ convoluted architectures with intermingled multimodal features, requiring large coregistered multimodal datasets for training. In this work, we present an efficient and modular RGB-X fusion network that can leverage and fuse pretrained single-modal models via scene-specific fusion modules, thereby enabling joint input-adaptive network architectures to be created using small, coregistered multimodal datasets. Our experiments demonstrate the superiority of our method compared to existing works on RGB-thermal and RGB-gated datasets, performing fusion using only a small amount of additional parameters. Our code is available at https://github.com/dsriaditya999/RGBXFusion.Comment: Accepted to 2024 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV 2024

Catching geomorphological response to volcanic activity on steep slope volcanoes using multi-platform remote sensing

The geomorphological evolution of the volcanic Island of Stromboli (Italy) between July 2010 and June 2019 has been reconstructed by using multi-temporal, multi-platform remote sensing data. Digital elevation models (DEMs) from PLÉIADES-1 tri-stereo images and from Light Detection and Ranging (LiDAR) acquisitions allowed for topographic changes estimation. Data were comprised of high-spatial-resolution (QUICKBIRD) and moderate spatial resolution (SENTINEL-2) satellite images that allowed for the mapping of areas that were affected by major lithological and morphological changes. PLÉIADES tri-stereo and LiDAR DEMs have been quantitatively and qualitatively compared and, although there are artefacts in the smaller structures (e.g., ridges and valleys), there is still a clear consistency between the two DEMs for the larger structures (as the main valleys and ridges). The period between July 2010 and May 2012 showed only minor changes consisting of volcanoclastic sedimentation and some overflows outside the crater. Otherwise, between May 2012 and May 2017, large topographic changes occurred that were related to the emplacement of the 2014 lava flow in the NE part of the Sciara del Fuoco and to the accumulation of a volcaniclastic wedge in the central part of the Sciara del Fuoco. Between 2017 and 2019, minor changes were again detected due to small accumulation next to the crater terrace and the erosion in lower Sciara del Fuoco.Publishedid 4385V. Processi eruttivi e post-eruttiviJCR Journa

A sensitive and specific neural signature for picture-induced negative affect

Neuroimaging has identified many correlates of emotion but has not yet yielded brain representations predictive of the intensity of emotional experiences in individuals. We used machine learning to identify a sensitive and specific signature of emotional responses to aversive images. This signature predicted the intensity of negative emotion in individual participants in cross validation (n =121) and test (n = 61) samples (high–low emotion = 93.5% accuracy). It was unresponsive to physical pain (emotion–pain = 92% discriminative accuracy), demonstrating that it is not a representation of generalized arousal or salience. The signature was comprised of mesoscale patterns spanning multiple cortical and subcortical systems, with no single system necessary or sufficient for predicting experience. Furthermore, it was not reducible to activity in traditional “emotion-related” regions (e.g., amygdala, insula) or resting-state networks (e.g., “salience,” “default mode”). Overall, this work identifies differentiable neural components of negative emotion and pain, providing a basis for new, brain-based taxonomies of affective processes

Co-registered photoacoustic and ultrasound tomographic imaging of human colorectal and ovarian cancer: light delivery, system development, and clinical study

Ovarian cancer remains the deadliest of all the gynecological malignancies. Conventional screening tests, including pelvic examination, transvaginal ultrasound (TVUS), and blood testing for cancer antigen 125 (CA-125), lack sufficient specificity for early ovarian cancer diagnosis. Imaging modalities such as computed tomography (CT), positron emission tomography (PET), and magnetic resonance imaging (MRI) have been used for surgical guidance. However, all of these modalities have limitations in detecting small lesions. Globally, colorectal cancer is the second most commonly diagnosed malignancy and the fourth most common cause of cancer mortality. Accurate staging and post-treatment surveillance of this prevalent disease are critical because treatment strategies are predicated upon the stage at presentation and the patient’s response to pre-surgical therapy – in some instances, detailed imaging allows certain patients to avoid surgery altogether. While colonoscopy and biopsy are the gold-standard diagnostic tests for colorectal cancers, multiple imaging modalities are also utilized, including optical imaging, endoscopic ultrasound (EUS), pelvic magnetic resonance imaging (MRI), computed tomography (CT), and positron emission tomography (PET). Unfortunately, each of these modalities has critical weakness in evaluating colorectal tumors. In staging colorectal tumors and evaluating their therapeutic response, more precise imaging modalities could transform the standard of care. This dissertation explores co-registered photoacoustic and ultrasound tomographic imaging of two fatal cancers: ovarian cancer and colorectal cancer. It is composed of three main parts: light delivery optimization/fiber optics, system development and optimization, and pilot patient and sample study. To begin, we designed, optimized, and evaluated an hand-held photoacoustic and ultrasound probe suitable for endo-cavity subsurface tumor imaging. Compared to previous designs, the prototype probe, consisting of four 1 mm multi-mode optical fibers attached to 1.5 mm diameter ball-shaped fiber tips sandwiched between a transvaginal ultrasound transducer and a custom-made sheath, demonstrated a higher light output and better beam homogeneity on the tissue subsurface. Second, we developed a novel fiber diffuser tip using microspheres dispersed within an ultraviolet (UV) adhesive to scatter light. This diffuser keeps the skin surface fluence under the maximum permissible exposure (MPE), while enabling higher laser energy injection to enhance the photoacoustic (PA) signal generated from the tissue. Third, we proposed an improved beamformer, named lag-based delay multiply and sum combined with coherence factor (DMAS-LAG-CF). Simulations and phantom experiments demonstrate that compared with conventional delay and sum (DAS), the proposed algorithm can provide 1.39 times better resolution and 10.77 dB higher contrast. For patient data, similar improvements in contrast ratios have been observed. However, since diagnostic accuracy in distinguishing between cancer and benign/normal groups is the significant measure, we have extracted the photoacoustic histogram features of mean, kurtosis, and skewness. When mean and skewness are used as features, DMAS-LAG-CF can improve cancer diagnosis, with an AUC of 0.91 in differentiating malignant from benign ovarian lesions. Fourth, to investigate the ability of co-registered photoacoustic and ultrasound tomographic imaging to assess human colorectal cancer, we conducted a pilot study on 23 ex-vivo human colorectal tissue samples that were collected immediately after surgical resection. Co-registered photoacoustic images of malignancies showed significantly increased PAT signals compared to normal regions of the same sample. We found statistically significant differences between untreated colorectal tumors and normal tissues, based on the quantitative relative total hemoglobin concentration (rHbT) computed from four optical wavelengths, spectral features such as the mean spectral slope and 0.5 MHz intercept extracted from PAT and US spectral data, and image features such as the first and second order statistics along with the standard deviation of the mean radon transform of PAT images. Using either a logistic regression model or a support vector machine, the best set of parameters of rHbT and PAT intercept achieved AUC values of 0.97 and 0.95 for the training and testing data sets, respectively, in predicting histologically confirmed invasive carcinoma. One limitation of the current system is its poor image resolution (~ 250 μm axial resolution) limited by the commercial endo-cavity ultrasound transducer array (6 MHz central frequency, 80% bandwidth). For better image resolution in high frequency PAT/US imaging, we first decoded the pin configuration of a high-frequency transducer array (15 MHz central frequency, 9-18 MHz bandwidth) and adapted it to our home-made 128 channel ultrasound pulsing and receiving system (sampling rate: 40 MHz). To further improve the performance, we are building a 64-channel FPGA-based high frequency photoacoustic imaging system with a sampling rate of 80 MHz and signal-to-noise ratio (SNR) of 40 dB. For in-vivo patient study, this system will be integrated with an endo-rectal probe with a side-firing fiber tip. Fifth, we investigated the laser safety of photoacoustic imaging, in preparation for its use in clinical reproductive medicine. Using preimplantation mouse blastocyst stage embryos, we assessed potential DNA damage from photoacoustic laser exposure. Different embryo groups were exposed to either 5- or 10- minute 15-Hz laser doses (typical clinical doses), or a 1-minute 1-kHz laser dose (a significantly higher dose). We demonstrated that typical lasers and exposure times used for photoacoustic imaging do not induce increased cell death in mouse blastocysts. Sixth, we demonstrated a novel fiber endface photoacoustic generator using IR 144 laser dye dispersed within an ultraviolet (UV) adhesive. The generator provides wide acoustic bandwidth in the transducer frequency range of 2-7 MHz, high thermal conversion efficiency (\u3e 90%), good PA intensity controllability (via the easily controlled IR 144 concentration), and high feasibility (simple procedures). Through a series of experimental validations, we show this fiber-based endface photoacoustic generator can be a useful tool for a broad range of biomedical applications, such as calibrating the local absorption coefficient of biological tissue for quantitative photoacoustic tomography

NASA/JPL Aircraft SAR Workshop Proceedings

Speaker-supplied summaries of the talks given at the NASA/JPL Aircraft SAR Workshop on February 4 and 5, 1985, are provided. These talks dealt mostly with composite quadpolarization imagery from a geologic or ecologic prespective. An overview and summary of the system characteristics of the L-band synthetic aperture radar (SAR) flown on the NASA CV-990 aircraft are included as supplementary information. Other topics ranging from phase imagery and interferometric techniques classifications of specific areas, and the potentials and limitations of SAR imagery in various applications are discussed

Martian impact craters: Continuing analysis of lobate ejecta sinuosity

The lobate ejecta morphology surrounding most fresh Martian impact craters can be quantitatively analyzed to determine variations in ejecta sinuosity with diameter, latitude, longitude, and terrain. The results of such studies provide another clue to the question of how these morphologies formed: are they the results of vaporization of subsurface volatiles or caused by ejecta entrainment in atmospheric gases. Kargel provided a simple expression to determine the degree of non-circularity of an ejecta blanket. This measure of sinuosity, called 'lobateness', is given by the ratio of the ejecta perimeter to the perimeter of a circle with the same area as that of the ejecta. The Kargel study of 538 rampart craters in selected areas of Mars led to the suggestion that lobateness increased with increasing diameter, decreased at higher latitude, and showed no dependence on elevation or geologic unit. Major problems with the Kargel analysis are the limited size and distribution of the data set and the lack of discrimination among the different types of lobate ejecta morphologies. Bridges and Barlow undertook a new lobateness study of 1582 single lobe (SL) and 251 double lobe (DL) craters. The results are summarized. These results agree with the finding of Kargel that lobateness increases with increasing diameter, but found no indication of a latitude dependence for SL craters. The Bridges and Barlow study has now been extended to multiple lobe (ML) craters. Three hundred and eighty ML craters located across the entire Martian surface were studied. ML craters provide more complications to lobateness studies than do SL and DL craters - in particular, the ejecta lobes surrounding the crater are often incomplete. Since the lobateness formula compares the perimeter of the ejecta lobe to that of a circle, the analysis was restricted only to complete lobes. The lobes are defined sequentially starting with the outermost lobe and moving inward

Preliminary test on structural elements health monitoring with a LiDAR-based approach

The safety and usability of infrastructures such as bridges, roads, and buildings must be monitored throughout their useful life. Traditional inspection methods are time-consuming and expensive, and innovative solutions using LiDAR-based techniques have developed. This study presents a semi-automatic method for detecting deteriorations on structural elements of a bridge using an integrated dataset of point clouds and radiometric information. The method involves using a Terrestrial Laser Scanner (TLS) to obtain high-resolution georeferenced point clouds of the bridge beams, which are then filtered to identify four classes of deteriorations. Six Machine Learning Classifiers are tested and compared using Overall Accuracy and F1-score metrics. The Random Forest emerged as the best-performing. It was then optimised by reducing the input features through an importance analysis and the accuracies measured. The results show promise and can be explored further on a larger dataset. The study aims to generalise the methodology to transfer it to actual cases

Resolving the brainstem contributions to attentional analgesia

Previous human imaging studies manipulating attention or expectancy have identified the periaqueductal gray (PAG) as a key brainstem structure implicated in endogenous analgesia. However, animal studies indicate that PAG analgesia is mediated largely via caudal brainstem structures, such as the rostral ventromedial medulla (RVM) and locus coeruleus (LC). To identify their involvement in endogenous analgesia, we used brainstem optimized, whole-brain imaging to record responses to concurrent thermal stimulation (left forearm) and visual attention tasks of titrated difficulty in 20 healthy subjects. The PAG, LC, and RVM were anatomically discriminated using a probabilistic atlas. Pain ratings disclosed the anticipated analgesic interaction between task difficulty and pain intensity (p < 0.001). Main effects of noxious thermal stimulation were observed across several brain regions, including operculoinsular, primary somatosensory, and cingulate cortices, whereas hard task difficulty was represented in anterior insular, parietal, and prefrontal cortices. Permutation testing within the brainstem nuclei revealed the following: main effects of task in dorsal PAG and right LC; and main effect of temperature in RVM and a task × temperature interaction in right LC. Intrasubject regression revealed a distributed network of supratentorial brain regions and the RVM whose activity was linearly related to pain intensity. Intersubject analgesia scores correlated to activity within a distinct region of the RVM alone. These results identify distinct roles for a brainstem triumvirate in attentional analgesia: with the PAG activated by attentional load; specific RVM regions showing pronociceptive and antinociceptive processes (in line with previous animal studies); and the LC showing lateralized activity during conflicting attentional demands. SIGNIFICANCE STATEMENT Attention modulates pain intensity, and human studies have identified roles for a network of forebrain structures plus the periaqueductal gray (PAG). Animal data indicate that the PAG acts via caudal brainstem structures to control nociception. We investigated this issue within an attentional analgesia paradigm with brainstem-optimized fMRI and analysis using a probabilistic brainstem atlas. We find pain intensity encoding in several forebrain structures, including the insula and attentional activation of the PAG. Discrete regions of the rostral ventromedial medulla bidirectionally influence pain perception, and locus coeruleus activity mirrors the interaction between attention and nociception. This approach has enabled the resolution of contributions from a hub of key brainstem structures to endogenous analgesia

Low-cost, High-contrast, and Miniature Optical Imaging Systems for Clinical Applications

Reducing the cost and size and enhancing the contrast of optical imaging systems improve their potentials for clinical applications. In this dissertation, we describe our endeavors towards development of low-cost and compact photoacoustic microscopy and spatial frequency domain imaging systems as well as improvement of photoacoustic tumor imaging using a specifically designed photoacoustic contrast agent. Chapters two and three focus on the development of low-cost and compact laser diode based photoacoustic microscopy systems. We first provided an improvement in light delivery of laser diode based photoacoustic microscopy systems that enables imaging biological tissue with high signal to noise ratio. We then developed a laser scanning laser diode based photoacoustic microscopy system that provides substantial improvement of imaging speed and eliminates the need for mechanical scanning of the sample, hence improving the potentials of low-cost and compact laser diode based photoacoustic microscopy for clinical applications. Chapter four describes synthesis and evaluation of a monomeric porphyrin-based photoacoustic contrast agent for improvement of in vivo tumor imaging. Absorption in near infrared wavelength range, solubility, stability, nontoxicity, and high photoacoustic generation efficiency of the dye were demonstrated. The contrast agent was evaluated for enhancing the photoacoustic images of implanted murine tumors revealing a multi-fold stronger enhancement and a slower washout compared to the benchmark FDA approved indocyanine green (ICG) dye. Favorable filtration and tumor accumulation of the dye further demonstrated its potential as a photoacoustic contrast agent for in vivo tumor imaging. Finally, chapter 5 describes development of a very low-cost, handheld, and multispectral spatial frequency domain imaging system that incorporates nine different light emitting diodes and all illumination and detection optical components in a small 3D-printed probe. The system performance was evaluated on biological tissue to assess its potentials