62 research outputs found
Direction-Aware Adaptive Online Neural Speech Enhancement with an Augmented Reality Headset in Real Noisy Conversational Environments
This paper describes the practical response- and performance-aware
development of online speech enhancement for an augmented reality (AR) headset
that helps a user understand conversations made in real noisy echoic
environments (e.g., cocktail party). One may use a state-of-the-art blind
source separation method called fast multichannel nonnegative matrix
factorization (FastMNMF) that works well in various environments thanks to its
unsupervised nature. Its heavy computational cost, however, prevents its
application to real-time processing. In contrast, a supervised beamforming
method that uses a deep neural network (DNN) for estimating spatial information
of speech and noise readily fits real-time processing, but suffers from drastic
performance degradation in mismatched conditions. Given such complementary
characteristics, we propose a dual-process robust online speech enhancement
method based on DNN-based beamforming with FastMNMF-guided adaptation. FastMNMF
(back end) is performed in a mini-batch style and the noisy and enhanced speech
pairs are used together with the original parallel training data for updating
the direction-aware DNN (front end) with backpropagation at a
computationally-allowable interval. This method is used with a blind
dereverberation method called weighted prediction error (WPE) for transcribing
the noisy reverberant speech of a speaker, which can be detected from video or
selected by a user's hand gesture or eye gaze, in a streaming manner and
spatially showing the transcriptions with an AR technique. Our experiment
showed that the word error rate was improved by more than 10 points with the
run-time adaptation using only twelve minutes of observation.Comment: IEEE/RSJ IROS 202
Immersive analytics for oncology patient cohorts
This thesis proposes a novel interactive immersive analytics tool and methods to interrogate the cancer patient cohort in an immersive virtual environment, namely Virtual Reality to Observe Oncology data Models (VROOM). The overall objective is to develop an immersive analytics platform, which includes a data analytics pipeline from raw gene expression data to immersive visualisation on virtual and augmented reality platforms utilising a game engine. Unity3D has been used to implement the visualisation. Work in this thesis could provide oncologists and clinicians with an interactive visualisation and visual analytics platform that helps them to drive their analysis in treatment efficacy and achieve the goal of evidence-based personalised medicine. The thesis integrates the latest discovery and development in cancer patients’ prognoses, immersive technologies, machine learning, decision support system and interactive visualisation to form an immersive analytics platform of complex genomic data. For this thesis, the experimental paradigm that will be followed is in understanding transcriptomics in cancer samples. This thesis specifically investigates gene expression data to determine the biological similarity revealed by the patient's tumour samples' transcriptomic profiles revealing the active genes in different patients. In summary, the thesis contributes to i) a novel immersive analytics platform for patient cohort data interrogation in similarity space where the similarity space is based on the patient's biological and genomic similarity; ii) an effective immersive environment optimisation design based on the usability study of exocentric and egocentric visualisation, audio and sound design optimisation; iii) an integration of trusted and familiar 2D biomedical visual analytics methods into the immersive environment; iv) novel use of the game theory as the decision-making system engine to help the analytics process, and application of the optimal transport theory in missing data imputation to ensure the preservation of data distribution; and v) case studies to showcase the real-world application of the visualisation and its effectiveness
Deep Hyperspectral and Multispectral Image Fusion with Inter-image Variability
Hyperspectral and multispectral image fusion allows us to overcome the
hardware limitations of hyperspectral imaging systems inherent to their lower
spatial resolution. Nevertheless, existing algorithms usually fail to consider
realistic image acquisition conditions. This paper presents a general imaging
model that considers inter-image variability of data from heterogeneous sources
and flexible image priors. The fusion problem is stated as an optimization
problem in the maximum a posteriori framework. We introduce an original image
fusion method that, on the one hand, solves the optimization problem accounting
for inter-image variability with an iteratively reweighted scheme and, on the
other hand, that leverages light-weight CNN-based networks to learn realistic
image priors from data. In addition, we propose a zero-shot strategy to
directly learn the image-specific prior of the latent images in an unsupervised
manner. The performance of the algorithm is illustrated with real data subject
to inter-image variability.Comment: IEEE Trans. Geosci. Remote sens., to be published. Manuscript
submitted August 23, 2022; revised Dec. 15, 2022, and Mar. 13, 2023; and
accepted Apr. 07, 202
Data Science
International audienceLa data science, ou science des données, est la discipline qui traite de la collecte, de la préparation, de la gestion, de l'analyse, de l'interprétation et de la visualisation de grands ensembles de données complexes. Elle n'est pas seulement concernée par les outils et les méthodes pour obtenir, gérer et analyser les données ; elle consiste aussi à en extraire de la valeur et de la connaissance. Cet ouvrage présente les fondements scientifiques et les composantes essentielles de la science des données, à un niveau accessible aux étudiants de master et aux élèves ingénieurs. Notre souci a été de proposer un exposé cohérent reliant la théorie aux algorithmes développés dans ces domaines. Il s'adresse aux chercheurs et ingénieurs qui abordent les problématiques liées à la science des données, aux data scientists de PME qui utilisent en profondeur les outils d'apprentissage, mais aussi aux étudiants de master, doctorants ou encore futurs ingénieurs qui souhaitent un ouvrage de référence en data science. À qui s'adresse ce livre ? • Aux développeurs, statisticiens, étudiants et chefs de projets ayant à résoudre des problèmes de data science. • Aux data scientists, mais aussi à toute personne curieuse d'avoir une vue d'ensemble de l'état de l'art du machine learning
Geometry- and Accuracy-Preserving Random Forest Proximities with Applications
Many machine learning algorithms use calculated distances or similarities between data observations to make predictions, cluster similar data, visualize patterns, or generally explore the data. Most distances or similarity measures do not incorporate known data labels and are thus considered unsupervised. Supervised methods for measuring distance exist which incorporate data labels and thereby exaggerate separation between data points of different classes. This approach tends to distort the natural structure of the data. Instead of following similar approaches, we leverage a popular algorithm used for making data-driven predictions, known as random forests, to naturally incorporate data labels into similarity measures known as random forest proximities. In this dissertation, we explore previously defined random forest proximities and demonstrate their weaknesses in popular proximity-based applications. Additionally, we develop a new proximity definition that can be used to recreate the random forest’s predictions. We call these random forest-geometry-and accuracy-Preserving proximities or RF-GAP. We show by proof and empirical demonstration can be used to perfectly reconstruct the random forest’s predictions and, as a result, we argue that RF-GAP proximities provide a truer representation of the random forest’s learning when used in proximity-based applications. We provide evidence to suggest that RF-GAP proximities improve applications including imputing missing data, detecting outliers, and visualizing the data. We also introduce a new random forest proximity-based technique that can be used to generate 2- or 3-dimensional data representations which can be used as a tool to visually explore the data. We show that this method does well at portraying the relationship between data variables and the data labels. We show quantitatively and qualitatively that this method surpasses other existing methods for this task
Hyperspectral and Multispectral Image Fusion Using the Conditional Denoising Diffusion Probabilistic Model
Hyperspectral images (HSI) have a large amount of spectral information
reflecting the characteristics of matter, while their spatial resolution is low
due to the limitations of imaging technology. Complementary to this are
multispectral images (MSI), e.g., RGB images, with high spatial resolution but
insufficient spectral bands. Hyperspectral and multispectral image fusion is a
technique for acquiring ideal images that have both high spatial and high
spectral resolution cost-effectively. Many existing HSI and MSI fusion
algorithms rely on known imaging degradation models, which are often not
available in practice. In this paper, we propose a deep fusion method based on
the conditional denoising diffusion probabilistic model, called DDPM-Fus.
Specifically, the DDPM-Fus contains the forward diffusion process which
gradually adds Gaussian noise to the high spatial resolution HSI (HrHSI) and
another reverse denoising process which learns to predict the desired HrHSI
from its noisy version conditioning on the corresponding high spatial
resolution MSI (HrMSI) and low spatial resolution HSI (LrHSI). Once the
training is completes, the proposed DDPM-Fus implements the reverse process on
the test HrMSI and LrHSI to generate the fused HrHSI. Experiments conducted on
one indoor and two remote sensing datasets show the superiority of the proposed
model when compared with other advanced deep learningbased fusion methods. The
codes of this work will be opensourced at this address:
https://github.com/shuaikaishi/DDPMFus for reproducibility
Literature Explorer: effective retrieval of scientific documents through nonparametric thematic topic detection
© 2020 The Authors. Published by Springer. This is an open access article available under a Creative Commons licence.
The published version can be accessed at the following link on the publisher’s website: https://doi.org/10.1007/s00371-019-01721-7Scientific researchers are facing a rapidly growing volume of literatures nowadays. While these publications offer rich and valuable information, the scale of the datasets makes it difficult for the researchers to manage and search for desired information efficiently. Literature Explorer is a new interactive visual analytics suite that facilitates the access to desired scientific literatures through mining and interactive visualisation. We propose a novel topic mining method that is able to uncover “thematic topics” from a scientific corpus. These thematic topics have an explicit semantic association to the research themes that are commonly used by human researchers in scientific fields, and hence are human interpretable. They also contribute to effective document retrieval. The visual analytics suite consists of a set of visual components that are closely coupled with the underlying thematic topic detection to support interactive document retrieval. The visual components are adequately integrated under the design rationale and goals. Evaluation results are given in both objective measurements and subjective terms through expert assessments. Comparisons are also made against the outcomes from the traditional topic modelling methods.This research is supported by the European Commission with project Dr Inventor (No 611383), MyHealthAvatar (No 60929), and by the UK Engineering and Physical Sciences Research Council with project MyLifeHub (EP/L023830/1).Published onlin
- …