Exploring information retrieval using image sparse representations:from circuit designs and acquisition processes to specific reconstruction algorithms

New advances in the field of image sensors (especially in CMOS technology) tend to question the conventional methods used to acquire the image. Compressive Sensing (CS) plays a major role in this, especially to unclog the Analog to Digital Converters which are generally representing the bottleneck of this type of sensors. In addition, CS eliminates traditional compression processing stages that are performed by embedded digital signal processors dedicated to this purpose. The interest is twofold because it allows both to consistently reduce the amount of data to be converted but also to suppress digital processing performed out of the sensor chip. For the moment, regarding the use of CS in image sensors, the main route of exploration as well as the intended applications aims at reducing power consumption related to these components (i.e. ADC & DSP represent 99% of the total power consumption). More broadly, the paradigm of CS allows to question or at least to extend the Nyquist-Shannon sampling theory. This thesis shows developments in the field of image sensors demonstrating that is possible to consider alternative applications linked to CS. Indeed, advances are presented in the fields of hyperspectral imaging, super-resolution, high dynamic range, high speed and non-uniform sampling. In particular, three research axes have been deepened, aiming to design proper architectures and acquisition processes with their associated reconstruction techniques taking advantage of image sparse representations. How the on-chip implementation of Compressed Sensing can relax sensor constraints, improving the acquisition characteristics (speed, dynamic range, power consumption) ? How CS can be combined with simple analysis to provide useful image features for high level applications (adding semantic information) and improve the reconstructed image quality at a certain compression ratio ? Finally, how CS can improve physical limitations (i.e. spectral sensitivity and pixel pitch) of imaging systems without a major impact neither on the sensing strategy nor on the optical elements involved ? A CMOS image sensor has been developed and manufactured during this Ph.D. to validate concepts such as the High Dynamic Range - CS. A new design approach was employed resulting in innovative solutions for pixels addressing and conversion to perform specific acquisition in a compressed mode. On the other hand, the principle of adaptive CS combined with the non-uniform sampling has been developed. Possible implementations of this type of acquisition are proposed. Finally, preliminary works are exhibited on the use of Liquid Crystal Devices to allow hyperspectral imaging combined with spatial super-resolution. The conclusion of this study can be summarized as follows: CS must now be considered as a toolbox for defining more easily compromises between the different characteristics of the sensors: integration time, converters speed, dynamic range, resolution and digital processing resources. However, if CS relaxes some material constraints at the sensor level, it is possible that the collected data are difficult to interpret and process at the decoder side, involving massive computational resources compared to so-called conventional techniques. The application field is wide, implying that for a targeted application, an accurate characterization of the constraints concerning both the sensor (encoder), but also the decoder need to be defined

Steady-state anatomical and quantitative magnetic resonance imaging of the heart using RF-frequencymodulated techniques

Cardiovascular disease (CVD) is the leading cause of death in the United States and Europe and generates healthcare costs of hundreds of billions of dollars annually. Conventional methods of diagnosing CVD are often invasive and carry risks for the patient. For example, the gold standard for diagnosing coronary artery disease, a major class of CVD, is x-ray coronary angiography, which has the disadvantages of being invasive, being expensive, using ionizing radiation, and having a ris k of complications. Conversely, coronary MR angiography (MRA) does not use ionizing radiation, can effectively visualize tissues without the need for exogenous contrast agents, and benefits from an adaptable temporal resolution. However, the acquisition time of cardiac MRI is far longer than the temporal scales of cardiac and respiratory motion, necessitating some method of compensating for this motion. The free-running framework is a novel development in our lab, benefitting from advances over the past three decades, that attempts to address disadvantages of previous cardiac MRI approaches: it provides fully self-gated 5D cardiac MRI with a simplified workflow, improved ease-of-use, reduced operator dependence, and automatic patient-specific motion detection. Free-running imaging increases the amount of information available to the clinician and is flexible enough to be translated to different app lications within cardiac MRI. Moreover, the self-gating of the free-running framework decoupled the acquisition from the motion compensation and thereby opened up cardiac MRI to the wider class of steady-state-based techniques utilizing balanced steady-state free precession (bSSFP) sequences, which have the benefits of practical simplicity and high signal-to-noise ratio. The focus of this thesis was therefore on the application of steady- state techniques to cardiac MRI. The first part addressed the long acquisition time of the current free-running framework and focused on anatomical coronary imaging. The published protocol of the free- running framework used an interrupted bSSFP acquisition where CHESS fat saturation modules were inserted to provide blood-fat contrast, as they suppress the signal of fat tissue surrounding the coronary arteries, and were followed by ramp-up pulses to reduce artefacts arising from the return to steady-state. This interrupted acquisition, however, suffered from an interrupted steady-state, reduced time efficiency, and higher specific absorption rate (SAR). Using novel lipid-insensitive binomial off-resonant RF excitation (LIBRE) pulses developed in our lab, the first project showed that LIBRE pulses incorporated into an uninterrupted free-running bSSFP sequence could be successfully used for 5D cardiac MRI at 1.5T. The free-running LIBRE approach reduced the acquisition time and SAR relative to the previous interrupted approach while maintaining image quality and vessel conspicuity. Furthermore, this had been the first successful use of a fat-suppressing RF excitation pulse in an uninterrupted bSSFP sequence for cardiac imaging, demonstrating that uninterrupted bSSFP can be used for cardiac MRI and addressing the problem of clinical sequence availability. Inspired by the feasibility of uninterrupted bSSFP for cardiac MRI, the second part investigated the potential of PLANET, a novel 3D multiparametric mapping technique, for free-running 5D myocardial mapping. PLANET utilizes a phase-cycled bSSFP acquisition and a direct ellipse-fitting algorithm to calculate T1 and T2 relaxation times, which suggested that it could be readily integrated into the free-running framework without interrupting the steady-state. After initially calibrating the acquisition, the possibility of accelerating the static PLANET acquisition was explored prior to applying it to the moving heart. It was shown that PLANET accuracy and precision could be maintained with two-fold acceleration with a 3D Cartesian spiral trajectory, suggesting that PLANET for myocardial mapping with the free-running 5D radial acquisition is feasible. Further work should investigate optimizing the reconstruction scheme, improving the coil sensitivity estimate, and examining the use of the radial trajectory with a view to implementing free-running 5D myocardial T1 and T2 mapping. This thesis presents two approaches utilizing RF-frequency-modulated steady-state techniques for cardiac MRI. The first approach involved the novel application of an uninterrupted bSSFP acquisition with off-resonant RF excitation for anatomical coronary imaging. The second approach investigated the use of phase-cycled bSSFP for free-running 5D myocardial T1 and T2 mapping. Both methods addressed the challenge of clinical availability of sequences in cardiac MRI, by showing that a common and simple sequence like bSSFP can be used for acquisition while the steps of motion compensation and reconstruction can be handled offline, and thus have the potential to improve adoption of cardiac MRI. -- Les maladies cardiovasculaires (MCV) représentent la principale cause de décès aux États-Unis et en Europe et génèrent des coûts de santé de plusieurs centaines de milliards de dollars par an. Les méthodes conventionnelles de diagnostic des MCV sont souvent invasives et comportent des risques pour le patient. Par exemple, la méthode de référence pour le diagnostic de la maladie coronarienne, une catégorie majeure de MCV, est la coronarographie par rayons X qui a comme inconvénients son caractère invasif, son coût, l’utilisation de rayonnements ionisants et le risque de complications. A l’inverse, l'angiographie coronarienne par résonance magnétique (ARM) n'utilise pas de rayonnements ionisants, permet de visualiser efficacement les tissus sans avoir recours à des agents de contraste exogènes et bénéficie d'une résolution temporelle ajustable. Cependant, le temps d'acquisition en IRM cardiaque est bien plus long que les échelles temporelles des mouvements cardiaques et respiratoires en jeu, ce qui rend la compensation de ces mouvements indispensable. Le cadre dit de « free -running » est un nouveau développement de notre laboratoire qui bénéficie des progrès réalisés au cours des trois dernières décennies et tente de remédier aux inconvénients des approches précédentes pour l'IRM cardiaque : il fournit une IRM cardiaque en cinq dimensions (5D) complètement « self-gated » , c’est-à-dire capable de détecter les mouvements cardiaques et respiratoires, forte d’une implémentation simplifiée, d’une plus grande facilité d'utilisation, d’une dépendance réduite vis-à-vis de l'opérateur et d’une détection automatique des mouvements spécifiques du patient. L'imagerie « free- running » augmente la quantité d'informations à disposition du clinicien et est suffisamment flexible pour être appliquée à différents domaines de l'IRM cardiaque. De plus, le « self-gating » du cadre « free-running » a découplé l'acquisition de la compensation de mouvement et a ainsi ouvert l'IRM cardiaque à la classe plus large des techniques basées sur l'état stationnaire utilisant des séquences de précession libre équilibrée en état stationnaire (bSSFP), qui se distinguent par leur simplicité d’utilisation et leur rapport signal sur bruit élevé. Le thème de cette thèse est donc l'application des techniques basées sur l'état stationnaire à l'IRM cardiaque. La première partie porte sur le long temps d'acquisition de l'actuel cadre « free-running» et se concentre sur l'imagerie anatomique coronaire. Le protocole publié utilise une acquisition bSSFP interrompue où des modules de saturation de graisse (CHESS) sont insérés de façon à fournir un contraste sang-graisse puisqu’ils suppriment le signal du tissu graisseux entourant les artères coronaires, et sont suivis par des impulsions en rampe pour réduire les artefacts résultant du retour à l'état stable. Cette acquisition interrompue souffre cependant d'un état d'équilibre interrompu, d'une efficacité temporelle réduite et d'un débit d'absorption spécifique (DAS) plus élevé. En utilisant les nouvelles impulsions d'excitation radiofréquence (RF) binomiales hors -résonance insensibles aux lipides (LIBRE) développées dans notre laboratoi re, ce premier projet montre que les impulsions LIBRE incorporées dans une séquence bSSFP ininterrompue et « free-running » peuvent être utilisées avec succès pour l'IRM cardiaque 5D à 1,5 T. L'approche « free-running LIBRE » permet de réduire le temps d'acquisition et le DAS par rapport à l'approche interrompue précédente, tout en maintenant la perceptibilité des artères coronariennes. En outre, il s'agit de la première utilisation réussie d'une impulsion d'excitation RF supprimant la graisse dans une séquence bSSFP ininterrompue pour l'imagerie cardiaque, ce qui démontre le potentiel d’utilisation de la séquence bSSFP ininterrompue pour l'IRM cardiaque et résout le problème de la disponibilité de la séquence en clinique. Inspirée par la faisabilité d’utilisation de la séquence bSSFP ininterrompue pour l'IRM cardiaque, la deuxième partie étudie le potentiel de PLANET, une nouvelle technique de cartographie 3D multiparamétrique, pour la cartographie 5D du myocarde via l’imagerie « free-running ». PLANET utilise une acquisition bSSFP à cycle de phase et un algorithme d'ajustement d'ellipse direct pour calculer les temps de relaxation T1 et T2, ce qui suggère que cette méthode pourrait être facilement intégrée au cadre « free - running » sans interruption de l’état d'équilibre. Après calibration de l'acquisition, nous explorons la possibilité d'accélérer l'acquisition statique de PLANET pour l'appliquer au cœur. Nous démontrons que l'exactitude et la précision de PLANET peuvent être maintenues pour une accélération double avec une trajectoire 3D cartésienne en spirale, ce qui suggère que PLANET est réalisable pour la cartographie du myocarde avec une acquisition radiale 5D « free-running ». D'autres travaux devraient porter sur l'optimisation du schéma de reconstruction, l'amélioration de l'estimation de la sensibilité de l’antenne et l'examen de l'utilisation de la trajectoire radiale en vue de la mise en œuvre de la cartographie 5D « free-running » T1 et T2 du myocarde. Cette thèse présente deux approches utilisant des techniques de modulation de fréquence radio en état stationnaire pour l'IRM cardiaque. La première approche implique l'application nouvelle d'une acquisition bSSFP ininterrompue avec une excitation RF hors résonance pour l'imagerie anatomique coronaire. La seconde approche porte sur l'utilisation d’une séquence bSSFP à cycle de phase pour la cartographie 5D T1 et T2 du myocarde. Ces deux méthodes permettent de répondre au défi posé par la disponibilité des séquences en IRM cardiaque en montrant qu'une séquence commune et simple comme la bSSFP peut être utilisée pour l'acquisition, tandis que les étapes de compensation du mouvement et de reconstruction peuvent être traitées hors ligne. Ainsi, ces méthodes ont le potentiel de favoriser l'adoption de l'IRM cardiaque

Roadmap on signal processing for next generation measurement systems

Signal processing is a fundamental component of almost any sensor-enabled system, with a wide range of applications across different scientific disciplines. Time series data, images, and video sequences comprise representative forms of signals that can be enhanced and analysed for information extraction and quantification. The recent advances in artificial intelligence and machine learning are shifting the research attention towards intelligent, data-driven, signal processing. This roadmap presents a critical overview of the state-of-the-art methods and applications aiming to highlight future challenges and research opportunities towards next generation measurement systems. It covers a broad spectrum of topics ranging from basic to industrial research, organized in concise thematic sections that reflect the trends and the impacts of current and future developments per research field. Furthermore, it offers guidance to researchers and funding agencies in identifying new prospects.AerodynamicsMicrowave Sensing, Signals & System

Improving Maternal and Fetal Cardiac Monitoring Using Artificial Intelligence

Early diagnosis of possible risks in the physiological status of fetus and mother during pregnancy and delivery is critical and can reduce mortality and morbidity. For example, early detection of life-threatening congenital heart disease may increase survival rate and reduce morbidity while allowing parents to make informed decisions. To study cardiac function, a variety of signals are required to be collected. In practice, several heart monitoring methods, such as electrocardiogram (ECG) and photoplethysmography (PPG), are commonly performed. Although there are several methods for monitoring fetal and maternal health, research is currently underway to enhance the mobility, accuracy, automation, and noise resistance of these methods to be used extensively, even at home. Artificial Intelligence (AI) can help to design a precise and convenient monitoring system. To achieve the goals, the following objectives are defined in this research: The first step for a signal acquisition system is to obtain high-quality signals. As the first objective, a signal processing scheme is explored to improve the signal-to-noise ratio (SNR) of signals and extract the desired signal from a noisy one with negative SNR (i.e., power of noise is greater than signal). It is worth mentioning that ECG and PPG signals are sensitive to noise from a variety of sources, increasing the risk of misunderstanding and interfering with the diagnostic process. The noises typically arise from power line interference, white noise, electrode contact noise, muscle contraction, baseline wandering, instrument noise, motion artifacts, electrosurgical noise. Even a slight variation in the obtained ECG waveform can impair the understanding of the patient's heart condition and affect the treatment procedure. Recent solutions, such as adaptive and blind source separation (BSS) algorithms, still have drawbacks, such as the need for noise or desired signal model, tuning and calibration, and inefficiency when dealing with excessively noisy signals. Therefore, the final goal of this step is to develop a robust algorithm that can estimate noise, even when SNR is negative, using the BSS method and remove it based on an adaptive filter. The second objective is defined for monitoring maternal and fetal ECG. Previous methods that were non-invasive used maternal abdominal ECG (MECG) for extracting fetal ECG (FECG). These methods need to be calibrated to generalize well. In other words, for each new subject, a calibration with a trustable device is required, which makes it difficult and time-consuming. The calibration is also susceptible to errors. We explore deep learning (DL) models for domain mapping, such as Cycle-Consistent Adversarial Networks, to map MECG to fetal ECG (FECG) and vice versa. The advantages of the proposed DL method over state-of-the-art approaches, such as adaptive filters or blind source separation, are that the proposed method is generalized well on unseen subjects. Moreover, it does not need calibration and is not sensitive to the heart rate variability of mother and fetal; it can also handle low signal-to-noise ratio (SNR) conditions. Thirdly, AI-based system that can measure continuous systolic blood pressure (SBP) and diastolic blood pressure (DBP) with minimum electrode requirements is explored. The most common method of measuring blood pressure is using cuff-based equipment, which cannot monitor blood pressure continuously, requires calibration, and is difficult to use. Other solutions use a synchronized ECG and PPG combination, which is still inconvenient and challenging to synchronize. The proposed method overcomes those issues and only uses PPG signal, comparing to other solutions. Using only PPG for blood pressure is more convenient since it is only one electrode on the finger where its acquisition is more resilient against error due to movement. The fourth objective is to detect anomalies on FECG data. The requirement of thousands of manually annotated samples is a concern for state-of-the-art detection systems, especially for fetal ECG (FECG), where there are few publicly available FECG datasets annotated for each FECG beat. Therefore, we will utilize active learning and transfer-learning concept to train a FECG anomaly detection system with the least training samples and high accuracy. In this part, a model is trained for detecting ECG anomalies in adults. Later this model is trained to detect anomalies on FECG. We only select more influential samples from the training set for training, which leads to training with the least effort. Because of physician shortages and rural geography, pregnant women's ability to get prenatal care might be improved through remote monitoring, especially when access to prenatal care is limited. Increased compliance with prenatal treatment and linked care amongst various providers are two possible benefits of remote monitoring. If recorded signals are transmitted correctly, maternal and fetal remote monitoring can be effective. Therefore, the last objective is to design a compression algorithm that can compress signals (like ECG) with a higher ratio than state-of-the-art and perform decompression fast without distortion. The proposed compression is fast thanks to the time domain B-Spline approach, and compressed data can be used for visualization and monitoring without decompression owing to the B-spline properties. Moreover, the stochastic optimization is designed to retain the signal quality and does not distort signal for diagnosis purposes while having a high compression ratio. In summary, components for creating an end-to-end system for day-to-day maternal and fetal cardiac monitoring can be envisioned as a mix of all tasks listed above. PPG and ECG recorded from the mother can be denoised using deconvolution strategy. Then, compression can be employed for transmitting signal. The trained CycleGAN model can be used for extracting FECG from MECG. Then, trained model using active transfer learning can detect anomaly on both MECG and FECG. Simultaneously, maternal BP is retrieved from the PPG signal. This information can be used for monitoring the cardiac status of mother and fetus, and also can be used for filling reports such as partogram