Visual to Sound: Generating Natural Sound for Videos in the Wild

As two of the five traditional human senses (sight, hearing, taste, smell, and touch), vision and sound are basic sources through which humans understand the world. Often correlated during natural events, these two modalities combine to jointly affect human perception. In this paper, we pose the task of generating sound given visual input. Such capabilities could help enable applications in virtual reality (generating sound for virtual scenes automatically) or provide additional accessibility to images or videos for people with visual impairments. As a first step in this direction, we apply learning-based methods to generate raw waveform samples given input video frames. We evaluate our models on a dataset of videos containing a variety of sounds (such as ambient sounds and sounds from people/animals). Our experiments show that the generated sounds are fairly realistic and have good temporal synchronization with the visual inputs.Comment: Project page: http://bvision11.cs.unc.edu/bigpen/yipin/visual2sound_webpage/visual2sound.htm

Sleep staging using contactless audio-based methods

Sleep stage classification is essential for evaluating sleep and its disorders. Most sleep studies make use of contact sensors which may interfere with natural sleep although recently the potential for sleep staging from audio signals has been acknowledged. This project presents a non-contact audio-based method for sleep staging. The objective of this work is to develop a method that can classify sleep stages from non-contact audio signals. To achieve the aforementioned objective a measurement acquisition setup has been presented alongside a validation of the acquired respiratory signal and a sleep staging algorithm. 11 subjects have been measured with the proposed method. The validation process compares the pre-processed acquired audio signal with a reference respiratory signal yielding good results in terms of error metrics, with a low deviation between the acquired respiratory cycles using the audio method and the reference method. The sleep stage algorithm classifies sixty-second epochs into NREM or REM stages with good results in terms of REM and NREM detection, with REM and NREM cycle duration similar to the ones that can be found in other studies present in the literature, thus validation the obtained results.La clasificación por fases del sueño es esencial para su evaluación y para la evaluación de sus trastornos. La mayoría de los estudios del sueño requieren del uso de sensores de contacto que podrían alterar la natura de este, aunque recientemente, se ha reconocido el potencial de otros métodos basados en señales de audio sin contacto. Este proyecto presenta un método de clasificación de las fases del sueño basado en señales de audio sin contacto. El objetivo del trabajo es desarrollar un método que permita clasificar las diferentes fases del sueño a partir de señales de audio sin contacto. Para alcanzar este objetivo se ha definido una configuración de medida junto a una validación de la señal respiratoria y un algoritmo de clasificación de las fases del sueño. Se han medido 11 sujetos usando la configuración de medida propuesta. Este proceso de validación compara la señal de audio con una señal de respiración de referencia, dando buenos resultados en términos de métrica de errores, con una baja desviación entre los ciclos respiratorios obtenidos mediante el método de audio propuesto y el método de referencia. El algoritmo de clasificación, clasifica en NREM y REM con buenos resultados en términos de detección de las fases, con una duración de ciclo REM y NREM similar a las que se pueden encontrar en otros estudios presentados en la literatura, validando así los resultados obtenidos.La classificació de les etapes de la son és essencial per la seva avaluació i la dels seus trastorns. La majoria dels estudis de la son fan ús de sensors de contacte que podrien interferir en la natura de la son, tot i que recentment, s'ha reconegut el potencial de mètodes de classificació de les etapes de la son basats en senyals d'àudio sense contacte. Aquest projecte presenta un mètode de classificació de les etapes de la son basat en senyals d'àudio sense contacte. L'objectiu d'aquest treball és desenvolupar un mètode que permeti classificar les diferents etapes de la son a partir de senyals d'àudio sense contacte. Per assolir aquest objectiu s'ha definit una configuració de mesura juntament amb una validació del senyal de respiració i un algorisme de classificació de les etapes de la son. S'han mesurat 11 subjectes utilitzant la configuració proposada. El procés de validació compara el senyal d'àudio capturat, una vegada preprocessat, amb un senyal de respiració com a referència, donant bons resultats en termes de mètriques d'error, amb una desviació baixa entre els cicles respiratoris obtinguts mitjançant el mètode d'àudio i el mètode de referència. L'algorisme de classificació de les etapes de la son, classifica trames de seixanta segons en REM o NREM amb bons resultats en termes de detecció REM o NREM, amb una durada de cicle REM i NREM similar a les que es poden trobar en altres estudis presents en la literatura, validant així els resultats obtinguts

Utility of AdaBoost to Detect Sleep Apnea-Hypopnea Syndrome From Single-Channel Airflow

Producción CientíficaThe purpose of this study is to evaluate the usefulness of the boosting algorithm AdaBoost (AB) in the context of the sleep apnea-hypopnea syndrome (SAHS) diagnosis. Methods: We characterize SAHS in single-channel airflow (AF) signals from 317 subjects by the extraction of spectral and non-linear features. Relevancy and redundancy analyses are conducted through the fast correlation-based filter (FCBF) to derive the optimum set of features among them. These are used to feed classifiers based on linear discriminant analysis (LDA) and classification and regression trees (CART). LDA and CART models are sequentially obtained through AB, which combines their performances to reach higher diagnostic ability than each of them separately. Results: Our AB-LDA and AB-CART approaches showed high diagnostic performance when determining SAHS and its severity. The assessment of different apnea-hypopnea index cutoffs using an independent test set derived into high accuracy: 86.5% (5 events/h), 86.5% (10 events/h), 81.0% (15 events/h), and 83.3% (30 events/h). These results widely outperformed those from logistic regression and a conventional event-detection algorithm applied to the same database. Conclusion: Our results suggest that AB applied to data from single-channel AF can be useful to determine SAHS and its severity. Significance: SAHS detection might be simplified through the only use of single-channel AF data.Ministerio de Economía y Competitividad (project TEC2011-22987)Junta de Castilla y León (project VA059U13

Multiscale entropy analysis of unattended oximetric recordings to assist in the screening of paediatric sleep apnoea at home

Producción CientíficaUntreated paediatric obstructive sleep apnoea syndrome (OSAS) can severely affect the development and quality of life of children. In-hospital polysomnography (PSG) is the gold standard for a definitive diagnosis though it is relatively unavailable and particularly intrusive. Nocturnal portable oximetry has emerged as a reliable technique for OSAS screening. Nevertheless, additional evidences are demanded. Our study is aimed at assessing the usefulness of multiscale entropy (MSE) to characterise oximetric recordings. We hypothesise that MSE could provide relevant information of blood oxygen saturation (SpO2) dynamics in the detection of childhood OSAS. In order to achieve this goal, a dataset composed of unattended SpO2 recordings from 50 children showing clinical suspicion of OSAS was analysed. SpO2 was parameterised by means of MSE and conventional oximetric indices. An optimum feature subset composed of five MSE-derived features and four conventional clinical indices were obtained using automated bidirectional stepwise feature selection. Logistic regression (LR) was used for classification. Our optimum LR model reached 83.5% accuracy (84.5% sensitivity and 83.0% specificity). Our results suggest that MSE provides relevant information from oximetry that is complementary to conventional approaches. Therefore, MSE may be useful to improve the diagnostic ability of unattended oximetry as a simplified screening test for childhood OSAS.Sociedad Española de Neumología y Cirugía Torácica (SEPAR) project 153/2015Junta de Castilla y León (Consejería de Educación) y el Fondo Europeo de Desarrollo Regional (FEDER), projects (RTC-2015-3446-1) y (TEC2014-53196-R)Ministerio de Economía y Competitividad (MINECO) y FEDER, y el proyecto POCTEP 0378_AD_EEGWA_2_P de la Comisión Europea. L.National Institutes of Health (NIH) grant 1R01HL130984-01Ministerio de Asuntos Económicos y Transformación Digital, grant IJCI-2014-2266

Models and Analysis of Vocal Emissions for Biomedical Applications

The MAVEBA Workshop proceedings, held on a biannual basis, collect the scientific papers presented both as oral and poster contributions, during the conference. The main subjects are: development of theoretical and mechanical models as an aid to the study of main phonatory dysfunctions, as well as the biomedical engineering methods for the analysis of voice signals and images, as a support to clinical diagnosis and classification of vocal pathologies

The 2nd International Electronic Conference on Applied Sciences

This book is focused on the works presented at the 2nd International Electronic Conference on Applied Sciences, organized by Applied Sciences from 15 to 31 October 2021 on the MDPI Sciforum platform. Two decades have passed since the start of the 21st century. The development of sciences and technologies is growing ever faster today than in the previous century. The field of science is expanding, and the structure of science is becoming ever richer. Because of this expansion and fine structure growth, researchers may lose themselves in the deep forest of the ever-increasing frontiers and sub-fields being created. This international conference on the Applied Sciences was started to help scientists conduct their own research into the growth of these frontiers by breaking down barriers and connecting the many sub-fields to cut through this vast forest. These functions will allow researchers to see these frontiers and their surrounding (or quite distant) fields and sub-fields, and give them the opportunity to incubate and develop their knowledge even further with the aid of this multi-dimensional network

A review of ECG-based diagnosis support systems for obstructive sleep apnea

Humans need sleep. It is important for physical and psychological recreation. During sleep our consciousness is suspended or least altered. Hence, our ability to avoid or react to disturbances is reduced. These disturbances can come from external sources or from disorders within the body. Obstructive Sleep Apnea (OSA) is such a disorder. It is caused by obstruction of the upper airways which causes periods where the breathing ceases. In many cases, periods of reduced breathing, known as hypopnea, precede OSA events. The medical background of OSA is well understood, but the traditional diagnosis is expensive, as it requires sophisticated measurements and human interpretation of potentially large amounts of physiological data. Electrocardiogram (ECG) measurements have the potential to reduce the cost of OSA diagnosis by simplifying the measurement process. On the down side, detecting OSA events based on ECG data is a complex task which requires highly skilled practitioners. Computer algorithms can help to detect the subtle signal changes which indicate the presence of a disorder. That approach has the following advantages: computers never tire, processing resources are economical and progress, in the form of better algorithms, can be easily disseminated as updates over the internet. Furthermore, Computer-Aided Diagnosis (CAD) reduces intra- and inter-observer variability. In this review, we adopt and support the position that computer based ECG signal interpretation is able to diagnose OSA with a high degree of accuracy

Pattern recognition applied to airflow recordings to help in sleep Apnea-Hypopnea Syndrome diagnosis

El Síndrome de la Apnea Hipopnea del Sueño (SAHS) es un trastorno caracterizado por pausas respiratorias durante el sueño. Se considera un grave problema de salud que afecta muy negativamente a la calidad de vida y está relacionada con las principales causas de mortalidad, como los accidentes cardiovasculares y cerebrovasculares. A pesar de su elevada prevalencia (2–7%) se considera una enfermedad infradiagnosticada. El diagnóstico estándar se realiza mediante polisomnografía (PSG) nocturna, que es un método complejo y de alto coste. Estas limitaciones han originado largas listas de espera. Esta Tesis Doctoral tiene como principal objetivo simplificar la metodología de diagnóstico del SAHS . Para ello, se propone el análisis exhaustivo de la señal de flujo aéreo monocanal. La metodología propuesta se basa en tres fases (i) extracción de características, (ii) selección de características, y (iii) procesado de la señal mediante métodos de reconocimiento de patrones. Los resultados obtenidos muestran un alto rendimiento diagnóstico de la propuesta tanto en la detección como en la determinación del grado de severidad del SAHS. Por ello, la principal conclusión de la Tesis Doctoral es que los métodos de reconocimiento automático de patrones aplicados sobre la señal de flujo aéreo monocanal resultan de utilidad para reducir la complejidad del proceso de diagnóstico del SAHS.Departamento de Teoría de la Señal y Comunicaciones e Ingeniería Telemátic

Methods for Detecting and Monitoring of Sleep Disordered Breathing in Children using Overnight Polysomnography

Sleep is crucial for the health of every individual, especially children. One of the common causes of disturbed sleep in children is disordered breathing. Children who suffer from sleep disordered breathing are likely to have severe consequences for their physical growth, heart health and neuropsychological function. Sleep disordered breathing (SDB) comprises a spectrum of severity from a mild form of upper airway resistance syndrome (UARS) to severe form of obstructive sleep apnea syndrome (OSAS). While OSAS is considered clinically significant, UARS and its health consequences have been underestimated. The most common treatment for OSAS in children is adenotonsillectomy. However, breathing disturbances related to UARS may persist even after adenotonsillectomy. The current diagnostic marker for OSAS, the Apnea-Hypopnea Index (AHI) often overlooks the less severe conditions of breathing disturbances. Therefore, the research objective of this thesis is to investigate the new alternative markers for SDB in children using non-invasive physiological measurements, such as thoracoabdominal signals and the photoplethysmogram. As the body experiences an array of complex changes, specifically in respiratory and autonomic nervous system activation during breathing disturbances, advanced signal processing and analysis techniques were used to identify the physiological variables that could reflect changes in those systems in children with SDB. Thoraco-abdominal asynchrony (TAA), heart period (HP) and pulse wave amplitude (PWA) were the three physiological variables were investigated. A total of five studies were conducted on two high-quality clinical research datasets to test the potential of the proposed physiological variables to effectively identify children with SDB. In the thesis: 1) Hilbert transform was applied for TAA estimation on the childhood adenotonsillectomy trial (CHAT) dataset; 2) symbolic dynamic analysis on HP was used to assess the effect of adenotonsillectomy on autonomic activations in children with SDB; 3) the conventional method of estimating PWA was combined with joint symbolic analysis of PWA and HP to analyse the effect of SDB on autonomic activation compared to healthy controls; 4) to improve the performance of the previous PWA measurement technique, a more robust and simpler method was proposed to estimate PWA using a simple envelope method, and a more extensive dynamic analysis method was created to capture more complete information; and 5) adding TAA and HP information with AHI, unsupervised machine learning method K-means clustering and linear discriminant analysis were used to discover the pathophysiology nature difference of children with SDB in CHAT dataset. The main results from this thesis suggest that children with SDB have higher values in all three physiological variables, which indicates a high respiratory effort and elevated frequency of autonomic activation. Adenotonsillectomy showed to reverse the effects on these physiological variables, suggesting it assisted in the reduce of pathophysiological symptoms in those children. Interestingly, TAA was found inversely correlated with quality of life and unreported baseline difference in HP in children who had their AHI normalised spontaneously. These findings further indicate the limitation of AHI as the only marker for paediatric sleep disordered breathing. By combining the TAA and HP information with AHI, the alternative proposed diagnosing approach could help doctors predict who may benefit from adenotonsillectomy or not. In conclusion, this thesis provides new evidence that TAA, HP and PWA can provide additional information and may yield more effective markers for diagnosing paediatric sleep disordered breathing.Thesis (Ph.D.) -- University of Adelaide, School of Electrical and Electronic Engineering, 201