Emerging research directions in computer science : contributions from the young informatics faculty in Karlsruhe

In order to build better human-friendly human-computer interfaces, such interfaces need to be enabled with capabilities to perceive the user, his location, identity, activities and in particular his interaction with others and the machine. Only with these perception capabilities can smart systems ( for example human-friendly robots or smart environments) become posssible. In my research I\u27m thus focusing on the development of novel techniques for the visual perception of humans and their activities, in order to facilitate perceptive multimodal interfaces, humanoid robots and smart environments. My work includes research on person tracking, person identication, recognition of pointing gestures, estimation of head orientation and focus of attention, as well as audio-visual scene and activity analysis. Application areas are humanfriendly humanoid robots, smart environments, content-based image and video analysis, as well as safety- and security-related applications. This article gives a brief overview of my ongoing research activities in these areas

Design for energy-efficient and reliable fog-assisted healthcare IoT systems

Cardiovascular disease and diabetes are two of the most dangerous diseases as they are the leading causes of death in all ages. Unfortunately, they cannot be completely cured with the current knowledge and existing technologies. However, they can be effectively managed by applying methods of continuous health monitoring. Nonetheless, it is difficult to achieve a high quality of healthcare with the current health monitoring systems which often have several limitations such as non-mobility support, energy inefficiency, and an insufficiency of advanced services. Therefore, this thesis presents a Fog computing approach focusing on four main tracks, and proposes it as a solution to the existing limitations. In the first track, the main goal is to introduce Fog computing and Fog services into remote health monitoring systems in order to enhance the quality of healthcare. In the second track, a Fog approach providing mobility support in a real-time health monitoring IoT system is proposed. The handover mechanism run by Fog-assisted smart gateways helps to maintain the connection between sensor nodes and the gateways with a minimized latency. Results show that the handover latency of the proposed Fog approach is 10%-50% less than other state-of-the-art mobility support approaches. In the third track, the designs of four energy-efficient health monitoring IoT systems are discussed and developed. Each energy-efficient system and its sensor nodes are designed to serve a specific purpose such as glucose monitoring, ECG monitoring, or fall detection; with the exception of the fourth system which is an advanced and combined system for simultaneously monitoring many diseases such as diabetes and cardiovascular disease. Results show that these sensor nodes can continuously work, depending on the application, up to 70-155 hours when using a 1000 mAh lithium battery. The fourth track mentioned above, provides a Fog-assisted remote health monitoring IoT system for diabetic patients with cardiovascular disease. Via several proposed algorithms such as QT interval extraction, activity status categorization, and fall detection algorithms, the system can process data and detect abnormalities in real-time. Results show that the proposed system using Fog services is a promising approach for improving the treatment of diabetic patients with cardiovascular disease

A contribution to unobtrusive video-based measurement of respiratory signals

Due to the growing popularity of video-based methods for physiological signal measurement, and taking into account the technological advancements of these type of devices, this work proposes a series of new novel methods to obtain the respiratory signal from a distance, based on video analysis. This thesis aims to improve the state of the art video methods for respiratory measurement, more specifically, by presenting methods that can be used to obtain respiratory variability or perform respiratory rhythm measurements. Moreover, this thesis also aims to present a new implementation of a time-frequency signal processing technique, to improve its computational efficiency when applied to the respiratory signals. In this document a first approach to video-based methods for respiratory signal measurement is performed, to assert the feasibility of using a consumer-grade camera, not only to measure the mean respiratory rate or frequency, but to assert if this hardware could be used to acquire the raw respiratory signal and the respiratory rhythm as well. In this regard a new video-based method was introduced that measures the respiratory signal of a subject at a distance, with the aid of a custom pattern placed on the thorax of the subject. Given the results from the first video-based method, a more broad approach was taken by comparing three different types of video hardware, with the aim to characterise if they could be used for respiratory signal acquisition and respiratory variability measurements. The comparative analysis was performed in terms of instantaneous frequency, as it allowed to characterise the methods in terms of respiratory variability and to compare them in the same terms with the reference method. Subsequently, and due to the previous obtained results, a new method was proposed using a stereo depth camera with the aim to tackle the limitations of the previous study. The proposed method uses an hybrid architecture were the synchronized infrared frame and depth point-cloud from the same camera are acquired. The infrared frame is used to detect the movements of the subject inside the scene, and to recompute on demand a region of interest to obtain the respiratory signal from the depth point-cloud. Furthermore, in this study an opportunistic approach is taken in order to process all the obtained data, as it is also the aim of this study to verify if using a more realistic approach to respiratory signal analysis in real-life conditions, would influence the respiratory rhythm measurement. Even though the depth camera method proved reliable in terms of respiratory rhythm measurement, the opportunistic approach relied on visual inspection of the obtained respiratory signal to properly define each piece. For this reason, a quality indicator had to be proposed that could objectively identify whenever a respiratory signal contained errors. Furthermore, from the idea to characterise the movements of a subject, and by changing the measuring point from a frontal to a lateral perspective to avoid most of the occlusions, a new method based on obtaining the movement of the thoraco-abdominal region using dense optical flow was proposed. This method makes us of the phase of the optical flow to obtain the respiratory signal of the subject, while using the modulus to compute a quality index. Finally, regarding the different signal processing methods used in this thesis to obtain the instantaneous frequency, there were none that could perform in real-time, making the analysis of the respiratory variability not possible in real-life systems where the signals have to be processed in a sample by sample basis. For this reason, as a final chapter a new implementation of the synchrosqueezing transform for time-frequency analysis in real-time is proposed, with the aim to provide a new tool for non-contact methods to obtain the variability of the respiratory signal in real-time.A causa de la creixent popularitat en la mesura de senyals fisiològics amb mètodes de vídeo, i tenint en compte els avenços tecnològics d'aquests dispositius, aquesta tesi proposa una sèrie de nous mètodes per tal d'obtenir la respiració a distància mitjançant l'anàlisi de vídeo. Aquesta tesi té com a objectiu millorar l'estat de l'art referent a la mesura de senyal respiratòria mitjançant els mètodes que en ella es descriuen, així com presentar mètodes que puguin ser usats per obtenir la variabilitat o el ritme respiratori. A més, aquesta tesi té com a objectiu presentar una nova implementació d'un mètode de processat de senyal temps-freqüencial, per tal de millorar-ne l'eficiència computacional quant s’aplica a senyals respiratoris. En aquest document, es realitza una primera aproximació a la mesura de senyal respiratòria mitjançant mètodes de vídeo per tal de verificar si és factible utilitzar una càmera de consum, no només per mesurar el senyal respiratori, sinó verificar si aquest tipus de hardware també pot ser emprat per obtenir el ritme respiratori. En aquest sentit, es presenta en aquest document un nou mètode d'adquisició de senyal respiratòria a distància basat en vídeo, el qual fa ús d'un patró ubicat al tòrax del subjecte per tal d'obtenir-ne la respiració. Un cop obtinguts els resultats del primers resultats, s'han analitzat tres tipus diferents de càmeres, amb la finalitat de caracteritzar-ne la viabilitat d'obtenir el senyal respiratori i la seva variabilitat. L'estudi comparatiu s'ha realitzat en termes de freqüència instantània, donat que permet caracteritzar els mètodes en termes de variabilitat respiratòria i comparar-los, en les mateixes condicions, amb el mètode de referencia. A continuació, s'ha presentat un nou mètode basat en una càmera de profunditat estèreo amb la finalitat de millorar i corregir les limitacions anteriors. El nou mètode proposat es basa en una arquitectura hibrida la qual utilitza els canals de vídeo infraroig i de profunditat de forma sincronitzada. El canal infraroig s'utilitza per detectar els moviments del subjecte dins l'escena i calcular, sota demanda, una regió d'interès que s'utilitza posteriorment en el canal de profunditat per extreure el senyal respiratori. A més a més, en aquest estudi s'ha utilitzat una aproximació oportunista en el processat del senyal respiratori, donat que també és un dels objectius d'aquest estudi, verificar si el fet d'utilitzar una aproximació més realista en l'adquisició de senyal, pot influir en la mesura del ritme respiratori. Tot i que el mètode anterior es mostra fiable en termes de mesura del ritme respiratori, la selecció oportunista del senyal necessita d’inspecció visual per tal de definir correctament cada fragment. Per aquest motiu, era necessari definir un índex de qualitat el qual permetés identificar de forma objectiva cada tram de senyal, així com detectar si el senyal conté errors. Partint de la idea de caracteritzar el moviment del subjecte de l'estudi anterior, i modificant el punt de mesura frontal cap a un de lateral per tal d'evitar oclusions, es proposa un nou mètode basat en l'obtenció del moviment toràcic-abdominal a partir del flux òptic del senyal de vídeo. Aquest mètode recupera el senyal respiratori del subjecte a partir de la fase del flux òptic, tot calculant un índex de qualitat a partir del mòdul. Finalment, i tenint en compte els diferents mètodes de processat utilitzats en aquesta tesi per tal de obtenir la freqüència instantània, es pot apreciar que cap d'ells és capaç de funcionar en temps real, fent inviable l'anàlisi de la variabilitat respiratòria en sistemes reals amb processat mostra a mostra. Per aquest motiu, en el capítol final d'aquesta tesi, s'ha proposat una nova implementació de la transformació "synchrosqueezing" per tal de realitzar l’anàlisi temporal-freqüencial en temps real, i proveir d'una nova eina per tal d'obtenir la variabilitat respiratòria en temps real, amb mètodes sense contacte

Face recognition by means of advanced contributions in machine learning

Face recognition (FR) has been extensively studied, due to both scientific fundamental challenges and current and potential applications where human identification is needed. FR systems have the benefits of their non intrusiveness, low cost of equipments and no useragreement requirements when doing acquisition, among the most important ones. Nevertheless, despite the progress made in last years and the different solutions proposed, FR performance is not yet satisfactory when more demanding conditions are required (different viewpoints, blocked effects, illumination changes, strong lighting states, etc). Particularly, the effect of such non-controlled lighting conditions on face images leads to one of the strongest distortions in facial appearance. This dissertation addresses the problem of FR when dealing with less constrained illumination situations. In order to approach the problem, a new multi-session and multi-spectral face database has been acquired in visible, Near-infrared (NIR) and Thermal infrared (TIR) spectra, under different lighting conditions. A theoretical analysis using information theory to demonstrate the complementarities between different spectral bands have been firstly carried out. The optimal exploitation of the information provided by the set of multispectral images has been subsequently addressed by using multimodal matching score fusion techniques that efficiently synthesize complementary meaningful information among different spectra. Due to peculiarities in thermal images, a specific face segmentation algorithm has been required and developed. In the final proposed system, the Discrete Cosine Transform as dimensionality reduction tool and a fractional distance for matching were used, so that the cost in processing time and memory was significantly reduced. Prior to this classification task, a selection of the relevant frequency bands is proposed in order to optimize the overall system, based on identifying and maximizing independence relations by means of discriminability criteria. The system has been extensively evaluated on the multispectral face database specifically performed for our purpose. On this regard, a new visualization procedure has been suggested in order to combine different bands for establishing valid comparisons and giving statistical information about the significance of the results. This experimental framework has more easily enabled the improvement of robustness against training and testing illumination mismatch. Additionally, focusing problem in thermal spectrum has been also addressed, firstly, for the more general case of the thermal images (or thermograms), and then for the case of facialthermograms from both theoretical and practical point of view. In order to analyze the quality of such facial thermograms degraded by blurring, an appropriate algorithm has been successfully developed. Experimental results strongly support the proposed multispectral facial image fusion, achieving very high performance in several conditions. These results represent a new advance in providing a robust matching across changes in illumination, further inspiring highly accurate FR approaches in practical scenarios.El reconeixement facial (FR) ha estat àmpliament estudiat, degut tant als reptes fonamentals científics que suposa com a les aplicacions actuals i futures on requereix la identificació de les persones. Els sistemes de reconeixement facial tenen els avantatges de ser no intrusius,presentar un baix cost dels equips d’adquisició i no la no necessitat d’autorització per part de l’individu a l’hora de realitzar l'adquisició, entre les més importants. De totes maneres i malgrat els avenços aconseguits en els darrers anys i les diferents solucions proposades, el rendiment del FR encara no resulta satisfactori quan es requereixen condicions més exigents (diferents punts de vista, efectes de bloqueig, canvis en la il·luminació, condicions de llum extremes, etc.). Concretament, l'efecte d'aquestes variacions no controlades en les condicions d'il·luminació sobre les imatges facials condueix a una de les distorsions més accentuades sobre l'aparença facial. Aquesta tesi aborda el problema del FR en condicions d'il·luminació menys restringides. Per tal d'abordar el problema, hem adquirit una nova base de dades de cara multisessió i multiespectral en l'espectre infraroig visible, infraroig proper (NIR) i tèrmic (TIR), sota diferents condicions d'il·luminació. En primer lloc s'ha dut a terme una anàlisi teòrica utilitzant la teoria de la informació per demostrar la complementarietat entre les diferents bandes espectrals objecte d’estudi. L'òptim aprofitament de la informació proporcionada pel conjunt d'imatges multiespectrals s'ha abordat posteriorment mitjançant l'ús de tècniques de fusió de puntuació multimodals, capaces de sintetitzar de manera eficient el conjunt d’informació significativa complementària entre els diferents espectres. A causa de les característiques particulars de les imatges tèrmiques, s’ha requerit del desenvolupament d’un algorisme específic per la segmentació de les mateixes. En el sistema proposat final, s’ha utilitzat com a eina de reducció de la dimensionalitat de les imatges, la Transformada del Cosinus Discreta i una distància fraccional per realitzar les tasques de classificació de manera que el cost en temps de processament i de memòria es va reduir de forma significa. Prèviament a aquesta tasca de classificació, es proposa una selecció de les bandes de freqüències més rellevants, basat en la identificació i la maximització de les relacions d'independència per mitjà de criteris discriminabilitat, per tal d'optimitzar el conjunt del sistema. El sistema ha estat àmpliament avaluat sobre la base de dades de cara multiespectral, desenvolupada pel nostre propòsit. En aquest sentit s'ha suggerit l’ús d’un nou procediment de visualització per combinar diferents bandes per poder establir comparacions vàlides i donar informació estadística sobre el significat dels resultats. Aquest marc experimental ha permès més fàcilment la millora de la robustesa quan les condicions d’il·luminació eren diferents entre els processos d’entrament i test. De forma complementària, s’ha tractat la problemàtica de l’enfocament de les imatges en l'espectre tèrmic, en primer lloc, pel cas general de les imatges tèrmiques (o termogrames) i posteriorment pel cas concret dels termogrames facials, des dels punt de vista tant teòric com pràctic. En aquest sentit i per tal d'analitzar la qualitat d’aquests termogrames facials degradats per efectes de desenfocament, s'ha desenvolupat un últim algorisme. Els resultats experimentals recolzen fermament que la fusió d'imatges facials multiespectrals proposada assoleix un rendiment molt alt en diverses condicions d’il·luminació. Aquests resultats representen un nou avenç en l’aportació de solucions robustes quan es contemplen canvis en la il·luminació, i esperen poder inspirar a futures implementacions de sistemes de reconeixement facial precisos en escenaris no controlats.Postprint (published version

Sensor Signal and Information Processing II

In the current age of information explosion, newly invented technological sensors and software are now tightly integrated with our everyday lives. Many sensor processing algorithms have incorporated some forms of computational intelligence as part of their core framework in problem solving. These algorithms have the capacity to generalize and discover knowledge for themselves and learn new information whenever unseen data are captured. The primary aim of sensor processing is to develop techniques to interpret, understand, and act on information contained in the data. The interest of this book is in developing intelligent signal processing in order to pave the way for smart sensors. This involves mathematical advancement of nonlinear signal processing theory and its applications that extend far beyond traditional techniques. It bridges the boundary between theory and application, developing novel theoretically inspired methodologies targeting both longstanding and emergent signal processing applications. The topic ranges from phishing detection to integration of terrestrial laser scanning, and from fault diagnosis to bio-inspiring filtering. The book will appeal to established practitioners, along with researchers and students in the emerging field of smart sensors processing

Seismic Waves

The importance of seismic wave research lies not only in our ability to understand and predict earthquakes and tsunamis, it also reveals information on the Earth's composition and features in much the same way as it led to the discovery of Mohorovicic's discontinuity. As our theoretical understanding of the physics behind seismic waves has grown, physical and numerical modeling have greatly advanced and now augment applied seismology for better prediction and engineering practices. This has led to some novel applications such as using artificially-induced shocks for exploration of the Earth's subsurface and seismic stimulation for increasing the productivity of oil wells. This book demonstrates the latest techniques and advances in seismic wave analysis from theoretical approach, data acquisition and interpretation, to analyses and numerical simulations, as well as research applications. A review process was conducted in cooperation with sincere support by Drs. Hiroshi Takenaka, Yoshio Murai, Jun Matsushima, and Genti Toyokuni

The Space and Earth Science Data Compression Workshop

This document is the proceedings from a Space and Earth Science Data Compression Workshop, which was held on March 27, 1992, at the Snowbird Conference Center in Snowbird, Utah. This workshop was held in conjunction with the 1992 Data Compression Conference (DCC '92), which was held at the same location, March 24-26, 1992. The workshop explored opportunities for data compression to enhance the collection and analysis of space and Earth science data. The workshop consisted of eleven papers presented in four sessions. These papers describe research that is integrated into, or has the potential of being integrated into, a particular space and/or Earth science data information system. Presenters were encouraged to take into account the scientists's data requirements, and the constraints imposed by the data collection, transmission, distribution, and archival system