Maximum Entropy Vector Kernels for MIMO system identification

Recent contributions have framed linear system identification as a nonparametric regularized inverse problem. Relying on $\ell_2$-type regularization which accounts for the stability and smoothness of the impulse response to be estimated, these approaches have been shown to be competitive w.r.t classical parametric methods. In this paper, adopting Maximum Entropy arguments, we derive a new $\ell_2$ penalty deriving from a vector-valued kernel; to do so we exploit the structure of the Hankel matrix, thus controlling at the same time complexity, measured by the McMillan degree, stability and smoothness of the identified models. As a special case we recover the nuclear norm penalty on the squared block Hankel matrix. In contrast with previous literature on reweighted nuclear norm penalties, our kernel is described by a small number of hyper-parameters, which are iteratively updated through marginal likelihood maximization; constraining the structure of the kernel acts as a (hyper)regularizer which helps controlling the effective degrees of freedom of our estimator. To optimize the marginal likelihood we adapt a Scaled Gradient Projection (SGP) algorithm which is proved to be significantly computationally cheaper than other first and second order off-the-shelf optimization methods. The paper also contains an extensive comparison with many state-of-the-art methods on several Monte-Carlo studies, which confirms the effectiveness of our procedure

Analysis of the dynamic response of a long span bridge using GPS/accelerometer/anemometer under typhoon loading

Large flexible engineering structures, such as long span bridges or tall buildings, are susceptible to quasistatic and dynamic deformations caused by different loadings, thus accurate displacement measurements are desirable to assess the integrity and reliability of the structure. In this study, an integrated system that includes Global Positioning System (GPS), accelerometer and anemometer was developed to obtain the responses of a long span bridge to the extreme wind loadings. Spectral analysis based on the Fast Fourier Transform (FFT) algorithm was first carried out to detect the dominant frequencies of the middle pylon. Then the noisy GPS displacement measurements and accelerometer data are de-noised using the Vondrak filter, and the low frequency disturbance was separated from GPS displacement time series. A least-squares based displacement reconstruction scheme using noise-mitigated accelerations was employed, and the Tikhonov regularization scheme with optimal selected regularization factor was used to alleviate the ill-posedness. At last, an adaptive recursive least squares (RLS) filter was adopted to separate the slow-varying movements, and the total displacement with enhanced measurement accuracy was obtained from the combined quasi-static and high-frequency dynamic displacements. A field monitoring data set collected on the Erqi Yangtze River Bridge, a three-tower cable-stayed bridge located in Wuhan, China, was used to validate the effectiveness of the proposed integration processing scheme. The GPS/accelerometer/anemometer installed on the center supporting tower was used to characterize the interaction between the responses and the ambient wind loadings. The results demonstrate the proposed technique can significantly improve the measurement accuracy of pylon displacement under strong winds. The deformation accuracy with the amplitude of several millimeters can be successfully detected,and the spectrum of the pylon response obtained from both GPS data and accelerometer data reveals the identified first dominant frequency of the middle pylon is 0.172 Hz

Cost-Effective Monitoring of Railroad Bridge Performance

The railroad network carries 40 % of the freight in the US. Railroad bridges are the most critical part of the network and they need to be properly maintained for safety of operations. Railroad managers need to inspect the bridges in order to assess their structural condition. Railroad managers are interested in measuring displacements under train crossing events to prioritize their bridge management and safety decisions. However, bridge displacements are difficult to collect in the field, because they require a fixed reference from where to measure. Accelerations can be used to estimate dynamic displacements but to this date, the pseudo-static displacements cannot be measured using reference-free sensors. This study proposes a method to estimate the total displacements of a railroad bridge under live train loads using acceleration and tilt data without a need for fixed reference. Researchers used real bridge displacement data representing different bridge serviceability level under train traffic. This study explores the design of a new bridge deck-pier experimental model that simulates the vibrations of railroad bridges under traffic. This experiment configuration includes the use of a shake table to input the recorded signal from the field into a railroad pile bent. Reference-free sensors measured both the inclination angle and accelerations of the pile cap. The different acceleration readings are used to estimate the total displacements of the bridge using data filtering. The estimated displacements are then compared to the true responses of the model measured with displacement sensors. The results show that this method can cost-effectively measure the total displacement of railroad bridges without a fixed reference. In addition, this paper studies the use of a low-cost data acquisition platform to measure reference-free dynamic displacements of railroad bridges by combining low-cost microcontrollers and accelerometers. Researchers used the new system to measure accelerations and reconstruct reference-free displacements from several railroad bridge crossing events. The results obtained from the proposed low-cost sensors were compared with those of commercial sensing equipment. The results show that low-cost sensors and commercial sensing systems can measure reference-free displacements with comparable accuracy. The results of this study show that the proposed platform estimates reference-free displacements with a peak error between 20 % and 30 % and a root mean square (RMS) error between 10 % and 20 %, which is similar to commercial SHM systems. The proposed low-cost system is approximately 300 times less expensive than the commercial sensing equipment. In conclusion, this study evaluates the accuracy of cost-effective systems to measure the reference-free displacement of railroad bridges. The conclusions of this study propose a cost-effective method to measure the reference-free displacement of railroad bridges that all railroad companies can afford. The ultimate goal of this research is to provide stakeholders with means to design, develop, own, and operate their own SHM systems

Robust and Regularized Algorithms for Vehicle Tractive Force Prediction and Mass Estimation

This work provides novel robust and regularized algorithms for parameter estimation with applications in vehicle tractive force prediction and mass estimation. Given a large record of real world data from test runs on public roads, recursive algorithms adjusted the unknown vehicle parameters under a broad variation of statistical assumptions for two linear gray-box models

Lichen Conservation in Eastern North America: Population Genomics, Climate Change, and Translocations

Conservation biology is a scientific discipline that draws on methods from diverse fields to address specific conservation concerns and inform conservation actions. This field is overwhelmingly focused on charismatic animals and vascular plants, often ignoring other diverse and ecologically important groups. This trend is slowly changing in some ways; for example, increasing number of fungal species are being added to the IUCN Red-List. However, a strong taxonomic bias still exists. Here I contribute four research chapters to further the conservation of lichens, one group of frequently overlooked organisms. I address specific conservation concerns in eastern North America using modern methods. The results of these studies provide insight into lichen conservation in each situation, implications for the broader ecosystems within the study regions, and advancement of methods for the study of lichen conservation and biology. The first research chapter (Chapter 2) is a population genomics study based on whole genome shotgun sequencing of Cetradonia linearis, an endangered, lichenized fungus. These data were used to 1) assemble and annotate a reference genome, 2) characterize the mating system, 3) test for isolation by distance (IBD) and isolation by environment (IBE), and 4) investigate the biogeographic history of the species. Approximately 70% of the genome (19.5 Mb) was assembled. Using this assembly, only a single mating type was located, suggesting the species could be unisexual. There was strong evidence for both low rates of recombination and for Isolation by Distance, but no evidence for Isolation by Environment. The hypothesis that C. linearis had a larger range during the last glacial maximum, especially in the southern portion of its current extent, was supported by Hindcast species distribution models and the spatial distribution of genetic diversity. Given the findings here, it is recommended that C. linearis remain protected by the U.S. Endangered Species Act and listed as Vulnerable on the International Union for the Conservation of Nature Red-List. The third chapter is an estimation of the impacts of climate change on high-elevation, endemic lichens in the southern Appalachians, a global diversity hotspot for many groups, including lichens. Extensive field surveys in the high elevations of the region were carried out to accurately document the current distributions of eight narrowly endemic species. These data were compared with herbarium records, and species distribution modeling was used to predict how much climatically suitable area will remain within, and north of, the current range of the target species at multiple time points and climate change scenarios. Fieldwork showed that target species ranged from extremely rare to locally abundant and models predicted average losses of suitable area within the current distribution of species ranging from 93.8 to 99.7%. The results indicate that climate change poses a significant threat to high-elevation lichens, and illustrates the application of current modeling techniques for rare, montane species. In the fourth chapter, a dataset of \u3e13,000 occurrence records for lichens in the Mid-Atlantic Coastal Plain (MACP) of eastern North America was used to model distributions of 193 species. The resulting models were used to quantify the amount of each species’ distribution that is occupied by unsuitable land use types, along with the potential area that will be lost to sea-level rise (SLR). These analyses showed that species have likely already lost an average of 32% of their distributional area to development and agriculture, and are predicted to lose an average of 12.4 and 33.7% of their distributional area with one foot (~0.3 m) and six feet (~1.8 m) of SLR, respectively. Functional and taxonomic groups were compared to identify specific effects of SLR. Species reproducing with symbiotic propagules were found to have significantly larger distributions than species that reproduce sexually with fungal spores alone, and the sexually reproducing species were predicted to lose greater distributional area to SLR. Cladonia species occupy significantly less area in the MACP than Parmotrema species and were predicted to lose more of their distributions to SLR. Patterns of total species diversity showed that the area with the highest diversity is the Dare Peninsula in North Carolina, which was also predicted to lose the most land area to SLR. The workflow established here is flexible and applicable to estimating SLR impacts worldwide and can provide essential insights for local conservation planning. The fifth chapter describes the results of three experiments conducted to test new and established methods for lichen transplantation. First, small fragments of Graphis sterlingiana, Hypotrachyna virginica, and Lepraria lanata were placed on medical gauze attached to each of the species’ most common substrate to test the feasibility of transplanting narrowly endemic species. Second, burlap, cheesecloth, medical gauze, and a plastic air filter were directly compared for their use as artificial transplant substrates with Lepraria finkii as the test lichen. Third, transplants of Usnea angulata were established to test its amenability to transplantation via hanging fragments on monofilament. The first two experiments were established on Roan Mountain, North Carolina and the third experiment at Highlands Biological Station, North Carolina. In the first two experiments medical gauze did not withstand local weather conditions and nearly all pieces fell from the trees within 6 months. The plastic air filter and burlap performed best as artificial substrates for transplants, with a 100% and 80% success rate, respectively. Cheesecloth remained attached to the trees, but only 20% of lichen fragments remained attached to the substrate after one year. In the third experiment U. angulata grew 3.5 ± 1.4 cm in 5 months, exceeding previously reported growth rates for this species. These results advance methods for conservation-focused lichen transplants, and expand established methods to a new region and new species

Modeling provincial Covid-19 epidemic data in Italy using an adjusted time-dependent SIRD model

In this paper we develop a predictive model for the spread of COVID-19 infection at a provincial (i.e. EU NUTS-3) level in Italy by using official data from the Italian Ministry of Health integrated with data extracted from daily official press conferences of regional authorities and from local newspaper websites. This integration is mainly concerned with COVID-19 cause specific death data which are not available at NUTS-3 level from open official data data channels. An adjusted time-dependent SIRD model is used to predict the behavior of the epidemic, specifically the number of susceptible, infected, deceased and recovered people. Predictive model performance is evaluated using comparison with real data

Detection and analysis of heartbeats in seismocardiogram signals

This paper presents an unsupervised methodology to analyze SeismoCardioGram (SCG) signals. Starting from raw accelerometric data, heartbeat complexes are extracted and annotated, using a two-step procedure. An unsupervised calibration procedure is added to better adapt to different user patterns. Results show that the performance scores achieved by the proposed methodology improve over related literature: on average, 98.5% sensitivity and 98.6% precision are achieved in beat detection, whereas RMS (Root Mean Square) error in heartbeat interval estimation is as low as 4.6 ms. This allows SCG heartbeat complexes to be reliably extracted. Then, the morphological information of such waveforms is further processed by means of a modular Convolutional Variational AutoEncoder network, aiming at extracting compressed, meaningful representation. After unsupervised training, the VAE network is able to recognize different signal morphologies, associating each user to its specific patterns with high accuracy, as indicated by specific performance metrics (including adjusted random and mutual information score, completeness, and homogeneity). Finally, a Linear Model is used to interpret the results of clustering in the learned latent space, highlighting the impact of different VAE architectural parameters (i.e., number of stacked convolutional units and dimension of latent space)

Development of a methodology for the diagnosis of internal combustion engines using non-invasive measurements based on the use of interpretable neural networks applicable to databases with multiple annotators

Pressure is one of the essential variables that give information for engine condition and monitoring. Direct recording of this signal is complex and invasive, while the angular velocity can be measured easily. Nonetheless, the challenge is to predict the cylinder pressure using the shaft kinematics accurately. On the other hand, the increasing popularity of crowdsourcing platforms, i.e., Amazon Mechanical Turk, changes how datasets for supervised learning are built. In these cases, instead of having datasets labeled by one source (which is supposed to be an expert who provided the absolute gold standard), databases holding multiple annotators are provided. However, most state-of-the-art methods devoted to learning from multiple experts assume that the labeler's behavior is homogeneous across the input feature space. Besides, independence constraints are imposed on annotators' outputs. This document presents a Regularized Chained Deep Neural Network to deal with classification tasks from multiple annotators. In this thesis, we develop 2 strategies aiming to avoid intrusive techniques that are commonly used to diagnose Internal Combustion Engines (ICE). The first consist of a time-delay neural network (TDNN), interpreted as a finite pulse response (FIR) filter to estimate the in-cylinder pressure of a single-cylinder ICE from fluctuations in shaft angular velocity. The experiments are conducted over data obtained from an ICE operating in 12 different states by changing the angular velocity and load. The TDNN's delay is adjusted to get the highest possible correlation-based score. Our methodology can predict pressure with an R2>0.9, avoiding complicated pre-processing steps. The second technique, termed RCDNN, jointly predicts the ground truth label and the annotators' performance from input space samples. In turn, RCDNN codes interdependencies among the experts by analyzing the layers' weights and includes l1, l2, and Monte-Carlo Dropout-based regularizers to deal with the overfitting issue in deep learning models. Obtained results (using both simulated and real-world annotators) demonstrate that RCDNN can deal with multi-labelers scenarios for classification tasks, defeating state-of-the-art techniques.La presión es una de las variables esenciales que dan información para el estado del motor y su monitorización. El registro directo de esta señal es complejo e invasivo, mientras que la velocidad angular puede medirse fácilmente. No obstante, el reto consiste en predecir la presión del cilindro utilizando la cinemática del eje con precisión. Por otro lado, la creciente popularidad de las plataformas de crowdsourcing, por ejemplo, Amazon Mechanical Turk, cambia la forma de construir conjuntos de datos para el aprendizaje supervisado. En estos casos, en lugar de tener conjuntos de datos etiquetados por una sola fuente (que se supone que es un experto que proporcionó el estándar de oro absoluto), se proporcionan bases de datos con múltiples anotadores. Sin embargo, la mayoría de los métodos de vanguardia dedicados al aprendizaje a partir de múltiples expertos suponen que el comportamiento del etiquetador es homogéneo en todo el espacio de características de entrada. Además, se imponen restricciones de independencia a los resultados de los anotadores. Este documento presenta una Red Neuronal Profunda Encadenada Regularizada para abordar tareas de clasificación a partir de múltiples anotadores. En esta tesis, desarrollamos dos estrategias con el objetivo de evitar las técnicas intrusivas que se utilizan habitualmente para diagnosticar motores de combustión interna (ICE). La primera consiste en una red neuronal de retardo temporal (TDNN), interpretada como un filtro de respuesta de pulso finito (FIR) para estimar la presión en el cilindro de un ICE de un solo cilindro a partir de las fluctuaciones de la velocidad angular del eje. Los experimentos se realizan sobre datos obtenidos de un ICE que opera en 12 estados diferentes cambiando la velocidad angular y la carga. El retardo de la TDNN se ajusta para obtener la mayor puntuación posible basada en la correlación. Nuestra metodología puede predecir la presión con un R2>0,9, evitando complicados pasos de preprocesamiento.MaestríaMagíster en Ingeniería EléctricaContent 1 Introduction 10 1.1 Problem statement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11 1.2 Justification . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14 1.3 Objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.3.1 General objective . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 1.3.2 Specific objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17 2 TDNN-based Engine In-cylinder Pressure Estimation from Shaft Velocity Spectral Representation 18 2.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19 2.2.1 Time Delay Neural Network fundamentals . . . . . . . . . . . . . . . 19 2.2.2 Harmonic prediction performance based on Magnitude-Squared Coherence . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.3 Experimental Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 2.3.1 Engine Measurements, Data Acquisition, and Preprocessing . . . . . 22 2.3.2 Pressure signal estimation . . . . . . . . . . . . . . . . . . . . . . . . 26 2.4 Results and discussion . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 26 2.5 Conclusions and future work . . . . . . . . . . . . . . . . . . . . . . . . . . . 35 3 Master Thesis: Content 3 Regularized Chained Deep Neural Network Classifier for Multiple Annotators 37 3.1 Abstract . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.2 Materials and Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.2.1 Related work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38 3.2.2 Methods . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39 3.3 Experimental set-up . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.3.1 Tested datasets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43 3.3.2 RCDNN detailed architecture and training . . . . . . . . . . . . . . . 46 3.3.3 Results and discussion . . . . . . . . . . . . . . . . . . . . . . . . . . 48 3.3.4 Introducing spammers and malicious annotators . . . . . . . . . . . . 55 3.3.5 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 56 4 Final Remarks 58 4.1 Conclusions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.1.1 TDNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.1.2 RCDNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59 4.2 Future Work . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.2.1 TDNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.2.2 RCDNN . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60 4.3 Publications . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6