Tuning of Learning Algorithms for Use in Automated Product Recommendations

In this thesis, we study the problem of predicting users' media preferences based on their, as well as other users' historical rating data. The model used is that a user's rating for an item can be explained by a sum of bias terms and an inner product between two vectors in some multidimensional feature space that is specic to the product domain. This model is tted using a stochastic descent type method, the stochastic diagonal Levenberg-Marquardt method. The method is studied with respect to its stability and how fast it adapts its parameter estimates and it is found that these characteristics can be accurately predicted for linear regression problems, but that the dynamics become much more complex when training the nonlinear features. The concept of regularization and the tuning of regularization parameters with the Nelder-Mead simplex method is also discussed, as well as some methods for making the Python implementation faster by using Cython

An Information Approach to Regularization Parameter Selection for the Solution of Ill-Posed Inverse Problems Under Model Misspecification

Engineering problems are often ill-posed, i.e. cannot be solved by conventional data-driven methods such as parametric linear and nonlinear regression or neural networks. A method of regularization that is used for the solution of ill-posed problems requires an a priori choice of the regularization parameter. Several regularization parameter selection methods have been proposed in the literature, yet, none is resistant to model misspecification. Since almost all models are incorrectly or approximately specified, misspecification resistance is a valuable option for engineering applications. Each data-driven method is based on a statistical procedure which can perform well on one data set and can fail on other. Therefore, another useful feature of a data- driven method is robustness. This dissertation proposes a methodology of developing misspecification-resistant and robust regularization parameter selection methods through the use of the information complexity approach. The original contribution of the dissertation to the field of ill-posed inverse problems in engineering is a new robust regularization parameter selection method. This method is misspecification-resistant, i.e. it works consistently when the model is misspecified. The method also improves upon the information-based regularization parameter selection methods by correcting inadequate penalization of estimation inaccuracy through the use of the information complexity framework. Such an improvement makes the proposed regularization parameter selection method robust and reduces the risk of obtaining grossly underregularized solutions. A method of misspecification detection is proposed based on the discrepancy between the proposed regularization parameter selection method and its correctly specified version. A detected misspecification indicates that the model may be inadequate for the particular problem and should be revised. The superior performance of the proposed regularization parameter selection method is demonstrated by practical examples. Data for the examples are from Carolina Power & Light\u27s Crystal River Nuclear Power Plant and a TVA fossil power plant. The results of applying the proposed regularization parameter selection method to the data demonstrate that the method is robust, i.e. does not produce grossly underregularized solutions, and performs well when the model is misspecified. This enables one to implement the proposed regularization parameter selection method in autonomous diagnostic and monitoring systems

Pruning neural networks using multi-armed bandits

The successful application of deep learning has led to increasing expectations of their use in embedded systems. This in turn, has created the need to find ways of reducing the size of neural networks. Decreasing the size of a neural network requires deciding which weights should be removed without compromising accuracy, which is analogous to the kind of problems addressed by multi-arm bandits. Hence, this paper explores the use of multi-armed bandits for reducing the number of parameters of a neural network. Different multi-armed bandit algorithms, namely e-greedy, win-stay, lose-shift, UCB1, KL-UCB, BayesUCB, UGapEb, Successive Rejects and Thompson sampling are evaluated and their performance compared to existing approaches. The results show that multi- armed bandit pruning methods, especially those based on UCB, outperform other pruning methods

Non-binary language in Spanish? Comprehension of non-binary morphological forms: a psycholinguistic study

There is empirical evidence in different languages on how the computation of gender morphology during psycholinguistic processing affects the construction of sex-generic representations. However, there are few experimental studies in Spanish and there is no empirical evidence about the psycholinguistic processing of morphological innovations used as non-binary forms (-x; -e) in contrast to the generic masculine variant (-o). To analyze this phenomenon, we designed a sentence comprehension task. We registered reading times, precision and response times. The results show the specialization of non-binary forms as generic morphological variants, as opposed to the generic masculine. The non-binary forms consistently elicited a reference to mixed groups of people and the response times indicated that these morphological variants do not carry a higher processing cost than the generic masculine. Contrary to what classical grammatical approaches propose, the generic masculine does not function in all cases as generic and its ability to refer to groups of people without uniform gender seems to be modulated by the stereotypicality of the role names.Fil: Stetie, Noelia Ayelen. Universidad de Buenos Aires. Facultad de Filosofía y Letras. Instituto de Lingüística; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; ArgentinaFil: Zunino, Gabriela Mariel. Universidad de Buenos Aires. Facultad de Filosofía y Letras. Instituto de Lingüística; Argentina. Consejo Nacional de Investigaciones Científicas y Técnicas; Argentin

Accélération d'une approche régularisée de reconstruction en tomographie à rayons X avec réduction des artéfacts métalliques

Résumé Ce travail porte sur l'imagerie par tomographie à rayons X des vaisseaux périphériques traités par angioplastie avec implantation d'un tuteur endovasculaire métallique. On cherche à détecter le développement de la resténose en mesurant la lumière du vaisseau sanguin imagé. Cette application nécessite la reconstruction d'images de haute résolution. De plus, la présence du tuteur métallique cause l'apparition d'artéfacts qui nuisent à la précision de la mesure dans les images reconstruites dans les appareils tomographiques utilisés en milieu clinique. On propose donc de réaliser la reconstruction à l'aide d'un algorithme axé sur la maximisation pénalisée de la log-vraisemblance conditionnelle de l'image. Cet algorithme est déduit d'un modèle de formation des données qui tient compte de la variation non linéaire de l'atténuation des photons X dans l'objet selon leur énergie, ainsi que du caractère polychromatique du faisceau X. L'algorithme réduit donc effectivement les artéfacts causés spécifiquement par le tuteur métallique. De plus, il peut être configuré de manière à obtenir un compromis satisfaisant entre la résolution de l'image et la variance de l'image reconstruite, selon le niveau de bruit des données. Cette méthode de reconstruction est reconnue pour donner des images d'excellente qualité. Toutefois, le temps de calcul nécessaire à la convergence de cet algorithme est excessivement long. Le but de ce travail est donc de réduire le temps de calcul de cet algorithme de reconstruction itératif. Cette réduction passe par la critique de la formulation du problème et de la méthode de reconstruction, ainsi que par la mise en oeuvre d'approches alternatives.---------- Abstract This thesis is concerned with X-ray tomography of peripheral vessels that have undergone angioplasty with implantation of an endovascular metal stent. We seek to detect the onset of restenosis by measuring the lumen of the imaged blood vessel. This application requires the reconstruction of high-resolution images. In addition, the presence of a metal stent causes streak artifacts that complicate the lumen measurements in images obtained with the usual algorithms, like those implemented in clinical scanners. A regularized statistical reconstruction algorithm, hinged on the maximization of the conditional log-likelihood of the image, is preferable in this case. We choose a variant deduced from a data formation model that takes into account the nonlinear variation of X~photon attenuation to photon energy, as well as the polychromatic character of the X-ray beam. This algorithm effectively reduces the artifacts specifically caused by the metal structures. Moreover, the algorithm may be set to determine a good compromise between image resolution and variance, according to data noise. This reconstruction method is thus known to yield images of excellent quality. However, the runtime to convergence is excessively long. The goal of this work is to reduce the reconstruction runtime

Short-term bitcoin market prediction via machine learning

We analyze the predictability of the bitcoin market across prediction horizons ranging from 1 to 60 min. In doing so, we test various machine learning models and find that, while all models outperform a random classifier, recurrent neural networks and gradient boosting classifiers are especially well-suited for the examined prediction tasks. We use a comprehensive feature set, including technical, blockchain-based, sentiment-/interest-based, and asset-based features. Our results show that technical features remain most relevant for most methods, followed by selected blockchain-based and sentiment-/interest-based features. Additionally, we find that predictability increases for longer prediction horizons. Although a quantile-based long-short trading strategy generates monthly returns of up to 39% before transaction costs, it leads to negative returns after taking transaction costs into account due to the particularly short holding periods

A Survey of Flow Cytometry Data Analysis Methods

Flow cytometry (FCM) is widely used in health research and in treatment for a variety of tasks, such as in the diagnosis and monitoring of leukemia and lymphoma patients, providing the counts of helper-T lymphocytes needed to monitor the course and treatment of HIV infection, the evaluation of peripheral blood hematopoietic stem cell grafts, and many other diseases. In practice, FCM data analysis is performed manually, a process that requires an inordinate amount of time and is error-prone, nonreproducible, nonstandardized, and not open for re-evaluation, making it the most limiting aspect of this technology. This paper reviews state-of-the-art FCM data analysis approaches using a framework introduced to report each of the components in a data analysis pipeline. Current challenges and possible future directions in developing fully automated FCM data analysis tools are also outlined

Model Selection Techniques for Kernel-Based Regression Analysis Using Information Complexity Measure and Genetic Algorithms

In statistical modeling, an overparameterized model leads to poor generalization on unseen data points. This issue requires a model selection technique that appropriately chooses the form, the parameters of the proposed model and the independent variables retained for the modeling. Model selection is particularly important for linear and nonlinear statistical models, which can be easily overfitted. Recently, support vector machines (SVMs), also known as kernel-based methods, have drawn much attention as the next generation of nonlinear modeling techniques. The model selection issues for SVMs include the selection of the kernel, the corresponding parameters and the optimal subset of independent variables. In the current literature, k-fold cross-validation is the widely utilized model selection method for SVMs by the machine learning researchers. However, cross-validation is computationally intensive since one has to fit the model k times. This dissertation introduces the use of a model selection criterion based on information complexity (ICOMP) measure for kernel-based regression analysis and its applications. ICOMP penalizes both the lack-of-fit and the complexity of the model to choose the optimal model with good generalization properties. ICOMP provides a simple index for each model and does not require any validation data. It is computationally efficient and it has been successfully applied to various linear model selection problems. In this dissertation, we introduce ICOMP to the nonlinear kernel-based modeling areas. Specifically, this dissertation proposes ICOMP and its various forms in the area of kernel ridge regression; kernel partial least squares regression; kernel principal component analysis; kernel principal component regression; relevance vector regression; relevance vector logistic regression and classification problems. The model selection tasks achieved by our proposed criterion include choosing the form of the kernel function, the parameters of the kernel function, the ridge parameter, the number of latent variables, the number of principal components and the optimal subset of input variables in a simultaneous fashion for intelligent data mining. The performance of the proposed model selection method is tested on simulation bench- mark data sets as well as real data sets. The predictive performance of the proposed model selection criteria are comparable to and even better than cross-validation, which is too costly to compute and not efficient. This dissertation combines the Genetic Algorithm with ICOMP in variable subsetting, which significantly decreases the computational time as compared to the exhaustive search of all possible subsets. GA procedure is shown to be robust and performs well in our repeated simulation examples. Therefore, this dissertation provides researchers an alternative computationally efficient model selection approach for data analysis using kernel methods