How to deal with hyperparmeter optimization in matrix decompositions: new existence theorems and novel stopping criteria

The problem of tuning hyperparameters in learning approaches is an existing and valuable issue that for its nature can affect the real data analysis. Unfortunately, for now these hyperparameters are choosing with Grid-Search or Cross Validation approaches and there is a lack of an automatic tuning procedure to deal with this problem, especially for unsupervised context such as Dimensionality Reduction (DR) approaches. In this paper we deal with the Hyperparameters Optimization problem (HPO) applied to Nonnegative Matrix Factorization (NMF) a particular Linear DR technique. Starting from results in one of our works, we want to enforce the algorithmic strategy through a theoretical foundation. Firstly, motivated by a bilevel formulation of the NMF, we state suitable conditions needed to derive the existence of a minimizer in a infinite-dimensional Hilbert space through a more general existence result. On the other hand, some theoretical results are employed for all those situations when it is not possible (or it is unnecessary) to obtain an exact minimizer. To meet these needs, we derive a stop criterion of an ``approximate" function via Ekeland's Variational Principle. These abstract issues will be applied to our algorithm and some suggestions to other models will be proposed, too

Nonnegative Matrix Factorization models for knowledge extraction from biomedical and other real world data

Inspect data for searching valuable information hidden in represents a key aspect in several fields. Fortunately, most of the available data presents an embedded mathematical structure which can be profitably exploited to better investigate latent patterns hidden in them. Dimensionality Reduction (DR) approaches represent one of the most suitable instrument to untangle latent information. These techniques aim to represent data under analysis onto a low-dimensional space allowing to consider most of all of intrinsic knowledge as ideal sources (namely basis) of the process under consideration. In this work we consider Nonnegative Matrix Factorizations (NMFs), which prove to be the most effective among DR approaches in analyzing real-life nonnegative data. NMF simulates the human part-based learning process which states that parts are combined additively to form a whole. Some variants of NMF will be also presented as minimization tasks to which regularization terms can be added in accordance to some additional characteristics (such as sparsity or orthogonality). We investigate significant computational and interpretative aspects related to NMF according to different application domains, with a specific attention to the analysis of biological data. Moreover we present a new NMF model designed for microarray data analysis that incorporates specific biological proprieties as different constraints. Since NMF and its variants are daily used in several application domains, we conclude stressing how NMF and its constrained variants work in some real life applications, showing some original works related to the analysis of data from engineering field

Low rank approaches for the analysis of real data from pre to post processing

Inspecting data for searching valuable information embedded in them represents a key aspect in several fields, becoming even more challenging because of the continual improvement of technologies which are able to furnish a very large amount of informative data. Fortunately, most of the available data presents an embedded mathematical structure that can be profitably exploited to better investigate latent patterns hidden in them. Analyzing real data covers a biggest set of approaches ranging from pre-processing to the actual discovery of information. In the first context, one of the main problems with real data is often related to the presence of anomalies that may spoil the resulting analysis as well as contain valuable information. In both cases, the ability to detect these occurrences is very important. Particularly, in the biomedical field, proper identification of outliers allows to develop of novel biological hypotheses not taken into consideration when experimental biological data are considered. On the other hand, the actual process of information discovery can be formulated with an optimization task underlying a matrix structure inside. This can be done with the help of Dimensionality Reduction (DR) that represents one of the most suitable instruments to untangle latent information at different levels. In particular, these methods aim to describe data under analysis onto a low-dimensional space allowing to consider most of all of the intrinsic knowledge as ideal sources (namely basis) of the process under consideration [3]. Often these approaches are also enriched by penalization terms can be added to enforce particular constraints able to emphasize useful properties. In this context, the tune of the hyperparameters controlling the weight of the additional constraints represents a problematic issue. In this talk, we focus on Linear DR methods for the analysis of data from the pre to the post processing, to untangle latent information at different levels representing data onto a low-dimensional space. In particular, the contribution of this talk will be twofold. We will first address the problem of detecting outlier samples with application to biomedical data, proposing an ensemble approach for anomalies detection in gene expression matrices based on the use of Hierarchical Clustering and Robust Principal Component Analysis, that allows deriving a novel pseudo-mathematical classification of anomalies [2]. Then, we will focus on Nonnegative Matrix Factorizations (NMFs), which prove to be the most effective among Linear DR methods in analyzing real-life nonnegative data [1]. Some variants of NMF will be also presented as minimization tasks to which penalization terms can be added in accordance with some additional characteristics. In particular, we regard the hyperparameters selection from an optimization point of view, incorporating their choice directly in the unsupervised algorithm as a part of the updating process in a bilevel formulation, providing theoretical and computational results to solve this problem. We will finally sketch future research directions. References [1] Gillis, 2020 Nonnegative Matix Factorization SIAM. [2] Selicato, L.; Esposito, F.; Gargano, G.; Vegliante, M.C.; Opinto, G.; Zaccaria, G.M.; Ciavarella, S.; Guarini, A.; Del Buono, N. 2021 A New Ensemble Method for Detecting Anomalies in Gene Expression Matrices In Mathematics 9, 882 MPDI. [3] Berry, M. W., Drmac Z., and Jessup E. R. 1999 Matrices, vector spaces, and information retrieval. SIAM review, 41(2):35-362

Anomalies Detection in Gene Expression Matrices: Towards a New Approach

One of the main problems in analyzing real data is often related to the presence of anomalies. Anomalous cases may, in fact, spoil the resulting analysis as well as contain valuable information at the same time. In both cases, the ability to detect these occurrences is very important. Particularly, in biomedical field, a proper identification of outliers allows to develop novel biological hypotheses not taken into consideration when experimental biological data are considered. In this paper, we address the problem of detecting outlier samples in gene expression data. We propose an ensemble approach for anomalies detection in gene expression matrices based on the use of hierarchical clustering and Robust Principal Component Analysis, that allows to derive a novel pseudo mathematical classification of anomalies

Bi-level Optimization for hyperparameters in Nonnegative Matrix Factorizations

Hyperparameters (HPs) Optimization in machine learning algorithms represents an open problem with a direct impact on algorithm performances and on the knowledge extraction process from data. Matrix Decompositions (MDs) has recently gained more attention in data science as mathematical techniques able to capture latent information embedded in large datasets. MDs can be formalized as penalized optimization problems in which the tuning of the penalization HPs represents an issue. Current literature panorama does not provide any general framework addressing optimally conditions for the best configuration of HPs. In this work, we focus on the role the HPs play in the penalized Nonnegative Matrix Factorizations (NMF) context and on the importance of their correct selection. Using a bi-level optimization approach, HPs selection is considered from an optimization point of view and their choice is directly incorporated in the unsupervised algorithm as a part of the updating process. Either theorems of existence and convergence of numerical solutions, under appropriate assumptions, are studied and a novel algorithm, namely AltBi-Alternate bi-level Algorithm, is proposed to tune the penalization HP in NMF problems.Comment: 26 pages, 11 Figure

A New Ensemble Method for Detecting Anomalies in Gene Expression Matrices

One of the main problems in the analysis of real data is often related to the presence of anomalies. Namely, anomalous cases can both spoil the resulting analysis and contain valuable information at the same time. In both cases, the ability to detect these occurrences is very important. In the biomedical field, a correct identification of outliers could allow the development of new biological hypotheses that are not considered when looking at experimental biological data. In this work, we address the problem of detecting outliers in gene expression data, focusing on microarray analysis. We propose an ensemble approach for detecting anomalies in gene expression matrices based on the use of Hierarchical Clustering and Robust Principal Component Analysis, which allows us to derive a novel pseudo-mathematical classification of anomalies

Effect of Preoperative Music Therapy Versus Intravenous Midazolam on Anxiety, Sedation and Stress in Stomatology Surgery: A Randomized Controlled Study

Background: Patients undergoing surgery and general anesthesia often experience anxiety, fear and stress, with negative bodily responses. These may be managed by the pre-procedural application of anxiolytic, analgesic, and anesthetic drugs that have, however, potential risks or side effects. Music therapy (MT) can be used as a complementary no-drug intervention alongside standard surgical care before, during and after medical procedures. The aim of this study was to evaluate the effects of preoperative MT intervention compared to premedication with midazolam on levels of anxiety, sedation and stress during general anesthesia for elective stomatology surgery. Methods: A two-arm randomized and controlled single-center, parallel-group, pre–post event study was conducted. In total, 70 patients affected by stage I or II (both clinically and instrumentally N0) micro-invasive oral cancer and undergoing elective surgery under general anesthesia were assigned to the control group (CG) or to the music therapy group (MTG). MTG patients received preoperative music therapy intervention (MT) from a certified music therapist before surgery, while the CG patients did not receive MT but instead received premedication with intravenous midazolam, 0.02 mg/kg. Anesthesia was the same in both groups. The systolic blood pressure (SBP), diastolic blood pressure (DBP) and heart rate (HR) were recorded at the entrance to the operating room, just before the induction of anesthesia and every 5 min until the end of surgery. An anxiety visual analogues scale (A-VAS) was used to evaluate the level of anxiety. The bispectral index (BIS) monitor was used to measure the depth of sedation just before and 10 min after both music intervention and midazolam administration. Stress response was assessed 5 min before and 20 min after surgery via the control of plasma prolactin (PRL), growth hormone (GH), and cortisol levels. The patient global impression of satisfaction (PGIS) was tested 1 h after surgery. Participants in the MTG were asked to answer 3 questions concerning their experience with MT. Results: No statistical differences among the PRL, GH and cortisol levels between the two groups were registered before and after the treatment, as well as for PAS, PAD and HR. Significant differences in the A-VAS scores between the MTG and CG (p < 0.01) was observed. Compared to the CG, MTG patients had a statistically significantly lower BIS score (p = 0.02) before induction. A PGIS score of 86.7% revealed that patients in the MTG were very satisfied, versus 80% in the CG (p < 0.05). Conclusion: Preoperative music therapy could be an alternative to intravenous midazolam when aiming to promote a preoperative and post-operative state of anxiolysis and sedation in stomatology surgery, even if no differences were found in terms of the surgery-related stress response according to physiological and hormonal determinations

Edoxaban versus warfarin in patients with atrial fibrillation

Contains fulltext : 125374.pdf (publisher's version ) (Open Access)BACKGROUND: Edoxaban is a direct oral factor Xa inhibitor with proven antithrombotic effects. The long-term efficacy and safety of edoxaban as compared with warfarin in patients with atrial fibrillation is not known. METHODS: We conducted a randomized, double-blind, double-dummy trial comparing two once-daily regimens of edoxaban with warfarin in 21,105 patients with moderate-to-high-risk atrial fibrillation (median follow-up, 2.8 years). The primary efficacy end point was stroke or systemic embolism. Each edoxaban regimen was tested for noninferiority to warfarin during the treatment period. The principal safety end point was major bleeding. RESULTS: The annualized rate of the primary end point during treatment was 1.50% with warfarin (median time in the therapeutic range, 68.4%), as compared with 1.18% with high-dose edoxaban (hazard ratio, 0.79; 97.5% confidence interval [CI], 0.63 to 0.99; P<0.001 for noninferiority) and 1.61% with low-dose edoxaban (hazard ratio, 1.07; 97.5% CI, 0.87 to 1.31; P=0.005 for noninferiority). In the intention-to-treat analysis, there was a trend favoring high-dose edoxaban versus warfarin (hazard ratio, 0.87; 97.5% CI, 0.73 to 1.04; P=0.08) and an unfavorable trend with low-dose edoxaban versus warfarin (hazard ratio, 1.13; 97.5% CI, 0.96 to 1.34; P=0.10). The annualized rate of major bleeding was 3.43% with warfarin versus 2.75% with high-dose edoxaban (hazard ratio, 0.80; 95% CI, 0.71 to 0.91; P<0.001) and 1.61% with low-dose edoxaban (hazard ratio, 0.47; 95% CI, 0.41 to 0.55; P<0.001). The corresponding annualized rates of death from cardiovascular causes were 3.17% versus 2.74% (hazard ratio, 0.86; 95% CI, 0.77 to 0.97; P=0.01), and 2.71% (hazard ratio, 0.85; 95% CI, 0.76 to 0.96; P=0.008), and the corresponding rates of the key secondary end point (a composite of stroke, systemic embolism, or death from cardiovascular causes) were 4.43% versus 3.85% (hazard ratio, 0.87; 95% CI, 0.78 to 0.96; P=0.005), and 4.23% (hazard ratio, 0.95; 95% CI, 0.86 to 1.05; P=0.32). CONCLUSIONS: Both once-daily regimens of edoxaban were noninferior to warfarin with respect to the prevention of stroke or systemic embolism and were associated with significantly lower rates of bleeding and death from cardiovascular causes. (Funded by Daiichi Sankyo Pharma Development; ENGAGE AF-TIMI 48 ClinicalTrials.gov number, NCT00781391.)