N-Dimensional Polynomial Neural Networks and their Applications

In addition to being extremely non-linear, modern machine learning problems require millions if not billions of parameters to solve or at least to get a good approximation of the solution, and neural networks are known to assimilate that complexity by deepening and widening their topology in order to increase the level of non-linearity needed for a better approximation. However, compact topologies are always preferred to deeper ones as they offer the advantage of using less computational units and less parameters. This compactness comes at the price of reduced non-linearity and thus, of limited solution search space. This thesis proposes the N-Dimensional Polynomial Neural Network (NDPNN) model that uses automatic polynomial kernel estimation for N-Dimensional Convolutional Neural Networks (NDCNNs) and introduces a high degree of non-linearity from the first layer which can compensate the need for deep and/or wide topologies. We first theoretically formalized the 1DPNN model which can process 1-dimensional signals and we demonstrated that its inherent non-linearity enables it to yield better results with less computational and spatial complexity than a regular 1DCNN on various classification and regression problems related to audio signals, even though it introduces more computational and spatial complexity on a neuronal level. The experiments were conducted on three publicly available datasets and demonstrate that the proposed 1DPNN model can extract more relevant information from the data than a 1DCNN in less time and with less memory. We subsequently extended the theoretical foundation of the 1DPNN to NDPNN which can process 2D signals such as images and 3D signals such as videos. Also, we theoretically created a general polynomial degree reduction formula that we used to develop a heuristic algorithm, which enables the degree reduction of any pre-trained NDPNN. This algorithm compresses an NDPNN without altering its performance, thus making the model faster and lighter. Following that, we used 2DPNNs and 3DPNNs to tackle the problem of plant species recognition on a publicly available plant species recognition dataset composed of 40,000 images with different sizes consisting of 8 plant species. As a result, we created a novel method, called Variably Overlapping Time—Coherent Sliding Window (VOTCSW), that transforms a dataset composed of images with variable size to a 3D representation with fixed size that is suitable for convolutional neural networks, and we demonstrated that this representation is more informative than resizing the images of the dataset to a given size. We theoretically formalized the use cases of the method as well as its inherent properties and proved that it has an oversampling and a regularization effect on the data. By combining the VOTCSW method with 3DPNNs, we were able to create a model that achieved a state-of-the-art accuracy of 99.9% on the considered dataset, surpassing well-known architectures such as ResNet and Inception. Furthermore, we established that the currently available plant species dataset could not be used for machine learning in its present form, due to a substantial class imbalance between the training set and the test set. Hence, we created a specific preprocessing and a model development framework that enabled us to improve the accuracy from 49.23% to 99.9%. The contributions of this thesis are the creation of a novel generic model called NDPNN that can extract more information from data than a NDCNN with less computational and spatial complexity, the evaluation of the performance of NDPNNs on audio signals, images and videos, the creation of a general direct polynomial reduction formula, the design of a heuristic algorithm for NDPNN compression that generates faster and lighter models, the formalization of an image transformation method that circumvents image resizing without altering fine-grained information, and the production of a state-of-the-art 3DPNN for plant species recognition."I wish to express my thanks and appreciation to Mitacs, the NSERC, and the Faculty of Graduate Studies of the University of Winnipeg for funding my research."Master of Science in Applied Computer Scienc

Self-Organized Operational Neural Networks with Generative Neurons

Operational Neural Networks (ONNs) have recently been proposed to address the well-known limitations and drawbacks of conventional Convolutional Neural Networks (CNNs) such as network homogeneity with the sole linear neuron model. ONNs are heterogenous networks with a generalized neuron model that can encapsulate any set of non-linear operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. However, Greedy Iterative Search (GIS) method, which is the search method used to find optimal operators in ONNs takes many training sessions to find a single operator set per layer. This is not only computationally demanding, but the network heterogeneity is also limited since the same set of operators will then be used for all neurons in each layer. Moreover, the performance of ONNs directly depends on the operator set library used, which introduces a certain risk of performance degradation especially when the optimal operator set required for a particular task is missing from the library. In order to address these issues and achieve an ultimate heterogeneity level to boost the network diversity along with computational efficiency, in this study we propose Self-organized ONNs (Self-ONNs) with generative neurons that have the ability to adapt (optimize) the nodal operator of each connection during the training process. Therefore, Self-ONNs can have an utmost heterogeneity level required by the learning problem at hand. Moreover, this ability voids the need of having a fixed operator set library and the prior operator search within the library in order to find the best possible set of operators. We further formulate the training method to back-propagate the error through the operational layers of Self-ONNs.Comment: 14 pages, 14 figures, journal articl

Exploiting Heterogeneity in Operational Neural Networks by Synaptic Plasticity

The recently proposed network model, Operational Neural Networks (ONNs), can generalize the conventional Convolutional Neural Networks (CNNs) that are homogenous only with a linear neuron model. As a heterogenous network model, ONNs are based on a generalized neuron model that can encapsulate any set of non-linear operators to boost diversity and to learn highly complex and multi-modal functions or spaces with minimal network complexity and training data. However, the default search method to find optimal operators in ONNs, the so-called Greedy Iterative Search (GIS) method, usually takes several training sessions to find a single operator set per layer. This is not only computationally demanding, also the network heterogeneity is limited since the same set of operators will then be used for all neurons in each layer. To address this deficiency and exploit a superior level of heterogeneity, in this study the focus is drawn on searching the best-possible operator set(s) for the hidden neurons of the network based on the Synaptic Plasticity paradigm that poses the essential learning theory in biological neurons. During training, each operator set in the library can be evaluated by their synaptic plasticity level, ranked from the worst to the best, and an elite ONN can then be configured using the top ranked operator sets found at each hidden layer. Experimental results over highly challenging problems demonstrate that the elite ONNs even with few neurons and layers can achieve a superior learning performance than GIS-based ONNs and as a result the performance gap over the CNNs further widens.Comment: 15 pages, 19 figures, journal manuscrip

Impact of seminal trace element and glutathione levels on semen quality of Tunisian infertile men

<p>Abstract</p> <p>Background</p> <p>Growing evidence indicates that oxidative stress can be a primary cause of male infertility. Non-enzymatic antioxidants play an important protective role against oxidative damages and lipid peroxidation. Human seminal plasma is a natural reservoir of antioxidants. The aim of this study was to determine glutathione (GSH) concentrations, trace element levels (zinc and selenium) and the lipid peroxidation end product, malondialdehyde (MDA), in the seminal plasma of men with different fertility potentials.</p> <p>Methods</p> <p>Semen samples from 60 fertile men (normozoospermics) and 190 infertile patients (74 asthenozoospermics, 56 oligozoospermics, and 60 teratozoospermics) were analyzed for physical and biochemical parameters. Zinc (Zn) and selenium (Se) levels were estimated by atomic absorption spectrophotometry. Total GSH (GSHt), oxidized GSH (GSSG), reduced GSH (GSHr) and MDA concentrations were measured spectrophotometrically.</p> <p>Results</p> <p>Zn and Se concentrations in seminal plasma of normozoospermics were more elevated than the three abnormal groups. Nevertheless, only the Zn showed significant differences. On the other hand, Zn showed positive and significant correlations with sperm motility (P = 0.03, r = 0.29) and count (P < 0.01, r = 0.49); however Se was significantly correlated only with sperm motility (P < 0.01, r = 0.36). GSHt, GSSG and GSHr were significantly higher in normozoospermics than in abnormal groups. We noted a significant association between seminal GSHt and sperm motility (P = 0.03). GSSG was highly correlated to sperm motility (P < 0.001) and negatively associated to abnormal morphology (P < 0.001). GSHr was significantly associated to total sperm motility (P < 0.001) and sperm count (P = 0.01). MDA levels were significantly higher in the three abnormal groups than in normozoospermics. Rates of seminal MDA were negatively associated to sperm motility (P < 0.01; r = -0.24) and sperm concentration (P = 0.003; r = -0.35) Meanwhile, there is a positive correlation between seminal lipid peroxidation and the percentage of abnormal morphology (P = 0.008).</p> <p>Conclusions</p> <p>This report revealed that decreased seminal GSH and trace element deficiencies are implicated in low sperm quality and may be an important indirect biomarker of idiopathic male infertility. Our results sustain that the evaluation of seminal antioxidant status in infertile men is necessary and can be helpful in fertility assessment from early stages.</p

Spatial, temporal, and demographic patterns in prevalence of chewing tobacco use in 204 countries and territories, 1990-2019 : a systematic analysis from the Global Burden of Disease Study 2019

Interpretation Chewing tobacco remains a substantial public health problem in several regions of the world, and predominantly in south Asia. We found little change in the prevalence of chewing tobacco use between 1990 and 2019, and that control efforts have had much larger effects on the prevalence of smoking tobacco use than on chewing tobacco use in some countries. Mitigating the health effects of chewing tobacco requires stronger regulations and policies that specifically target use of chewing tobacco, especially in countries with high prevalence. Findings In 2019, 273 center dot 9 million (95% uncertainty interval 258 center dot 5 to 290 center dot 9) people aged 15 years and older used chewing tobacco, and the global age-standardised prevalence of chewing tobacco use was 4 center dot 72% (4 center dot 46 to 5 center dot 01). 228 center dot 2 million (213 center dot 6 to 244 center dot 7; 83 center dot 29% [82 center dot 15 to 84 center dot 42]) chewing tobacco users lived in the south Asia region. Prevalence among young people aged 15-19 years was over 10% in seven locations in 2019. Although global agestandardised prevalence of smoking tobacco use decreased significantly between 1990 and 2019 (annualised rate of change: -1 center dot 21% [-1 center dot 26 to -1 center dot 16]), similar progress was not observed for chewing tobacco (0 center dot 46% [0 center dot 13 to 0 center dot 79]). Among the 12 highest prevalence countries (Bangladesh, Bhutan, Cambodia, India, Madagascar, Marshall Islands, Myanmar, Nepal, Pakistan, Palau, Sri Lanka, and Yemen), only Yemen had a significant decrease in the prevalence of chewing tobacco use, which was among males between 1990 and 2019 (-0 center dot 94% [-1 center dot 72 to -0 center dot 14]), compared with nine of 12 countries that had significant decreases in the prevalence of smoking tobacco. Among females, none of these 12 countries had significant decreases in prevalence of chewing tobacco use, whereas seven of 12 countries had a significant decrease in the prevalence of tobacco smoking use for the period. Summary Background Chewing tobacco and other types of smokeless tobacco use have had less attention from the global health community than smoked tobacco use. However, the practice is popular in many parts of the world and has been linked to several adverse health outcomes. Understanding trends in prevalence with age, over time, and by location and sex is important for policy setting and in relation to monitoring and assessing commitment to the WHO Framework Convention on Tobacco Control. Methods We estimated prevalence of chewing tobacco use as part of the Global Burden of Diseases, Injuries, and Risk Factors Study 2019 using a modelling strategy that used information on multiple types of smokeless tobacco products. We generated a time series of prevalence of chewing tobacco use among individuals aged 15 years and older from 1990 to 2019 in 204 countries and territories, including age-sex specific estimates. We also compared these trends to those of smoked tobacco over the same time period. Findings In 2019, 273 & middot;9 million (95% uncertainty interval 258 & middot;5 to 290 & middot;9) people aged 15 years and older used chewing tobacco, and the global age-standardised prevalence of chewing tobacco use was 4 & middot;72% (4 & middot;46 to 5 & middot;01). 228 & middot;2 million (213 & middot;6 to 244 & middot;7; 83 & middot;29% [82 & middot;15 to 84 & middot;42]) chewing tobacco users lived in the south Asia region. Prevalence among young people aged 15-19 years was over 10% in seven locations in 2019. Although global age standardised prevalence of smoking tobacco use decreased significantly between 1990 and 2019 (annualised rate of change: -1 & middot;21% [-1 & middot;26 to -1 & middot;16]), similar progress was not observed for chewing tobacco (0 & middot;46% [0 & middot;13 to 0 & middot;79]). Among the 12 highest prevalence countries (Bangladesh, Bhutan, Cambodia, India, Madagascar, Marshall Islands, Myanmar, Nepal, Pakistan, Palau, Sri Lanka, and Yemen), only Yemen had a significant decrease in the prevalence of chewing tobacco use, which was among males between 1990 and 2019 (-0 & middot;94% [-1 & middot;72 to -0 & middot;14]), compared with nine of 12 countries that had significant decreases in the prevalence of smoking tobacco. Among females, none of these 12 countries had significant decreases in prevalence of chewing tobacco use, whereas seven of 12 countries had a significant decrease in the prevalence of tobacco smoking use for the period. Interpretation Chewing tobacco remains a substantial public health problem in several regions of the world, and predominantly in south Asia. We found little change in the prevalence of chewing tobacco use between 1990 and 2019, and that control efforts have had much larger effects on the prevalence of smoking tobacco use than on chewing tobacco use in some countries. Mitigating the health effects of chewing tobacco requires stronger regulations and policies that specifically target use of chewing tobacco, especially in countries with high prevalence. Copyright (c) 2021 The Author(s). Published by Elsevier Ltd. This is an Open Access article under the CC BY 4.0 license.Peer reviewe

Global, regional, and national age-sex-specific mortality for 282 causes of death in 195 countries and territories, 1980-2017 : a systematic analysis for the Global Burden of Disease Study 2017

Background Global development goals increasingly rely on country-specific estimates for benchmarking a nation's progress. To meet this need, the Global Burden of Diseases, Injuries, and Risk Factors Study (GBD) 2016 estimated global, regional, national, and, for selected locations, subnational cause-specific mortality beginning in the year 1980. Here we report an update to that study, making use of newly available data and improved methods. GBD 2017 provides a comprehensive assessment of cause-specific mortality for 282 causes in 195 countries and territories from 1980 to 2017. Methods The causes of death database is composed of vital registration (VR), verbal autopsy (VA), registry, survey, police, and surveillance data. GBD 2017 added ten VA studies, 127 country-years of VR data, 502 cancer-registry country-years, and an additional surveillance country-year. Expansions of the GBD cause of death hierarchy resulted in 18 additional causes estimated for GBD 2017. Newly available data led to subnational estimates for five additional countries Ethiopia, Iran, New Zealand, Norway, and Russia. Deaths assigned International Classification of Diseases (ICD) codes for non-specific, implausible, or intermediate causes of death were reassigned to underlying causes by redistribution algorithms that were incorporated into uncertainty estimation. We used statistical modelling tools developed for GBD, including the Cause of Death Ensemble model (CODErn), to generate cause fractions and cause specific death rates for each location, year, age, and sex. Instead of using UN estimates as in previous versions, GBD 2017 independently estimated population size and fertility rate for all locations. Years of life lost (YLLs) were then calculated as the sum of each death multiplied by the standard life expectancy at each age. All rates reported here are age-standardised. Findings At the broadest grouping of causes of death (Level 1), non-communicable diseases (NC Ds) comprised the greatest fraction of deaths, contributing to 73.4% (95% uncertainty interval [UI] 72.5-74.1) of total deaths in 2017, while communicable, maternal, neonatal, and nutritional (CMNN) causes accounted for 186% (17.9-19.6), and injuries 8.0% (7.7-8.2). Total numbers of deaths from NCD causes increased from 2007 to 2017 by 22.7% (21.5-23.9), representing an additional 7.61 million (7. 20-8.01) deaths estimated in 2017 versus 2007. The death rate from NCDs decreased globally by 7.9% (7.08.8). The number of deaths for CMNN causes decreased by 222% (20.0-24.0) and the death rate by 31.8% (30.1-33.3). Total deaths from injuries increased by 2.3% (0-5-4-0) between 2007 and 2017, and the death rate from injuries decreased by 13.7% (12.2-15.1) to 57.9 deaths (55.9-59.2) per 100 000 in 2017. Deaths from substance use disorders also increased, rising from 284 000 deaths (268 000-289 000) globally in 2007 to 352 000 (334 000-363 000) in 2017. Between 2007 and 2017, total deaths from conflict and terrorism increased by 118.0% (88.8-148.6). A greater reduction in total deaths and death rates was observed for some CMNN causes among children younger than 5 years than for older adults, such as a 36.4% (32.2-40.6) reduction in deaths from lower respiratory infections for children younger than 5 years compared with a 33.6% (31.2-36.1) increase in adults older than 70 years. Globally, the number of deaths was greater for men than for women at most ages in 2017, except at ages older than 85 years. Trends in global YLLs reflect an epidemiological transition, with decreases in total YLLs from enteric infections, respirator}, infections and tuberculosis, and maternal and neonatal disorders between 1990 and 2017; these were generally greater in magnitude at the lowest levels of the Socio-demographic Index (SDI). At the same time, there were large increases in YLLs from neoplasms and cardiovascular diseases. YLL rates decreased across the five leading Level 2 causes in all SDI quintiles. The leading causes of YLLs in 1990 neonatal disorders, lower respiratory infections, and diarrhoeal diseases were ranked second, fourth, and fifth, in 2017. Meanwhile, estimated YLLs increased for ischaemic heart disease (ranked first in 2017) and stroke (ranked third), even though YLL rates decreased. Population growth contributed to increased total deaths across the 20 leading Level 2 causes of mortality between 2007 and 2017. Decreases in the cause-specific mortality rate reduced the effect of population growth for all but three causes: substance use disorders, neurological disorders, and skin and subcutaneous diseases. Interpretation Improvements in global health have been unevenly distributed among populations. Deaths due to injuries, substance use disorders, armed conflict and terrorism, neoplasms, and cardiovascular disease are expanding threats to global health. For causes of death such as lower respiratory and enteric infections, more rapid progress occurred for children than for the oldest adults, and there is continuing disparity in mortality rates by sex across age groups. Reductions in the death rate of some common diseases are themselves slowing or have ceased, primarily for NCDs, and the death rate for selected causes has increased in the past decade. Copyright (C) 2018 The Author(s). Published by Elsevier Ltd.Peer reviewe