Sliced Cramer synaptic consolidation for preserving deeply learned representations

Deep neural networks suffer from the inability to preserve the learned data representation (i.e., catastrophic forgetting) in domains where the input data distribution is non-stationary, and it changes during training. Various selective synaptic plasticity approaches have been recently proposed to preserve network parameters, which are crucial for previously learned tasks while learning new tasks. We explore such selective synaptic plasticity approaches through a unifying lens of memory replay and show the close relationship between methods like Elastic Weight Consolidation (EWC) and Memory-Aware-Synapses (MAS). We then propose a fundamentally different class of preservation methods that aim at preserving the distribution of the network’s output at an arbitrary layer for previous tasks while learning a new one. We propose the sliced Cramer distance as a suitable ´ choice for such preservation and evaluate our Sliced Cramer Preservation (SCP) ´ algorithm through extensive empirical investigations on various network architectures in both supervised and unsupervised learning settings. We show that SCP consistently utilizes the learning capacity of the network better than online-EWC and MAS methods on various incremental learning tasks

Environmental engineering applications of electronic nose systems based on MOX gas sensors

Nowadays, the electronic nose (e-nose) has gained a huge amount of attention due to its ability to detect and differentiate mixtures of various gases and odors using a limited number of sensors. Its applications in the environmental fields include analysis of the parameters for environmental control, process control, and confirming the efficiency of the odor-control systems. The e-nose has been developed by mimicking the olfactory system of mammals. This paper investigates e-noses and their sensors for the detection of environmental contaminants. Among different types of gas chemical sensors, metal oxide semiconductor sensors (MOXs) can be used for the detection of volatile compounds in air at ppm and sub-ppm levels. In this regard, the advantages and disadvantages of MOX sensors and the solutions to solve the problems arising upon these sensors’ applications are addressed, and the research works in the field of environmental contamination monitoring are overviewed. These studies have revealed the suitability of e-noses for most of the reported applications, especially when the tools were specifically developed for that application, e.g., in the facilities of water and wastewater management systems. As a general rule, the literature review discusses the aspects related to various applications as well as the development of effective solutions. However, the main limitation in the expansion of the use of e-noses as an environmental monitoring tool is their complexity and lack of specific standards, which can be corrected through appropriate data processing methods applications

A Domain-Agnostic Approach for Characterization of Lifelong Learning Systems

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to "real world" events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of "Lifelong Learning" systems that are capable of 1) Continuous Learning, 2) Transfer and Adaptation, and 3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development - both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future.Comment: To appear in Neural Network

The Effects of Drop Height, Conveyor Velocity and Contact Surface Material on Area and Volume Bruising of “Golden Delicious” Apple

Apple fruits are subjected to different loading from harvesting to supermarket shelf. Bruising has been attracted many researchers as one of the most important damage criteria. In this research, the effects of some factors such as contact surface material, drop height and linear velocity of apples were investigated. Influence of these factors on bruising of “Golden Delicious” variety was analyzed by a Completely Randomized Design (CRD) with factorial test at three level of drop height (10, 20 and 30 cm), linear velocity (0.05, 0.1 and 0.15 ms-1) and four contact surface (wooden, steel, plastic and cardboard). Tests were conducted at three replications with 108 treatments. Analysis of variance results showed that the effects of drop height and contact surface material on bruising area at 1% level was significant while bruising volume only affected by contact surface material (meaningful difference at 1% level). Mean comparison test indicated that there significant difference between levels of drop height on the bruise area. Also, there was a meaningful difference between contact surface of cardboard with steel, wood and plastic. There was a significant difference between surface materials of steel and wooden with cardboard and plastic. Therefore, drop height and contact surface material must be considered in designing the apple processing systems

Accurate diagnosis of thyroid follicular lesions from nuclear morphology using supervised learning.

<p>Follicular lesions of the thyroid remain significant diagnostic challenges in surgical pathology and cytology. The diagnosis often requires considerable resources and ancillary tests including immunohistochemistry, molecular studies, and expert consultation. Visual analyses of nuclear morphological features, generally speaking, have not been helpful in distinguishing this group of lesions. Here we describe a method for distinguishing between follicular lesions of the thyroid based on nuclear morphology. The method utilizes an optimal transport-based linear embedding for segmented nuclei, together with an adaptation of existing classification methods. We show the method outputs assignments (classification results) which are near perfectly correlated with the clinical diagnosis of several lesion types' lesions utilizing a database of 94 patients in total. Experimental comparisons also show the new method can significantly outperform standard numerical feature-type methods in terms of agreement with the clinical diagnosis gold standard. In addition, the new method could potentially be used to derive insights into biologically meaningful nuclear morphology differences in these lesions. Our methods could be incorporated into a tool for pathologists to aid in distinguishing between follicular lesions of the thyroid. In addition, these results could potentially provide nuclear morphological correlates of biological behavior and reduce health care costs by decreasing histotechnician and pathologist time and obviating the need for ancillary testing.</p

A collective AI via lifelong learning and sharing at the edge

One vision of a future artificial intelligence (AI) is where many separate units can learn independently over a lifetime and share their knowledge with each other. The synergy between lifelong learning and sharing has the potential to create a society of AI systems, as each individual unit can contribute to and benefit from the collective knowledge. Essential to this vision are the abilities to learn multiple skills incrementally during a lifetime, to exchange knowledge among units via a common language, to use both local data and communication to learn, and to rely on edge devices to host the necessary decentralized computation and data. The result is a network of agents that can quickly respond to and learn new tasks, that collectively hold more knowledge than a single agent and that can extend current knowledge in more diverse ways than a single agent. Open research questions include when and what knowledge should be shared to maximize both the rate of learning and the long-term learning performance. Here we review recent machine learning advances converging towards creating a collective machine-learned intelligence. We propose that the convergence of such scientific and technological advances will lead to the emergence of new types of scalable, resilient and sustainable AI systems.</p

A domain-agnostic approach for characterization of lifelong learning systems

Despite the advancement of machine learning techniques in recent years, state-of-the-art systems lack robustness to “real world” events, where the input distributions and tasks encountered by the deployed systems will not be limited to the original training context, and systems will instead need to adapt to novel distributions and tasks while deployed. This critical gap may be addressed through the development of “Lifelong Learning” systems that are capable of (1) Continuous Learning, (2) Transfer and Adaptation, and (3) Scalability. Unfortunately, efforts to improve these capabilities are typically treated as distinct areas of research that are assessed independently, without regard to the impact of each separate capability on other aspects of the system. We instead propose a holistic approach, using a suite of metrics and an evaluation framework to assess Lifelong Learning in a principled way that is agnostic to specific domains or system techniques. Through five case studies, we show that this suite of metrics can inform the development of varied and complex Lifelong Learning systems. We highlight how the proposed suite of metrics quantifies performance trade-offs present during Lifelong Learning system development — both the widely discussed Stability-Plasticity dilemma and the newly proposed relationship between Sample Efficient and Robust Learning. Further, we make recommendations for the formulation and use of metrics to guide the continuing development of Lifelong Learning systems and assess their progress in the future