Efficient Neural Network Synthesis and Its Application in Smart Healthcare

Abstract

Deep neural networks (DNNs) have become the driving force behind recent artificial intelligence (AI) research. With the help of a vast amount of training data, neural networks can perform better than traditional machine learning algorithms in many applications. However, there are still several issues in the training and deployment of DNNs that limit the use of DNNs in various applications. First, an important problem with implementing a DNN is the design of its architecture. Typically, such an architecture is obtained manually by exploring its hyperparameter space and kept fixed during training. Due to the need to navigate a search space based on a large number of hyperparameters, this can become challenging. This forces the space of possible architectures to grow exponentially. As a result, using a trial-and-error design approach is very time-consuming and leads to sub-optimal architectures. In addition, approaches like neural architecture search (NAS) based on reinforcement learning (RL) and differentiable gradient-based architecture search often incur huge computational costs or significant memory requirements. Another issue is that modern neural networks often contain millions of parameters, whereas many applications require small inference models due to imposed resource constraints, such as energy constraints on battery-operated devices. However, efforts to migrate DNNs to such devices typically entail a significant loss of classification accuracy. In addition, unlike the human brain that has the ability to carry out new tasks with limited experience, DNNs generally need large amounts of data for training. However, this requirement is not met in many settings, and hence is a limiting factor in the applicability of DNNs in several applications such as smart healthcare. To address these problems we propose neural network synthesis frameworks to generate very compact and accurate DNN architectures, optimize various hyperparameters of the DNN models, and reduce the need for large training datasets. We first introduce a two-step neural network synthesis methodology, called DR+SCANN, that combines two complementary approaches to design compact and accurate DNNs. At the core of our framework is the SCANN methodology that uses three basic architecture-changing operations, namely connection growth, neuron growth, and connection pruning, to synthesize feed-forward architectures with arbitrary structure. Then, we develop the CURIOUS DNN synthesis methodology. It uses a performance predictor to efficiently navigate the architectural search space with an evolutionary search process. To address the need for large amounts of data in training DNN models, we propose the TUTOR DNN synthesis framework. TUTOR relies on generation, verification, and labeling of synthetic data to address this challenge. We then apply the proposed DNN synthesis frameworks to smart healthcare. The emergence of wearable medical sensors (WMSs) alongside efficient DNN models points to a promising solution for the problem of disease diagnosis on edge devices, such as smartphones and smartwatches. In this area, we first propose a framework for repeated large-scale testing of SARS-CoV-2/COVID-19, called CovidDeep. CovidDeep combines efficient DNNs with commercially available WMSs for pervasive testing of the virus and the resultant disease. It achieves the highest test accuracy of 98.1% for a three-way classification among three, cohorts including healthy, asymptomatic (to detect the virus), and symptomatic (to detect the disease) patients. We also propose a framework for mental health disorder diagnosis, called MHDeep. MHDeep utilizes efficient DNN models and data obtained from sensors integrated in a smartwatch and smartphone to diagnose three important mental health disorders: schizoaffective, major depressive, and bipolar