Hardware and Software Optimizations for Accelerating Deep Neural Networks: Survey of Current Trends, Challenges, and the Road Ahead

Currently, Machine Learning (ML) is becoming ubiquitous in everyday life. Deep Learning (DL) is already present in many applications ranging from computer vision for medicine to autonomous driving of modern cars as well as other sectors in security, healthcare, and finance. However, to achieve impressive performance, these algorithms employ very deep networks, requiring a significant computational power, both during the training and inference time. A single inference of a DL model may require billions of multiply-and-accumulated operations, making the DL extremely compute-and energy-hungry. In a scenario where several sophisticated algorithms need to be executed with limited energy and low latency, the need for cost-effective hardware platforms capable of implementing energy-efficient DL execution arises. This paper first introduces the key properties of two brain-inspired models like Deep Neural Network (DNN), and Spiking Neural Network (SNN), and then analyzes techniques to produce efficient and high-performance designs. This work summarizes and compares the works for four leading platforms for the execution of algorithms such as CPU, GPU, FPGA and ASIC describing the main solutions of the state-of-the-art, giving much prominence to the last two solutions since they offer greater design flexibility and bear the potential of high energy-efficiency, especially for the inference process. In addition to hardware solutions, this paper discusses some of the important security issues that these DNN and SNN models may have during their execution, and offers a comprehensive section on benchmarking, explaining how to assess the quality of different networks and hardware systems designed for them

The modified Glasgow prognostic score in prostate cancer: results from a retrospective clinical series of 744 patients

<p>Background: As the incidence of prostate cancer continues to rise steeply, there is an increasing need to identify more accurate prognostic markers for the disease. There is some evidence that a higher modified Glasgow Prognostic Score (mGPS) may be associated with poorer survival in patients with prostate cancer but it is not known whether this is independent of other established prognostic factors. Therefore the aim of this study was to describe the relationship between mGPS and survival in patients with prostate cancer after adjustment for other prognostic factors.</p> <p>Methods: Retrospective clinical series on patients in Glasgow, Scotland, for whom data from the Scottish Cancer Registry, including Gleason score, Prostate Specific Antigen (PSA), C-reactive protein (CRP) and albumin, six months prior to or following the diagnosis, were included in this study.</p> <p>The mGPS was constructed by combining CRP and albumin. Five-year and ten-year relative survival and relative excess risk of death were estimated by mGPS categories after adjusting for age, socioeconomic circumstances, Gleason score, PSA and previous in-patient bed days.</p> <p>Results: Seven hundred and forty four prostate cancer patients were identified; of these, 497 (66.8%) died during a maximum follow up of 11.9 years. Patients with mGPS of 2 had poorest 5-year and 10-year relative survival, of 32.6% and 18.8%, respectively. Raised mGPS also had a significant association with excess risk of death at five years (mGPS 2: Relative Excess Risk = 3.57, 95% CI 2.31-5.52) and ten years (mGPS 2: Relative Excess Risk = 3.42, 95% CI 2.25-5.21) after adjusting for age, socioeconomic circumstances, Gleason score, PSA and previous in-patient bed days.</p> <p>Conclusions: The mGPS is an independent and objective prognostic indicator for survival of patients with prostate cancer. It may be useful in determining the clinical management of patients with prostate cancer in addition to established prognostic markers.</p&gt

Blunt abdominal trauma: the experience in rural India and review of literature

Background: Blunt Abdominal trauma is one of the most common injuries caused due to road traffic accidents. The rapid increase in number of motor vehicles and its aftermath has caused rapid increase in number of victims due to blunt abdominal trauma. As the care of patients with blunt abdominal injuries is largely a surgical responsibility and abdominal injuries involving major hemorrhage from solid viscera constitute surgical emergencies. Abdominal blunt traumas represent a real diagnostic and therapeutic challenge to even a most experienced surgeon, thereby representing importance of its study. Early diagnosis and effective management help in decreasing mortality in blunt abdominal trauma.Methods: Prospective study of 50 patients admitted to the institute with history of Blunt Abdominal Trauma. After initial resuscitation of the patients, thorough assessments for injuries were carried out in all the patients. Documentation of patients, which included identification, history, clinical findings, diagnostic test, operative findings, operative procedures and complications during the stay in the hospital were all recorded on a Performa specially prepared. The management was decided depending upon history, clinical examination and investigations.Results: Males were predominantly affected, and most cases were between the age group of 21-40 years (76%). Majority of the patients (90%) presented with the complaint of abdominal pain followed by abdominal distension (56%). 36(60%) patients were managed conservatively while operative interventions were required in 24(40%) patients. The common surgeries performed in the patients included splenectomy, primary closure of perforation and resection and anastomosis of bowel. Majority of the patients (80%) were discharged within 20 days of admission. The mortality in present study was 13.3%.Conclusions: Blunt Abdominal Trauma is one of the important causes of morbidity and mortality in young adults. Immediate resuscitative measures, management of associated injuries and appropriate operative intervention are important parts of management of such cases

Ab initio prediction of semiconductivity in a novel two-dimensional Sb2X3 (X= S, Se, Te) monolayers with orthorhombic structure

Sb 2S 3 and Sb 2Se 3 are well-known layered bulk structures with weak van der Waals interactions. In this work we explore the atomic lattice, dynamical stability, electronic and optical properties of Sb 2S 3, Sb 2Se 3 and Sb 2Te 3 monolayers using the density functional theory simulations. Molecular dynamics and phonon dispersion results show the desirable thermal and dynamical stability of studied nanosheets. On the basis of HSE06 and PBE/GGA functionals, we show that all the considered novel monolayers are semiconductors. Using the HSE06 functional the electronic bandgap of Sb 2S 3, Sb 2Se 3 and Sb 2Te 3 monolayers are predicted to be 2.15, 1.35 and 1.37 eV, respectively. Optical simulations show that the first absorption coefficient peak for Sb 2S 3, Sb 2Se 3 and Sb 2Te 3 monolayers along in-plane polarization is suitable for the absorption of the visible and IR range of light. Interestingly, optically anisotropic character along planar directions can be desirable for polarization-sensitive photodetectors. Furthermore, we systematically investigate the electrical transport properties with combined first-principles and Boltzmann transport theory calculations. At optimal doping concentration, we found the considerable larger power factor values of 2.69, 4.91, and 5.45 for hole-doped Sb 2S 3, Sb 2Se 3, and Sb 2Te 3, respectively. This study highlights the bright prospect for the application of Sb 2S 3, Sb 2Se 3 and Sb 2Te 3 nanosheets in novel electronic, optical and energy conversion systems. © 2021, The Author(s)

R&D intensity and firms dividend policy: evidence from BRICS countries

Purpose: Given the importance of both research and development (R&D) investments and dividend policy in the growth of firms, this paper examines the moderating effects of investor protection and other country-level governance mechanisms on the relationship between R&D investments and dividend payments in the firms from Brazil, Russia, India, China and South Africa (BRICS countries). Design/methodology/approach: This empirical study uses a sample of 22,073 firm year observations from the BRICS countries over a period of 2008–2020 and employs both ordinary least squared (OLS) and system generalized method of moments (GMM) estimation methods. The GMM estimation controls for unobservable heterogeneity and endogeneity and reduces estimation bias. Findings: The findings indicate that although R&D intensity is negatively related with the cash dividend payments, with the interaction of investor protection and other country-level mechanisms the relationship between R&D intensity and dividend payments becomes positive. The results further show that investor protection has stronger impact on the relationship between R&D intensity and firm cash dividend payments than other selected country-level governance factors. Practical implications: The research findings should encourage the policy makers in BRICS countries to strengthen investor protection and enhance quality of their institutions to make a right balance between retaining their growth potential and maintaining the value of the firms. Originality/value: This is the first study to provide evidence of the moderating effects of investor protection and other country-level governance mechanisms on the relationship between R&D investments and dividend payments using the data from BRICS countries

Monitoring and evaluation of irrigation and drainage facilities for pilot distributaries in Sindh Province, Pakistan. Volume 4 - Heran Distributary, Sanghar District

Irrigation management / Monitoring / Evaluation / Irrigation canals / Distributary canals / Drainage / Irrigation practices / Water delivery / Watercourses / Maintenance / Water table / Groundwater / Water quality / Pakistan / Sindh Province / Sanghar District / Heran Distributary

In vivo investigation of ear canal pulse oximetry during hypothermia

Pulse oximeters rely on the technique of photoplethysmography (PPG) to estimate arterial oxygen saturation (SpO(Formula presented.)). In conditions of poor peripheral perfusion such as hypotension, hypothermia, and vasoconstriction, the PPG signals detected are often weak and noisy, or in some cases unobtainable. Hence, pulse oximeters produce erroneous SpO(Formula presented.) readings in these circumstances. The problem arises as most commercial pulse oximeter probes are designed to be attached to peripheral sites such as the finger or toe, which are easily affected by vasoconstriction. In order to overcome this problem, the ear canal was investigated as an alternative site for measuring reliable SpO(Formula presented.) on the hypothesis that blood flow to this central site is preferentially preserved. A novel miniature ear canal PPG sensor was developed along with a state of the art PPG processing unit to investigate PPG measurements from the bottom surface of the ear canal. An in vivo study was carried out in 15 healthy volunteers to validate the developed technology. In this comparative study, red and infrared PPGs were acquired from the ear canal and the finger of the volunteers, whilst they were undergoing artificially induced hypothermia by means of cold exposure (10 (Formula presented.)C). Normalised Pulse Amplitude (NPA) and SpO(Formula presented.) was calculated from the PPG signals acquired from the ear canal and the finger. Good quality baseline PPG signals with high signal-to-noise ratio were obtained from both the PPG sensors. During cold exposure, significant differences were observed in the NPA of the finger PPGs. The mean NPA of the red and infrared PPGs from the finger have dropped by >80%. Contrary to the finger, the mean NPA of red and infrared ear canal PPGs had dropped only by 0.2 and 13% respectively. The SpO(Formula presented.)s estimated from the finger sensor have dropped below 90% in five volunteers (failure) by the end of the cold exposure. The ear canal sensor, on the other hand, had only failed in one volunteer. These results strongly suggest that the ear canal may be used as a suitable alternative site for monitoring PPGs and arterial blood oxygen saturation at times were peripheral perfusion is compromised

Evaluation of economic loss caused by Indian crested porcupine (Hystrix indica) in agricultural land of district Muzaffarabad, Azad Jammu and Kashmir, Pakistan

The Indian crested porcupine (Hystrix indica) is a vertebrate pest of agricultural lands and forest. The study was aimed to report the damage to local crops by the Indian crested porcupine (Hystrix indica) in the Muzaffarabad District. A survey was conducted to identify the porcupine-affected areas and assess the crop damage to the local farmers in district Muzaffarabad Azad Jammu and Kashmir (AJK) from May 2017 to October 2017. Around 19 villages were surveyed, and a sum of 191 semi-structured questionnaires was distributed among farmers. Crop damage was found highest in village Dhanni where a porcupine destroyed 175 Kg/Kanal of the crops. Regarding the total magnitude of crop loss, village Danna and Koomi kot were the most affected areas. More than half (51.8%) of the respondents in the study area suffered the economic loss within the range of 101-200$, and (29.8$ annually. Among all crops, maize (Zea mays) was found to be the most damaged crop ranging between 1-300 Kg annually. In the study area, porcupine also inflicted a lot of damages to some important vegetables, including spinach (Spinacia oleracea), potato (Solanum tuberosum) and onion (Allium cepa). It was estimated that, on average, 511Kg of vegetables are destroyed by porcupine every year in the agricultural land of Muzaffarabad. It was concluded that the Indian crested porcupine has a devastating effect on agriculture which is an important source of income and food for the local community. Developing an effective pest control strategy with the help of the local government and the Wildlife department could help the farmers to overcome this problem

NASCaps: A Framework for Neural Architecture Search to Optimize the Accuracy and Hardware Efficiency of Convolutional Capsule Networks

Deep Neural Networks (DNNs) have made significant improvements to reach the desired accuracy to be employed in a wide variety of Machine Learning (ML) applications. Recently the Google Brain's team demonstrated the ability of Capsule Networks (CapsNets) to encode and learn spatial correlations between different input features, thereby obtaining superior learning capabilities compared to traditional (i.e., non-capsule based) DNNs. However, designing CapsNets using conventional methods is a tedious job and incurs significant training effort. Recent studies have shown that powerful methods to automatically select the best/optimal DNN model configuration for a given set of applications and a training dataset are based on the Neural Architecture Search (NAS) algorithms. Moreover, due to their extreme computational and memory requirements, DNNs are employed using the specialized hardware accelerators in IoT-Edge/CPS devices. In this paper, we propose NASCaps, an automated framework for the hardware-aware NAS of different types of DNNs, covering both traditional convolutional DNNs and CapsNets. We study the efficacy of deploying a multi-objective Genetic Algorithm (e.g., based on the NSGA-II algorithm). The proposed framework can jointly optimize the network accuracy and the corresponding hardware efficiency, expressed in terms of energy, memory, and latency of a given hardware accelerator executing the DNN inference. Besides supporting the traditional DNN layers, our framework is the first to model and supports the specialized capsule layers and dynamic routing in the NAS-flow. We evaluate our framework on different datasets, generating different network configurations, and demonstrate the tradeoffs between the different output metrics. We will open-source the complete framework and configurations of the Pareto-optimal architectures at https://github.com/ehw-fit/nascaps.Comment: To appear at the IEEE/ACM International Conference on Computer-Aided Design (ICCAD '20), November 2-5, 2020, Virtual Event, US