Investigating and Testing Performance Issues in Deep Learning Frameworks

Machine Learning (ML) and Deep Learning (DL) applications are becoming more popular due to the availability of DL frameworks such as PyTorch, Keras, and TensorFlow. Therefore, the quality of DL frameworks is essential to ensure DL/ML application quality. Given the computationally expensive nature of DL tasks (e.g., training), performance is a critical aspect of DL frameworks. However, optimizing DL frameworks may have its own unique challenges due to the peculiarities of DL (e.g., hardware integration and the nature of the computation). In this thesis, we first aim to better understand performance bugs in DL frameworks by conducting an empirical study. We conduct our study on PyTorch and TensorFlow by mining and studying their performance and non-performance bug reports from their respective GitHub repositories. We find that 1) the proportion of newly reported performance bugs increases faster than fixed performance bugs, and the ratio of performance bugs among all bugs increases over time; 2) performance bugs take more time to fix, have larger fix sizes, and more community engagement (e.g., discussion) compared to non-performance bugs; and 3) we manually derived a taxonomy of 12 categories and 19 sub-categories of the root causes of performance bugs in DL frameworks by studying all performance bug fixes. We then aim to investigate the potential of differential testing as a viable technique to detect and prevent performance bugs in DL frameworks. To do so, we train and evaluate two state-of-the-art CNN and RNN architectures (i.e., the Lenet-5 architecture on the MNIST dataset and the LSTM architecture on the IMDB movie review dataset), using different DL frameworks (i.e., PyTorch, Keras, and TensorFlow), and different configurations (i.e., the training dataset sample size, the batch size, the number of epochs, the weight initialization technique, the data type, the hardware used, the learning rate, and the dropout rate). To assess the performance of the DL models, we use a variety of performance metrics (i.e., training/inference time, hardware (CPU or GPU) usage during training/inference, and memory (RAM or GPU VRAM) usage during training/inference). Then, we compare the performance of the DL models across the DL frameworks. We train and evaluate 21,870 Lenet5 models and 21,870 LSTM models across the DL frameworks, for a grand total of 43,740 models. Our experiments took over 42 days. We find that 1) differences in performance between different DL frameworks, for the same task, may be indicative of a performance optimization opportunity/performance bug; 2) our approach is viable when training and evaluating a smaller number of DL models, which makes it more accessible for developers. Finally, we present some potential avenues for future work that aim to further study performance bugs in DL frameworks

Macrophages loaded with gold nanoshells for photothermal ablation of glioma: An in vitro model

The current median survival of patients with glioblastoma multiforme (GBM), the most common type of glioma, remains at 14.6 months despite multimodal treatments (surgery, radiotherapy and chemotherapy). This research aims to study the feasibility of photothermal ablation of glioma using gold nanoshells that are heated upon laser irradiation at their resonance wavelength. The novelty of our approach lies in improving nanoshell tumor delivery by loading them in macrophages, which are known to be recruited to gliomas via tumor-released chemoattractive agents. Ferumoxides, superparamagnetic iron oxide (SPIO) nanoparticles, are needed as an additional macrophage load in order to visualize macrophage accumulation in the tumor with magnetic resonance imaging (MRI) prior to laser irradiation. The feasibility of this approach was studied in an in vitro model of glioma spheroids with the use of continuous wave (CW) laser light for ablation. The optimal loading of both murine and rat macrophages with Ferumoxides was determined using inductively coupled plasma atomic emission spectroscopy (ICP-AES). Higher concentrations of SPIO were observed in rat macrophages, and the optimal concentration was chosen at 100 μg Fe/ml. Macrophages were found to be very sensitive iv to near infra-red (NIR) laser irradiation, and their use as vehicles was thus not expected to hinder the function of loaded nanoshells as tumor-ablating tools. The intracellular presence of gold nanoshells in macrophages was confirmed with TEM imaging. Next, the loading of both murine and rat macrophages with gold nanoshells was studied using UV/Vis spectrophotometry, where higher nanoshell uptake was found in rat macrophages. Incubation of loaded murine and rat macrophages with rat C-6 and human ACBT spheroids, respectively, resulted in their infiltration of the spheroids. Subsequent laser irradiation at 55 W/cm2 for 10 min and follow-up of spheroid average diameter size over 14 days post-irradiation showed that ACBT, but not C-6, spheroids responded to laser-activated nanoshell therapy starting from Day 12. The lack of C-6 response was attributed primarily to the lower nanoshell loading of murine macrophages. Finally, the attempt to double-load macrophages with both Ferumoxides and nanoshells failed under both simultaneous and sequential co-incubation. However, in vivo tracking of nanoshell-loaded macrophages with Ferumoxide is likely feasible using injections containing a mixture of Ferumoxide-loaded and nanoshell-loaded macrophages. Overall, the proof-of-principle studies suggest that photothermal ablation of gliomas via macrophage-mediated delivery of nanoparticles is a promising approach and the work described herein establishes the guidelines and experimental parameters for subsequent in vivo trial

Occurence and management of Chickpea chlorotic dwarf virus in chickpea fields in northern Sudan

A field survey of chickpea for viruses was carried out in River Nile State, northern Sudan, in the 1996– 1997 and 1999–2000 growing seasons. A total of 42 fields (24 in the first season and 18 in the second) were visually inspected. The fields surveyed covered the main areas of chickpea production. On the basis of the main symptoms observed, stunting and yellowing, the range of virus incidence was 7–25% in the first season and 25–62% in the second. Tissue-blot immunoassay (TBIA) of 264 chickpea samples with symptoms, and collected in 1996–2000, indicated that Chickpea chlorotic dwarf virus (CpCDV, Family Geminiviridae) was the most common (72.7%). Field trials at Hudeiba Research Station in northern Sudan in the 1999–2000 and 2000–2001 growing seasons to evaluate the influence of cultivar, sowing date and irrigation interval on the natural spread of CpCDV in chickpea fields revealed that virus incidence in “Shendi” was lower than that in “ICCV-2”, regardless of planting date. Delayed sowing reduced CpCDV incidence in the three growing seasons from 1999 to 2002. Virus incidence was also reduced by short irrigation intervals during the growing seasons 1999–2000 and 2001–2002. Therefore, the combined effect of partial resistance, delayed planting and irrigation at short intervals proved useful in chickpea stunt management in chickpea fields in northern Sudan

Low-cost paper can be used in tissue-blot immunoassay for detection of cereal and legume viruses

In an effort to reduce the cost of virus assays, different types of regular paper were evaluated as possible replacements for the commonly used nitrocellulose membrane (NCM) as the solid phase in the tissue-blot immunoassay (TBIA) used to detect Alfalfa mosaic virus, Bean yellow mosaic virus and Broad bean stain virus (BBSV) in faba bean tissue, and Barley yellow striate mosaic virus in barley tissue. Among the many types of paper evaluated, Hewlett-Packard (HP) non-glossy plotter paper proved to be adequate for detection of all the above viruses. After printing (blotting) of the samples to be tested and blocking with either 2% gelatin (for one hour at 37oC) or using 0.1% Roche blocking reagent (for one hour at room temperature), satisfactory results were produced. This paper could also be used to detect BBSV in groups consisting of 15 young lentil seedlings. HP non-glossy plotter paper was not as effective when testing for phloem-limited legume viruses such as Bean leaf roll virus (BLRV), Faba bean necrotic yellows virus (FBNYV) and Barley yellow dwarf virus (BYDV). White paper (manufactured by Soporcel, Portugal) was slightly more sensitive when used for BLRV and FBNYV detection in faba bean tissues and for BYDV in barley tissues. Since NCM represents 40–50% of the cost of test reagents, using ordinary paper reduced costs considerably

Molecular, serological and biological variation among chickpea chlorotic stunt virus isolates from five countries of North Africa and West Asia

Chickpea chlorotic stunt virus (CpCSV), a proposed new member of the genus Polerovirus (family Luteoviridae), has been reported only from Ethiopia. In attempts to determine the geographical distribution and variability of CpCSV, a pair of degenerate primers derived from conserved domains of the luteovirus coat protein (CP) gene was used for RT-PCR analysis of various legume samples originating from five countries and containing unidentified luteoviruses. Sequencing of the amplicons provided evidence for the occurrence of CpCSV also in Egypt, Morocco, Sudan, and Syria. Phylogenetic analysis of the CP nucleotide sequences of 18 samples from the five countries revealed the existence of two geographic groups of CpCSV isolates differing in CP sequences by 8–10%. Group I included isolates from Ethiopia and Sudan, while group II comprised those from Egypt, Morocco and Syria. For distinguishing these two groups, a simple RFLP test using HindIII and/or PvuII for cleavage of CP-gene-derived PCR products was developed. In ELISA and immunoelectron microscopy, however, isolates from these two groups could not be distinguished with rabbit antisera raised against a group-I isolate from Ethiopia (CpCSV-Eth) and a group-II isolate from Syria (CpCSV-Sy). Since none of the ten monoclonal antibodies (MAbs) that had been produced earlier against CpCSV-Eth reacted with group-II isolates, further MAbs were produced. Of the seven MAbs raised against CpCSV-Sy, two reacted only with CpCSV-Sy and two others with both CpCSV-Sy and -Eth. This indicated that there are group I- and II-specific and common (species-specific) epitopes on the CpCSV CP and that the corresponding MAbs are suitable for specific detection and discrimination of CpCSV isolates. Moreover, CpCSV-Sy (group II) caused more severe stunting and yellowing in faba bean than CpCSV-Eth (group I). In conclusion, our data indicate the existence of a geographically associated variation in the molecular, serological and presumably biological properties of CpCSV

Viruses affecting lentil (Lens culinaris Medik.) in Greece; incidence and genetic variability of Bean leafroll virus and Pea enation mosaic virus

In Greece, lentil (Lens culinaris Medik.) crops are mainly established with non-certified seeds of local landraces, implying high risks for seed transmitted diseases. During April and May of the 2007–2012 growing seasons, surveys were conducted in eight regions of Greece (Attiki, Evros, Fthiotida, Korinthos, Kozani, Larissa, Lefkada and Viotia) to monitor virus incidence in lentil fields. A total of 1216 lentil samples, from plants exhibiting symptoms suggestive of virus infection, were analyzed from 2007 to 2009, using tissue-blot immunoassays (TBIA). Pea seed-borne mosaic virus (PSbMV) overall incidence was 4.9%, followed by Alfalfa mosaic virus (AMV) (2.4%) and Bean yellow mosaic virus (BYMV) (1.0%). When 274 of the samples were tested for the presence of luteoviruses, 38.8% were infected with Bean leafroll virus (BLRV). Since BLRV was not identified in the majority of the samples collected from 2007 to 2009, representative symptomatic plants (360 samples) were collected in further surveys performed from 2010 to 2012 and tested by ELISA. Two viruses prevailed in those samples: BLRV (36.1%) was associated with stunting, yellowing, and reddening symptoms and Pea enation mosaic virus-1 (PEMV-1) (35.0%) was associated with mosaic and mottling symptoms. PSbMV (2.2%), AMV (2.2%), BYMV (3.9%) and CMV (2.8%) were also detected. When the molecular variability was analyzed for representative isolates, collected from the main Greek lentil production areas, five BLRV isolates showed 95% identity for the coat protein (CP) gene and 99% for the 3’ end region. Three Greek PEMV isolates co-clustered with an isolate from Germany when their CP sequence was compared with isolates with no mutation in the aphid transmission gene. Overall, limited genetic variability was detected among Greek isolates of BLRV and PEMV

Survey for legume and cereal viruses in Iraq

A survey was conducted in April 2000 to identify virus diseases affecting legume (chickpea, faba bean and lentil) and cereal (wheat and barley) crops at different locations in Iraq (Baghdad, Al-Anbar, Diyala, At-Tamim and Ninawa governorates). The survey covered 54 randomly selected legume fields (36 faba bean, 8 chickpea, and 10 lentil) and 23 cereal fields (18 wheat and 5 barley). All viruses were identified and their incidence determined by laboratory testing 100–200 randomly collected samples and 20–25 symptomatic samples from each field. A total of 7663 legume and 3455 cereal samples was collected and tested for 12 legume and 5 cereal viruses by tissue blot immunoassay (TBIA). In faba bean fields, Bean yellow mosaic virus (BYMV, genus Potyvirus, family Potyviridae) was the most common, followed by Bean leaf roll virus (BLRV, family Luteoviridae). In the 18 most extensively infected faba bean fields (more than 50%), total virus incidence ranged from 80 to 100%, and BYMV was by far the most common. Other viruses that were detected in legumes at low incidence were: Alfalfa mosaic virus (AlMV, genus Alfamovirus, family Bromoviridae), Faba bean necrotic yellows virus (FBNYV, genus Nanovirus), Cucumber mosaic virus (CMV, genus Cucumovirus, family Bromoviridae), Broad bean wilt virus (BBWV, genus Fabavirus, family Comoviridae), Beet western yellows virus (BWYV, genus Polerovirus, family Luteoviridae) and Chickpea chlorotic dwarf virus (CpCDV, genus Mastervirus, family Geminiviridae). This is the first report of FBNYV, BWYV and CpCDV infecting legumes in Iraq. In the cereal fields, Barley yellow dwarf virus-PAV (BYDV-PAV, genus Luteovirus, family Luteoviridae) was detected in about 1% of random samples. Virus disease incidence in all cereal fields based on field observations was less than 5%. Other viruses were rarely detected in cereals: BYDV-MAV (genus Luteovirus, family Luteoviridae) and Cereal yellow dwarf virus-RPV (CYDV-RPV, genus Polerovirus, family Luteoviridae)

Molecular characterization of a Bean yellow mosaic virus isolate from Syria

Bean yellow mosaic virus (BYMV, genus Potyvirus, family Potyviridae) was studied by comparing sequences from the coat protein (CP) gene of a Syrian isolate with sequences of six other isolates from the NCBI database. A homology tree of the CP sequences was developed using DNAMAN Software. BYMV isolates were grouped into two clusters of which the first comprised the Syrian isolate together with the Indian, Australian and Japanese isolates, and the second the BYMV isolates from China, the Netherlands and the USA. Moreover, the homology tree showed that the Syrian isolate was very close to the Indian one, with 99% homology

Survey of Faba Bean («Vicia faba» L.) Virus Diseases in Ethiopia

Surveys conducted in 1996 and 1997 to assess the status of virus diseases affecting faba bean in the major growing areas of Ethiopia indicated that leaf yellowing, rolling, necrosis and stunting were the most common disease symptoms. The highest visually-observed disease incidence in a field was 85%, recorded in the Wello region (north-eastern Ethiopia). When 3049 symptomatic samples collected from 211 fields from all over Ethiopia were tested by the tissue blot immunoassay (TBIA) for 14 viruses, 1592 samples (52.2%) were found to be infected with at least one virus. Faba bean necrotic yellows virus (FBNYV, genus Nanovirus) was the most frequent (63.2%), followed by luteoviruses (28.5%) [such as Beet western yellows virus (BWYV, genus Polerovirus, family Luteoviridae) and Bean leaf roll virus (BLRV, family Luteoviridae)] and Chickpea chlorotic dwarf virus (CCDV, genus Mastrevirus, family Geminiviridae) (3.1%). Mosaic/mottling symptoms were observed in some fields but incidence was always very low

Survey of Viruses Affecting Legume Crops in the Amhara and Oromia Regions of Ethiopia

Field surveys were undertaken to identify the viral diseases affecting lentil, faba bean, chickpea, pea, fenugreek and grass pea in two regions of Ethiopia. The surveys were conducted in the regions of Amhara (Gonder and Gojam administrative zones) and Oromia (Bale administrative zone) during the 2003/2004 and 2004/2005 growing seasons, respectively. The survey covered 138 randomly selected fields (48 faba bean, 10 pea, 38 grass pea, 34 chickpea, 8 lentil) in the Amhara region, and 51 legume fields (29 faba bean, 12 pea, 3 lentil, 5 fenugreek, 2 chickpea) in the Oromia region. Virus disease incidence was determined by laboratory testing of 100–200 randomly-collected samples from each field against the antisera of 12 legume viruses. Of the 189 fields surveyed, 121 and 7 had, at the time of the survey, a virus disease incidence of 1% or less and more than 6%, respectively, based on visual inspection in the field; later laboratory testing showed that the number of fields in these two categories was in fact 99 and 56, respectively. Serological tests indicated that the most important viruses in the Amhara region were Faba bean necrotic yellows virus (FBNYV), Bean yellow mosaic virus (BYMV), Pea seed-borne mosaic virus (PSbMV) and the luteoviruses [e.g. Beet western yellows virus (BWYV), Bean leaf roll virus (BLRV), Soybean dwarf virus (SbDV)]. By contrast, only FBNYV and the luteoviruses were detected in the Oromia region. Other viruses, such as Broad bean mottle virus (BBMV) and Alfalfa mosaic virus (AMV), were rarely detected in the Amhara region. This is the first report in Ethiopia of natural infection of faba bean, pea and fenugreek with SbDV, of fenugreek with BWYV, and of grass pea with BYMV, PSbMV and BWYV, and it is also the first recorded instance of BBMV infecting legume crops in Ethiopia