Replication study: Development and validation of deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs

Replication studies are essential for validation of new methods, and are crucial to maintain the high standards of scientific publications, and to use the results in practice. We have attempted to replicate the main method in 'Development and validation of a deep learning algorithm for detection of diabetic retinopathy in retinal fundus photographs' published in JAMA 2016; 316(22). We re-implemented the method since the source code is not available, and we used publicly available data sets. The original study used non-public fundus images from EyePACS and three hospitals in India for training. We used a different EyePACS data set from Kaggle. The original study used the benchmark data set Messidor-2 to evaluate the algorithm's performance. We used the same data set. In the original study, ophthalmologists re-graded all images for diabetic retinopathy, macular edema, and image gradability. There was one diabetic retinopathy grade per image for our data sets, and we assessed image gradability ourselves. Hyper-parameter settings were not described in the original study. But some of these were later published. We were not able to replicate the original study. Our algorithm's area under the receiver operating curve (AUC) of 0.94 on the Kaggle EyePACS test set and 0.80 on Messidor-2 did not come close to the reported AUC of 0.99 in the original study. This may be caused by the use of a single grade per image, different data, or different not described hyper-parameter settings. This study shows the challenges of replicating deep learning, and the need for more replication studies to validate deep learning methods, especially for medical image analysis. Our source code and instructions are available at: https://github.com/mikevoets/jama16-retina-replicationComment: The third version of this paper includes results from replication after certain hyper-parameters were published in later article. 16 pages, 6 figures, 1 table, presented at NOBIM 201

nsroot: Minimalist Process Isolation Tool Implemented With Linux Namespaces

Data analyses in the life sciences are moving from tools run on a personal computer to services run on large computing platforms. This creates a need to package tools and dependencies for easy installation, configuration and deployment on distributed platforms. In addition, for secure execution there is a need for process isolation on a shared platform. Existing virtual machine and container technologies are often more complex than traditional Unix utilities, like chroot, and often require root privileges in order to set up or use. This is especially challenging on HPC systems where users typically do not have root access. We therefore present nsroot, a lightweight Linux namespaces based process isolation tool. It allows restricting the runtime environment of data analysis tools that may not have been designed with security as a top priority, in order to reduce the risk and consequences of security breaches, without requiring any special privileges. The codebase of nsroot is small, and it provides a command line interface similar to chroot. It can be used on all Linux kernels that implement user namespaces. In addition, we propose combining nsroot with the AppImage format for secure execution of packaged applications. nsroot is open sourced and available at: https://github.com/uit-no/nsroo

MORTAL - Multiparadigm Optimizing Retargetable Transdisciplinary Abstraction Language

This short paper describes MORTAL, a new general-purpose programming language and compiler for high-performance scientific applications. MORTAL aims to bridge the knowledge gap between computer scientists and scientists by offering a multiparadigm programming environment that allows connecting the mathematical formulae written by scientist to algorithms implemented by the software engineer in a natural way, and understood by both. We provide the rationale for MORTAL, give an overview of the language design and the MORTAL compiler. The compiler is self-hosting, and our initial evaluation shows that MORTAL programs have similar performance as C programs

Work Extraction and Landauer's Principle in a Quantum Spin Hall Device

Landauer's principle states that erasure of each bit of information in a system requires at least a unit of energy $k_B T \ln 2$ to be dissipated. In return, the blank bit may possibly be utilized to extract usable work of the amount $k_B T \ln 2$, in keeping with the second law of thermodynamics. While in principle any collection of spins can be utilized as information storage, work extraction by utilizing this resource in principle requires specialized engines that are capable of using this resource. In this work, we focus on heat and charge transport in a quantum spin Hall device in the presence of a spin bath. We show how a properly initialized nuclear spin subsystem can be used as a memory resource for a Maxwell's Demon to harvest available heat energy from the reservoirs to induce charge current that can power an external electrical load. We also show how to initialize the nuclear spin subsystem using applied bias currents which necessarily dissipate energy, hence demonstrating Landauer's principle. This provides an alternative method of "energy storage" in an all-electrical device. We finally propose a realistic setup to experimentally observe a Landauer erasure/work extraction cycle.Comment: Accepted for publication PRB, 9 pages, 4 figures, RevTe

Convolutional neural network for breathing phase detection in lung sounds

We applied deep learning to create an algorithm for breathing phase detection in lung sound recordings, and we compared the breathing phases detected by the algorithm and manually annotated by two experienced lung sound researchers. Our algorithm uses a convolutional neural network with spectrograms as the features, removing the need to specify features explicitly. We trained and evaluated the algorithm using three subsets that are larger than previously seen in the literature. We evaluated the performance of the method using two methods. First, discrete count of agreed breathing phases (using 50% overlap between a pair of boxes), shows a mean agreement with lung sound experts of 97% for inspiration and 87% for expiration. Second, the fraction of time of agreement (in seconds) gives higher pseudo-kappa values for inspiration (0.73-0.88) than expiration (0.63-0.84), showing an average sensitivity of 97% and an average specificity of 84%. With both evaluation methods, the agreement between the annotators and the algorithm shows human level performance for the algorithm. The developed algorithm is valid for detecting breathing phases in lung sound recordings

The Beauty of Complex Designs

The increasing use of omics data in epidemiology enables many novel study designs, but also introduces challenges for data analysis. We describe the possibilities for systems epidemiological designs in the Norwegian Women and Cancer (NOWAC) study and show how the complexity of NOWAC enables many beautiful new study designs. We discuss the challenges of implementing designs and analyzing data. Finally, we propose a systems architecture for swift design and exploration of epidemiological studies

Cancer detection for white urban Americans

Poster presentation at the NORA Annual Conference 2023 05.06. - 06.06.23, Tromsø, Norway.Development, validation and comparison of machine learning methods require access to data, sometimes lots of data. Within health applications, data sharing can be restricted due to patient privacy, and the few publicly available data sets become even more valuable for the machine learning community. One such type of data are H&E whole slide images (WSI), which are stained tumour tissue, used in hospitals to detect and classify cancer, see Fig. 1. The Cancer Genome Atlas (TCGA) has made an enormous contribution to publicly available data sets. For breast cancer H&E WSI they are by far the largest data set, with more than 1,000 patients, twice as many as the second largest contributor, the two Camelyon competition data sets [1] with 399 + 200 patients

Teaching Electronics and Programming in Norwegian Schools Using the air:bit Sensor Kit

We describe lessons learned from using the air:bit project to introduce more than 150 students in the Norwegian upper secondary school to computer programming, engineering and environmental sciences. In the air:bit project, students build and code a portable air quality sensor kits, and use their air:bit to collect data to investigate patterns in air quality in their local environment. When the project ended students had collected more than 400,000 measurements with their air:bit kits, and could describe local patterns in air quality. Students participate in all parts of the project, from soldering components and programming the sensors, to analyzing the air quality measurements. We conducted a survey after the project and describe our lessons learned from the project. The results show that the project successfully taught the students fundamental concepts in computer programming, electronics, and the scientific method. In addition, all the participating teachers reported that their students had showed good learning outcomes

nsroot: Minimalist process isolation tool implemented with Linux namespaces

services run on large computing platforms.. This creates a need to package tools and dependencies for easy installation,, configuration and deployment on distributed platforms.. In addition,, for secure execution there is a need for process isolation on a shared platform.. Existing virtual machine and container technologies are often more complex than trad itional Unix utilities,, like chroot,, and often require root privileges in order to set up or use.. This is especially challenging on HPC systems where users typically do not have root access.. We therefore present nsroot,, a lightweight Linux namespaces based process isolation tool.. It allows restricting the runtime environment of data analysis tools that may not have been designed with security as a top priority,, in order to reduce the risk and consequences of security breaches,, without requiring any special privileges.. The codebase of nsroot is small,, and it provides a command line interface similar to chroot.. It can be used on all Linux kernels that implement user namespaces.. In addition,, we propose combining nsroot with the AppImage format for secure execu tion of packaged applications.. nsroot is open sourced and available at:: https://github.com/uit-no/nsroot