DeepCell 2.0: Automated cloud deployment of deep learning models for large-scale cellular image analysis

Deep learning is transforming the ability of life scientists to extract information from images. While these techniques have superior accuracy in comparison to conventional approaches and enable previously impossible analyses, their unique hardware and software requirements have prevented widespread adoption by life scientists. To meet this need, we have developed DeepCell 2.0, an open source library for training and delivering deep learning models with cloud computing. This library enables users to configure and manage a cloud deployment of DeepCell 2.0 on all commonly used operating systems. Using single-cell segmentation as a use case, we show that users with suitable training data can train models and analyze data with those models through a web interface. We demonstrate that by matching analysis tasks with their hardware requirements, we can efficiently use computational resources in the cloud and scale those resources to meet demand, significantly reducing the time necessary for large-scale image analysis. By reducing the barriers to entry, this work will empower life scientists to apply deep learning methods to their data. A persistent deployment is available at http://www.deepcell.org

Accurate cell tracking and lineage construction in live-cell imaging experiments with deep learning

Live-cell imaging experiments have opened an exciting window into the behavior of living systems. While these experiments can produce rich data, the computational analysis of these datasets is challenging. Single-cell analysis requires that cells be accurately identified in each image and subsequently tracked over time. Increasingly, deep learning is being used to interpret microscopy image with single cell resolution. In this work, we apply deep learning to the problem of tracking single cells in live-cell imaging data. Using crowdsourcing and a human-in-the-loop approach to data annotation, we constructed a dataset of over 11,000 trajectories of cell nuclei that includes lineage information. Using this dataset, we successfully trained a deep learning model to perform cell tracking within a linear programming framework. Benchmarking tests demonstrate that our method achieves state-of-the-art performance on the task of cell tracking with respect to multiple accuracy metrics. Further, we show that our deep learning-based method generalizes to perform cell tracking for both fluorescent and brightfield images of the cell cytoplasm, despite having never been trained those data types. This enables analysis of live-cell imaging data collected across imaging modalities. A persistent cloud deployment of our cell tracker is available at http://www.deepcell.org

DeepCell 2.0: Automated cloud deployment of deep learning models for large-scale cellular image analysis

Deep learning is transforming the ability of life scientists to extract information from images. While these techniques have superior accuracy in comparison to conventional approaches and enable previously impossible analyses, their unique hardware and software requirements have prevented widespread adoption by life scientists. To meet this need, we have developed DeepCell 2.0, an open source library for training and delivering deep learning models with cloud computing. This library enables users to configure and manage a cloud deployment of DeepCell 2.0 on all commonly used operating systems. Using single-cell segmentation as a use case, we show that users with suitable training data can train models and analyze data with those models through a web interface. We demonstrate that by matching analysis tasks with their hardware requirements, we can efficiently use computational resources in the cloud and scale those resources to meet demand, significantly reducing the time necessary for large-scale image analysis. By reducing the barriers to entry, this work will empower life scientists to apply deep learning methods to their data. A persistent deployment is available at http://www.deepcell.org

Accurate cell tracking and lineage construction in live-cell imaging experiments with deep learning

Live-cell imaging experiments have opened an exciting window into the behavior of living systems. While these experiments can produce rich data, the computational analysis of these datasets is challenging. Single-cell analysis requires that cells be accurately identified in each image and subsequently tracked over time. Increasingly, deep learning is being used to interpret microscopy image with single cell resolution. In this work, we apply deep learning to the problem of tracking single cells in live-cell imaging data. Using crowdsourcing and a human-in-the-loop approach to data annotation, we constructed a dataset of over 11,000 trajectories of cell nuclei that includes lineage information. Using this dataset, we successfully trained a deep learning model to perform cell tracking within a linear programming framework. Benchmarking tests demonstrate that our method achieves state-of-the-art performance on the task of cell tracking with respect to multiple accuracy metrics. Further, we show that our deep learning-based method generalizes to perform cell tracking for both fluorescent and brightfield images of the cell cytoplasm, despite having never been trained those data types. This enables analysis of live-cell imaging data collected across imaging modalities. A persistent cloud deployment of our cell tracker is available at http://www.deepcell.org

Whole-cell segmentation of tissue images with human-level performance using large-scale data annotation and deep learning

Understanding the spatial organization of tissues is of critical importance for both basic and translational research. While recent advances in tissue imaging are opening an exciting new window into the biology of human tissues, interpreting the data that they create is a significant computational challenge. Cell segmentation, the task of uniquely identifying each cell in an image, remains a substantial barrier for tissue imaging, as existing approaches are inaccurate or require a substantial amount of manual curation to yield useful results. Here, we addressed the problem of cell segmentation in tissue imaging data through large-scale data annotation and deep learning. We constructed TissueNet, an image dataset containing >1 million paired whole-cell and nuclear annotations for tissue images from nine organs and six imaging platforms. We created Mesmer, a deep learning-enabled segmentation algorithm trained on TissueNet that performs nuclear and whole-cell segmentation in tissue imaging data. We demonstrated that Mesmer has better speed and accuracy than previous methods, generalizes to the full diversity of tissue types and imaging platforms in TissueNet, and achieves human-level performance for whole-cell segmentation. Mesmer enabled the automated extraction of key cellular features, such as subcellular localization of protein signal, which was challenging with previous approaches. We further showed that Mesmer could be adapted to harness cell lineage information present in highly multiplexed datasets. We used this enhanced version to quantify cell morphology changes during human gestation. All underlying code and models are released with permissive licenses as a community resource

Neurological and psychiatric risk trajectories after SARS-CoV-2 infection: an analysis of 2-year retrospective cohort studies including 1 284 437 patients

Background COVID-19 is associated with increased risks of neurological and psychiatric sequelae in the weeks and months thereafter. How long these risks remain, whether they affect children and adults similarly, and whether SARS-CoV-2 variants differ in their risk profiles remains unclear. Methods In this analysis of 2-year retrospective cohort studies, we extracted data from the TriNetX electronic health records network, an international network of de-identified data from health-care records of approximately 89 million patients collected from hospital, primary care, and specialist providers (mostly from the USA, but also from Australia, the UK, Spain, Bulgaria, India, Malaysia, and Taiwan). A cohort of patients of any age with COVID-19 diagnosed between Jan 20, 2020, and April 13, 2022, was identified and propensity-score matched (1:1) to a contemporaneous cohort of patients with any other respiratory infection. Matching was done on the basis of demographic factors, risk factors for COVID-19 and severe COVID-19 illness, and vaccination status. Analyses were stratified by age group (age <18 years [children], 18–64 years [adults], and ≥65 years [older adults]) and date of diagnosis. We assessed the risks of 14 neurological and psychiatric diagnoses after SARS-CoV-2 infection and compared these risks with the matched comparator cohort. The 2-year risk trajectories were represented by time-varying hazard ratios (HRs) and summarised using the 6-month constant HRs (representing the risks in the earlier phase of follow-up, which have not yet been well characterised in children), the risk horizon for each outcome (ie, the time at which the HR returns to 1), and the time to equal incidence in the two cohorts. We also estimated how many people died after a neurological or psychiatric diagnosis during follow-up in each age group. Finally, we compared matched cohorts of patients diagnosed with COVID-19 directly before and after the emergence of the alpha (B.1.1.7), delta (B.1.617.2), and omicron (B.1.1.529) variants. Findings We identified 1 487 712 patients with a recorded diagnosis of COVID-19 during the study period, of whom 1 284 437 (185 748 children, 856 588 adults, and 242 101 older adults; overall mean age 42·5 years [SD 21·9]; 741 806 [57·8%] were female and 542 192 [42·2%] were male) were adequately matched with an equal number of patients with another respiratory infection. The risk trajectories of outcomes after SARS-CoV-2 infection in the whole cohort differed substantially. While most outcomes had HRs significantly greater than 1 after 6 months (with the exception of encephalitis; Guillain-Barré syndrome; nerve, nerve root, and plexus disorder; and parkinsonism), their risk horizons and time to equal incidence varied greatly. Risks of the common psychiatric disorders returned to baseline after 1–2 months (mood disorders at 43 days, anxiety disorders at 58 days) and subsequently reached an equal overall incidence to the matched comparison group (mood disorders at 457 days, anxiety disorders at 417 days). By contrast, risks of cognitive deficit (known as brain fog), dementia, psychotic disorders, and epilepsy or seizures were still increased at the end of the 2-year follow-up period. Post-COVID-19 risk trajectories differed in children compared with adults: in the 6 months after SARS-CoV-2 infection, children were not at an increased risk of mood (HR 1·02 [95% CI 0·94–1·10) or anxiety (1·00 [0·94–1·06]) disorders, but did have an increased risk of cognitive deficit, insomnia, intracranial haemorrhage, ischaemic stroke, nerve, nerve root, and plexus disorders, psychotic disorders, and epilepsy or seizures (HRs ranging from 1·20 [1·09–1·33] to 2·16 [1·46–3·19]). Unlike adults, cognitive deficit in children had a finite risk horizon (75 days) and a finite time to equal incidence (491 days). A sizeable proportion of older adults who received a neurological or psychiatric diagnosis, in either cohort, subsequently died, especially those diagnosed with dementia or epilepsy or seizures. Risk profiles were similar just before versus just after the emergence of the alpha variant (n=47 675 in each cohort). Just after (vs just before) the emergence of the delta variant (n=44 835 in each cohort), increased risks of ischaemic stroke, epilepsy or seizures, cognitive deficit, insomnia, and anxiety disorders were observed, compounded by an increased death rate. With omicron (n=39 845 in each cohort), there was a lower death rate than just before emergence of the variant, but the risks of neurological and psychiatric outcomes remained similar. Interpretation This analysis of 2-year retrospective cohort studies of individuals diagnosed with COVID-19 showed that the increased incidence of mood and anxiety disorders was transient, with no overall excess of these diagnoses compared with other respiratory infections. In contrast, the increased risk of psychotic disorder, cognitive deficit, dementia, and epilepsy or seizures persisted throughout. The differing trajectories suggest a different pathogenesis for these outcomes. Children have a more benign overall profile of psychiatric risk than do adults and older adults, but their sustained higher risk of some diagnoses is of concern. The fact that neurological and psychiatric outcomes were similar during the delta and omicron waves indicates that the burden on the health-care system might continue even with variants that are less severe in other respects. Our findings are relevant to understanding individual-level and population-level risks of neurological and psychiatric disorders after SARS-CoV-2 infection and can help inform our responses to them. Funding National Institute for Health and Care Research Oxford Health Biomedical Research Centre, The Wolfson Foundation, and MQ Mental Health Research

Neurological and psychiatric risk trajectories after SARS-CoV-2 infection: an analysis of 2-year retrospective cohort studies including 1 284 437 patients.

BACKGROUND: COVID-19 is associated with increased risks of neurological and psychiatric sequelae in the weeks and months thereafter. How long these risks remain, whether they affect children and adults similarly, and whether SARS-CoV-2 variants differ in their risk profiles remains unclear. METHODS: In this analysis of 2-year retrospective cohort studies, we extracted data from the TriNetX electronic health records network, an international network of de-identified data from health-care records of approximately 89 million patients collected from hospital, primary care, and specialist providers (mostly from the USA, but also from Australia, the UK, Spain, Bulgaria, India, Malaysia, and Taiwan). A cohort of patients of any age with COVID-19 diagnosed between Jan 20, 2020, and April 13, 2022, was identified and propensity-score matched (1:1) to a contemporaneous cohort of patients with any other respiratory infection. Matching was done on the basis of demographic factors, risk factors for COVID-19 and severe COVID-19 illness, and vaccination status. Analyses were stratified by age group (age <18 years [children], 18-64 years [adults], and ≥65 years [older adults]) and date of diagnosis. We assessed the risks of 14 neurological and psychiatric diagnoses after SARS-CoV-2 infection and compared these risks with the matched comparator cohort. The 2-year risk trajectories were represented by time-varying hazard ratios (HRs) and summarised using the 6-month constant HRs (representing the risks in the earlier phase of follow-up, which have not yet been well characterised in children), the risk horizon for each outcome (ie, the time at which the HR returns to 1), and the time to equal incidence in the two cohorts. We also estimated how many people died after a neurological or psychiatric diagnosis during follow-up in each age group. Finally, we compared matched cohorts of patients diagnosed with COVID-19 directly before and after the emergence of the alpha (B.1.1.7), delta (B.1.617.2), and omicron (B.1.1.529) variants. FINDINGS: We identified 1 487 712 patients with a recorded diagnosis of COVID-19 during the study period, of whom 1 284 437 (185 748 children, 856 588 adults, and 242 101 older adults; overall mean age 42·5 years [SD 21·9]; 741 806 [57·8%] were female and 542 192 [42·2%] were male) were adequately matched with an equal number of patients with another respiratory infection. The risk trajectories of outcomes after SARS-CoV-2 infection in the whole cohort differed substantially. While most outcomes had HRs significantly greater than 1 after 6 months (with the exception of encephalitis; Guillain-Barré syndrome; nerve, nerve root, and plexus disorder; and parkinsonism), their risk horizons and time to equal incidence varied greatly. Risks of the common psychiatric disorders returned to baseline after 1-2 months (mood disorders at 43 days, anxiety disorders at 58 days) and subsequently reached an equal overall incidence to the matched comparison group (mood disorders at 457 days, anxiety disorders at 417 days). By contrast, risks of cognitive deficit (known as brain fog), dementia, psychotic disorders, and epilepsy or seizures were still increased at the end of the 2-year follow-up period. Post-COVID-19 risk trajectories differed in children compared with adults: in the 6 months after SARS-CoV-2 infection, children were not at an increased risk of mood (HR 1·02 [95% CI 0·94-1·10) or anxiety (1·00 [0·94-1·06]) disorders, but did have an increased risk of cognitive deficit, insomnia, intracranial haemorrhage, ischaemic stroke, nerve, nerve root, and plexus disorders, psychotic disorders, and epilepsy or seizures (HRs ranging from 1·20 [1·09-1·33] to 2·16 [1·46-3·19]). Unlike adults, cognitive deficit in children had a finite risk horizon (75 days) and a finite time to equal incidence (491 days). A sizeable proportion of older adults who received a neurological or psychiatric diagnosis, in either cohort, subsequently died, especially those diagnosed with dementia or epilepsy or seizures. Risk profiles were similar just before versus just after the emergence of the alpha variant (n=47 675 in each cohort). Just after (vs just before) the emergence of the delta variant (n=44 835 in each cohort), increased risks of ischaemic stroke, epilepsy or seizures, cognitive deficit, insomnia, and anxiety disorders were observed, compounded by an increased death rate. With omicron (n=39 845 in each cohort), there was a lower death rate than just before emergence of the variant, but the risks of neurological and psychiatric outcomes remained similar. INTERPRETATION: This analysis of 2-year retrospective cohort studies of individuals diagnosed with COVID-19 showed that the increased incidence of mood and anxiety disorders was transient, with no overall excess of these diagnoses compared with other respiratory infections. In contrast, the increased risk of psychotic disorder, cognitive deficit, dementia, and epilepsy or seizures persisted throughout. The differing trajectories suggest a different pathogenesis for these outcomes. Children have a more benign overall profile of psychiatric risk than do adults and older adults, but their sustained higher risk of some diagnoses is of concern. The fact that neurological and psychiatric outcomes were similar during the delta and omicron waves indicates that the burden on the health-care system might continue even with variants that are less severe in other respects. Our findings are relevant to understanding individual-level and population-level risks of neurological and psychiatric disorders after SARS-CoV-2 infection and can help inform our responses to them. FUNDING: National Institute for Health and Care Research Oxford Health Biomedical Research Centre, The Wolfson Foundation, and MQ Mental Health Research