Dream Formulations and Deep Neural Networks: Humanistic Themes in the Iconology of the Machine-Learned Image

This paper addresses the interpretability of deep learning-enabled image recognition processes in computer vision science in relation to theories in art history and cognitive psychology on the vision-related perceptual capabilities of humans. Examination of what is determinable about the machine-learned image in comparison to humanistic theories of visual perception, particularly in regard to art historian Erwin Panofsky's methodology for image analysis and psychologist Eleanor Rosch's theory of graded categorization according to prototypes, finds that there are surprising similarities between the two that suggest that researchers in the arts and the sciences would have much to benefit from closer collaborations. Utilizing the examples of Google's DeepDream and the Machine Learning and Perception Lab at Georgia Tech's Grad-CAM: Gradient-weighted Class Activation Mapping programs, this study suggests that a revival of art historical research in iconography and formalism in the age of AI is essential for shaping the future navigation and interpretation of all machine-learned images, given the rapid developments in image recognition technologies.Comment: 29 pages, 8 Figures, This paper was originally presented as Dream Formulations and Image Recognition: Algorithms for the Study of Renaissance Art, at Critical Approaches to Digital Art History, The Villa I Tatti, The Harvard University Center for Italian Renaissance Studies and The Newberry Center for Renaissance Studies, Renaissance Society of America Annual Meeting, Chicago, 31 March 201

NSAID use and clinical outcomes in COVID-19 patients: a 38-center retrospective cohort study.

BACKGROUND: Non-steroidal anti-inflammatory drugs (NSAIDs) are commonly used to reduce pain, fever, and inflammation but have been associated with complications in community-acquired pneumonia. Observations shortly after the start of the COVID-19 pandemic in 2020 suggested that ibuprofen was associated with an increased risk of adverse events in COVID-19 patients, but subsequent observational studies failed to demonstrate increased risk and in one case showed reduced risk associated with NSAID use. METHODS: A 38-center retrospective cohort study was performed that leveraged the harmonized, high-granularity electronic health record data of the National COVID Cohort Collaborative. A propensity-matched cohort of 19,746 COVID-19 inpatients was constructed by matching cases (treated with NSAIDs at the time of admission) and 19,746 controls (not treated) from 857,061 patients with COVID-19 available for analysis. The primary outcome of interest was COVID-19 severity in hospitalized patients, which was classified as: moderate, severe, or mortality/hospice. Secondary outcomes were acute kidney injury (AKI), extracorporeal membrane oxygenation (ECMO), invasive ventilation, and all-cause mortality at any time following COVID-19 diagnosis. RESULTS: Logistic regression showed that NSAID use was not associated with increased COVID-19 severity (OR: 0.57 95% CI: 0.53-0.61). Analysis of secondary outcomes using logistic regression showed that NSAID use was not associated with increased risk of all-cause mortality (OR 0.51 95% CI: 0.47-0.56), invasive ventilation (OR: 0.59 95% CI: 0.55-0.64), AKI (OR: 0.67 95% CI: 0.63-0.72), or ECMO (OR: 0.51 95% CI: 0.36-0.7). In contrast, the odds ratios indicate reduced risk of these outcomes, but our quantitative bias analysis showed E-values of between 1.9 and 3.3 for these associations, indicating that comparatively weak or moderate confounder associations could explain away the observed associations. CONCLUSIONS: Study interpretation is limited by the observational design. Recording of NSAID use may have been incomplete. Our study demonstrates that NSAID use is not associated with increased COVID-19 severity, all-cause mortality, invasive ventilation, AKI, or ECMO in COVID-19 inpatients. A conservative interpretation in light of the quantitative bias analysis is that there is no evidence that NSAID use is associated with risk of increased severity or the other measured outcomes. Our results confirm and extend analogous findings in previous observational studies using a large cohort of patients drawn from 38 centers in a nationally representative multicenter database

Risk factors associated with post-acute sequelae of SARS-CoV-2: an N3C and NIH RECOVER study

Background More than one-third of individuals experience post-acute sequelae of SARS-CoV-2 infection (PASC, which includes long-COVID). The objective is to identify risk factors associated with PASC/long-COVID diagnosis. Methods This was a retrospective case–control study including 31 health systems in the United States from the National COVID Cohort Collaborative (N3C). 8,325 individuals with PASC (defined by the presence of the International Classification of Diseases, version 10 code U09.9 or a long-COVID clinic visit) matched to 41,625 controls within the same health system and COVID index date within ± 45 days of the corresponding case's earliest COVID index date. Measurements of risk factors included demographics, comorbidities, treatment and acute characteristics related to COVID-19. Multivariable logistic regression, random forest, and XGBoost were used to determine the associations between risk factors and PASC. Results Among 8,325 individuals with PASC, the majority were > 50 years of age (56.6%), female (62.8%), and non-Hispanic White (68.6%). In logistic regression, middle-age categories (40 to 69 years; OR ranging from 2.32 to 2.58), female sex (OR 1.4, 95% CI 1.33–1.48), hospitalization associated with COVID-19 (OR 3.8, 95% CI 3.05–4.73), long (8–30 days, OR 1.69, 95% CI 1.31–2.17) or extended hospital stay (30 + days, OR 3.38, 95% CI 2.45–4.67), receipt of mechanical ventilation (OR 1.44, 95% CI 1.18–1.74), and several comorbidities including depression (OR 1.50, 95% CI 1.40–1.60), chronic lung disease (OR 1.63, 95% CI 1.53–1.74), and obesity (OR 1.23, 95% CI 1.16–1.3) were associated with increased likelihood of PASC diagnosis or care at a long-COVID clinic. Characteristics associated with a lower likelihood of PASC diagnosis or care at a long-COVID clinic included younger age (18 to 29 years), male sex, non-Hispanic Black race, and comorbidities such as substance abuse, cardiomyopathy, psychosis, and dementia. More doctors per capita in the county of residence was associated with an increased likelihood of PASC diagnosis or care at a long-COVID clinic. Our findings were consistent in sensitivity analyses using a variety of analytic techniques and approaches to select controls. Conclusions This national study identified important risk factors for PASC diagnosis such as middle age, severe COVID-19 disease, and specific comorbidities. Further clinical and epidemiological research is needed to better understand underlying mechanisms and the potential role of vaccines and therapeutics in altering PASC course. Supplementary Information The online version contains supplementary material available at 10.1186/s12889-023-16916-w

The National COVID Cohort Collaborative (N3C): Rationale, design, infrastructure, and deployment.

OBJECTIVE: Coronavirus disease 2019 (COVID-19) poses societal challenges that require expeditious data and knowledge sharing. Though organizational clinical data are abundant, these are largely inaccessible to outside researchers. Statistical, machine learning, and causal analyses are most successful with large-scale data beyond what is available in any given organization. Here, we introduce the National COVID Cohort Collaborative (N3C), an open science community focused on analyzing patient-level data from many centers. MATERIALS AND METHODS: The Clinical and Translational Science Award Program and scientific community created N3C to overcome technical, regulatory, policy, and governance barriers to sharing and harmonizing individual-level clinical data. We developed solutions to extract, aggregate, and harmonize data across organizations and data models, and created a secure data enclave to enable efficient, transparent, and reproducible collaborative analytics. RESULTS: Organized in inclusive workstreams, we created legal agreements and governance for organizations and researchers; data extraction scripts to identify and ingest positive, negative, and possible COVID-19 cases; a data quality assurance and harmonization pipeline to create a single harmonized dataset; population of the secure data enclave with data, machine learning, and statistical analytics tools; dissemination mechanisms; and a synthetic data pilot to democratize data access. CONCLUSIONS: The N3C has demonstrated that a multisite collaborative learning health network can overcome barriers to rapidly build a scalable infrastructure incorporating multiorganizational clinical data for COVID-19 analytics. We expect this effort to save lives by enabling rapid collaboration among clinicians, researchers, and data scientists to identify treatments and specialized care and thereby reduce the immediate and long-term impacts of COVID-19

Dream Formulations and Deep Neural Networks: Humanistic Themes in the Iconology of the Machine-Learned Image

This paper addresses the interpretability of deep learning-enabled image recognition processes in computer vision science in relation to theories in art history and cognitive psychology on the vision-related perceptual capabilities of humans. Examination of what is determinable about the machine-learned image in comparison to humanistic theories of visual perception, particularly in regard to art historian Erwin Panofsky’s methodology for image analysis and psychologist Eleanor Rosch’s theory of graded categorization according to prototypes, finds that there are surprising similarities between the two that suggest that researchers in the arts and the sciences would have much to benefit from closer collaborations. Utilizing the examples of Google’s DeepDream and the Machine Learning and Perception Lab at Georgia Tech’s Grad-CAM: Gradient-weighted Class Activation Mapping programs, this study suggests that a revival of art historical research in iconography and formalism in the age of AI is essential for shaping the future navigation and interpretation of all machine-learned images, given the rapid developments in image recognition technologies

The Augmented Dataset: Artistic Appropriations of GANs and their Bearings on Ethical Considerations of Artificial Intelligence

International audienceGenerative Adversarial Networks (GANs) have received much recent attention as they have been employed to falsify information through various media channels and to visually mislead viewers in their interpretation of still images and video. Decried under the rubric of fake news, GANs are often held up as malefactors in the crusade against unethical AI, yet their applications are wide ranging and their potential has yet to be fully realized. This presentation investigates the use of GANs by artists as an alternative to this narrative and considers the role of dataset formation in the Artificial Intelligence artistic process. Since such a significant number of images is required for machine learning systems to function well, the need to augment a dataset is often encountered and how this is overcome plays a considerable role in the final visual form of the GANs-produced image. Indeed, artistic approaches to the hurdle to create new digital images through a repository of so many existing ones offer insights on what constitutes ethical Artificial Intelligence practices. The examples considered include those by notable Artificial Intelligence artists along with a recent project on gastronomic algorithms undertaken by the author with the Chef Alain Passard. Together, these artworks and projects lead one to the question: How can an image that is created through its computational yet obscured connection to a plethora of images be measured at all

Byzantium not Forgotten: Constructing the Artistic and Cultural Legacy of an Empire between East and West in the Early Modern Period

Constantinople, the capital of the Byzantine Empire, fell to the Ottomans on May 29, 1453, a date that has come to symbolize the end of Eastern medieval grandeur in Europe and to inspire a shift in interest to the burgeoning Italian Renaissance. While Byzantium as a political entity had indeed expired by the end of the fifteenth century, its dissolution had begun much earlier. Nonetheless, its cultural legacy, most notably Orthodoxy, tenaciously survived as is evidenced by the uninterrupted tradition of icon painting from the fall of the empire to the Balkan independence movements of the early nineteenth century (and beyond). Largely neglected or examined in isolation due to the limited accessibility of objects from this period and the lesser-known languages in which they have been published, if at all, the material culture of Byzantium as it continued and transformed in the early modern period has received insufficient scholarly attention and has lacked methodological consideration. This study proposes a model of Post-Byzantine art history to address the lacuna in our knowledge of the perpetuation of Byzantine visual culture and the response of Orthodox art to drastically different socio-political and religious contexts than had existed in Byzantium. By examining the various trends in religious art that developed across the former empire’s lands under Venetian, Ottoman, and Slavic rule, the role of Orthodoxy in the historical remembrance of Byzantium is examined, and is demonstrated to have significantly affected the creation of Christian community identities under differing colonial circumstances. In this analysis, modern constructions of Byzantium are challenged and the role of the development of Post-Byzantine iconographies for our understanding of the Byzantine legacy in the early modern world is underscored. By extension, the need for a new resource on Post-Byzantine icons, one that allows for the examination of iconographic types and their dissemination patterns, is made evident. The catalogue of icons organized according to iconographic type presented in Volume Two, which provides the groundwork for Volume One, offers a foundation for a new approach to Post-Byzantine art history

Therapeutic deep brain stimulation disrupts movement-related subthalamic nucleus activity in parkinsonian mice

Subthalamic nucleus deep brain stimulation (STN DBS) relieves many motor symptoms of Parkinson's disease (PD), but its underlying therapeutic mechanisms remain unclear. Since its advent, three major theories have been proposed: (1) DBS inhibits the STN and basal ganglia output; (2) DBS antidromically activates motor cortex; and (3) DBS disrupts firing dynamics within the STN. Previously, stimulation-related electrical artifacts limited mechanistic investigations using electrophysiology. We used electrical artifact-free GCaMP fiber photometry to investigate activity in basal ganglia nuclei during STN DBS in parkinsonian mice. To test whether the observed changes in activity were sufficient to relieve motor symptoms, we then combined electrophysiological recording with targeted optical DBS protocols. Our findings suggest that STN DBS exerts its therapeutic effect through the disruption of movement-related STN activity, rather than inhibition or antidromic activation. These results provide insight into optimizing PD treatments and establish an approach for investigating DBS in other neuropsychiatric conditions