A General-Purpose Transferable Predictor for Neural Architecture Search

Understanding and modelling the performance of neural architectures is key to Neural Architecture Search (NAS). Performance predictors have seen widespread use in low-cost NAS and achieve high ranking correlations between predicted and ground truth performance in several NAS benchmarks. However, existing predictors are often designed based on network encodings specific to a predefined search space and are therefore not generalizable to other search spaces or new architecture families. In this paper, we propose a general-purpose neural predictor for NAS that can transfer across search spaces, by representing any given candidate Convolutional Neural Network (CNN) with a Computation Graph (CG) that consists of primitive operators. We further combine our CG network representation with Contrastive Learning (CL) and propose a graph representation learning procedure that leverages the structural information of unlabeled architectures from multiple families to train CG embeddings for our performance predictor. Experimental results on NAS-Bench-101, 201 and 301 demonstrate the efficacy of our scheme as we achieve strong positive Spearman Rank Correlation Coefficient (SRCC) on every search space, outperforming several Zero-Cost Proxies, including Synflow and Jacov, which are also generalizable predictors across search spaces. Moreover, when using our proposed general-purpose predictor in an evolutionary neural architecture search algorithm, we can find high-performance architectures on NAS-Bench-101 and find a MobileNetV3 architecture that attains 79.2% top-1 accuracy on ImageNet.Comment: Accepted to SDM2023; version includes supplementary material; 12 Pages, 3 Figures, 6 Table

AIO-P: Expanding Neural Performance Predictors Beyond Image Classification

Evaluating neural network performance is critical to deep neural network design but a costly procedure. Neural predictors provide an efficient solution by treating architectures as samples and learning to estimate their performance on a given task. However, existing predictors are task-dependent, predominantly estimating neural network performance on image classification benchmarks. They are also search-space dependent; each predictor is designed to make predictions for a specific architecture search space with predefined topologies and set of operations. In this paper, we propose a novel All-in-One Predictor (AIO-P), which aims to pretrain neural predictors on architecture examples from multiple, separate computer vision (CV) task domains and multiple architecture spaces, and then transfer to unseen downstream CV tasks or neural architectures. We describe our proposed techniques for general graph representation, efficient predictor pretraining and knowledge infusion techniques, as well as methods to transfer to downstream tasks/spaces. Extensive experimental results show that AIO-P can achieve Mean Absolute Error (MAE) and Spearman's Rank Correlation (SRCC) below 1% and above 0.5, respectively, on a breadth of target downstream CV tasks with or without fine-tuning, outperforming a number of baselines. Moreover, AIO-P can directly transfer to new architectures not seen during training, accurately rank them and serve as an effective performance estimator when paired with an algorithm designed to preserve performance while reducing FLOPs.Comment: AAAI 2023 Oral Presentation; version includes supplementary material; 16 Pages, 4 Figures, 22 Table

GENNAPE: Towards Generalized Neural Architecture Performance Estimators

Predicting neural architecture performance is a challenging task and is crucial to neural architecture design and search. Existing approaches either rely on neural performance predictors which are limited to modeling architectures in a predefined design space involving specific sets of operators and connection rules, and cannot generalize to unseen architectures, or resort to zero-cost proxies which are not always accurate. In this paper, we propose GENNAPE, a Generalized Neural Architecture Performance Estimator, which is pretrained on open neural architecture benchmarks, and aims to generalize to completely unseen architectures through combined innovations in network representation, contrastive pretraining, and fuzzy clustering-based predictor ensemble. Specifically, GENNAPE represents a given neural network as a Computation Graph (CG) of atomic operations which can model an arbitrary architecture. It first learns a graph encoder via Contrastive Learning to encourage network separation by topological features, and then trains multiple predictor heads, which are soft-aggregated according to the fuzzy membership of a neural network. Experiments show that GENNAPE pretrained on NAS-Bench-101 can achieve superior transferability to 5 different public neural network benchmarks, including NAS-Bench-201, NAS-Bench-301, MobileNet and ResNet families under no or minimum fine-tuning. We further introduce 3 challenging newly labelled neural network benchmarks: HiAML, Inception and Two-Path, which can concentrate in narrow accuracy ranges. Extensive experiments show that GENNAPE can correctly discern high-performance architectures in these families. Finally, when paired with a search algorithm, GENNAPE can find architectures that improve accuracy while reducing FLOPs on three families.Comment: AAAI 2023 Oral Presentation; includes supplementary materials with more details on introduced benchmarks; 14 Pages, 6 Figures, 10 Table

The Role of Whole Blood Impedance Aggregometry and Its Utilisation in the Diagnosis and Prognosis of Patients with Systemic Inflammatory Response Syndrome and Sepsis in Acute Critical Illness

Objective: To assess the prognostic and diagnostic value of whole blood impedance aggregometry in patients with sepsis and SIRS and to compare with whole blood parameters (platelet count, haemoglobin, haematocrit and white cell count). Methods: We performed an observational, prospective study in the acute setting. Platelet function was determined using whole blood impedance aggregometry (multiplate) on admission to the Emergency Department or Intensive Care Unit and at 6 and 24 hours post admission. Platelet count, haemoglobin, haematocrit and white cell count were also determined. Results: 106 adult patients that met SIRS and sepsis criteria were included. Platelet aggregation was significantly reduced in patients with severe sepsis/septic shock when compared to SIRS/uncomplicated sepsis (ADP: 90.7±37.6 vs 61.4±40.6; p<0.001, Arachadonic Acid 99.9±48.3 vs 66.3±50.2; p = 0.001, Collagen 102.6±33.0 vs 79.1±38.8; p = 0.001; SD ± mean)). Furthermore platelet aggregation was significantly reduced in the 28 day mortality group when compared with the survival group (Arachadonic Acid 58.8±47.7 vs 91.1±50.9; p<0.05, Collagen 36.6±36.6 vs 98.0±35.1; p = 0.001; SD ± mean)). However haemoglobin, haematocrit and platelet count were more effective at distinguishing between subgroups and were equally effective indicators of prognosis. Significant positive correlations were observed between whole blood impedance aggregometry and platelet count (ADP 0.588 p<0.0001, Arachadonic Acid 0.611 p<0.0001, Collagen 0.599 p<0.0001 (Pearson correlation)). Conclusions: Reduced platelet aggregometry responses were not only significantly associated with morbidity and mortality in sepsis and SIRS patients, but also correlated with the different pathological groups. Whole blood aggregometry significantly correlated with platelet count, however, when we adjust for the different groups we investigated, the effect of platelet count appears to be non-significant

LSST: from Science Drivers to Reference Design and Anticipated Data Products

(Abridged) We describe here the most ambitious survey currently planned in the optical, the Large Synoptic Survey Telescope (LSST). A vast array of science will be enabled by a single wide-deep-fast sky survey, and LSST will have unique survey capability in the faint time domain. The LSST design is driven by four main science themes: probing dark energy and dark matter, taking an inventory of the Solar System, exploring the transient optical sky, and mapping the Milky Way. LSST will be a wide-field ground-based system sited at Cerro Pach\'{o}n in northern Chile. The telescope will have an 8.4 m (6.5 m effective) primary mirror, a 9.6 deg$^2$ field of view, and a 3.2 Gigapixel camera. The standard observing sequence will consist of pairs of 15-second exposures in a given field, with two such visits in each pointing in a given night. With these repeats, the LSST system is capable of imaging about 10,000 square degrees of sky in a single filter in three nights. The typical 5$\sigma$ point-source depth in a single visit in $r$ will be $\sim 24.5$ (AB). The project is in the construction phase and will begin regular survey operations by 2022. The survey area will be contained within 30,000 deg$^2$ with $\delta<+34.5^\circ$, and will be imaged multiple times in six bands, $ugrizy$, covering the wavelength range 320--1050 nm. About 90\% of the observing time will be devoted to a deep-wide-fast survey mode which will uniformly observe a 18,000 deg$^2$ region about 800 times (summed over all six bands) during the anticipated 10 years of operations, and yield a coadded map to $r\sim27.5$. The remaining 10\% of the observing time will be allocated to projects such as a Very Deep and Fast time domain survey. The goal is to make LSST data products, including a relational database of about 32 trillion observations of 40 billion objects, available to the public and scientists around the world.Comment: 57 pages, 32 color figures, version with high-resolution figures available from https://www.lsst.org/overvie

The effect of sepsis and its inflammatory response on mechanical clot characteristics: a prospective observational study

Purpose: Sepsis and its progression are known to have a major influence on the coagulation system. Current coagulation tests are of limited use when assessing coagulation in sepsis patients. This study aims to assess the potential for a new functional biomarker of clot microstructure, fractal dimension, to identify changes in the mechanical properties of clot microstructure across the sepsis spectrum (sepsis, severe sepsis and septic shock). Methods: A total of 100 patients that presented acutely to a large teaching hospital were included in this prospective observational study (50 sepsis, 20 severe sepsis and 30 septic shock) against a matched control of 44 healthy volunteers. Fractal analysis was performed, as well as standard markers of coagulation, and six plasma markers of inflammation. Results: Fractal dimension was significantly higher in the sepsis and severe sepsis groups than the healthy control (1.78 ± 0.07 and 1.80 ± 0.05 respectively vs 1.74 ± 0.03) (p < 0.001), indicating a significant increase in mechanical clot strength and elasticity consistent with a hypercoagulable state. Conversely, fractal dimension was significantly lower in septic shock (1.66 ± 0.10, p < 0.001), indicating a significant reduction in mechanical clot strength and functionality consistent with a hypocoagulable state. This corresponded with a significant increase in the inflammatory response. Conclusions: This study confirms that clot microstructure is significantly altered through the various stages of sepsis. Of particular importance was the marked change in clot development between severe sepsis and septic shock, which has not been previously reported

Sustained proliferation in cancer: mechanisms and novel therapeutic targets

Proliferation is an important part of cancer development and progression. This is manifest by altered expression and/or activity of cell cycle related proteins. Constitutive activation of many signal transduction pathways also stimulates cell growth. Early steps in tumor development are associated with a fibrogenic response and the development of a hypoxic environment which favors the survival and proliferation of cancer stem cells. Part of the survival strategy of cancer stem cells may manifested by alterations in cell metabolism. Once tumors appear, growth and metastasis may be supported by overproduction of appropriate hormones (in hormonally dependent cancers), by promoting angiogenesis, by undergoing epithelial to mesenchymal transition, by triggering autophagy, and by taking cues from surrounding stromal cells. A number of natural compounds (e.g., curcumin, resveratrol, indole-3-carbinol, brassinin, sulforaphane, epigallocatechin-3-gallate, genistein, ellagitannins, lycopene and quercetin) have been found to inhibit one or more pathways that contribute to proliferation (e.g., hypoxia inducible factor 1, nuclear factor kappa B, phosphoinositide 3 kinase/Akt, insulin-like growth factor receptor 1, Wnt, cell cycle associated proteins, as well as androgen and estrogen receptor signaling). These data, in combination with bioinformatics analyses, will be very important for identifying signaling pathways and molecular targets that may provide early diagnostic markers and/or critical targets for the development of new drugs or drug combinations that block tumor formation and progression

Pan-Cancer Analysis of lncRNA Regulation Supports Their Targeting of Cancer Genes in Each Tumor Context

Long noncoding RNAs (lncRNAs) are commonly dys-regulated in tumors, but only a handful are known toplay pathophysiological roles in cancer. We inferredlncRNAs that dysregulate cancer pathways, onco-genes, and tumor suppressors (cancer genes) bymodeling their effects on the activity of transcriptionfactors, RNA-binding proteins, and microRNAs in5,185 TCGA tumors and 1,019 ENCODE assays.Our predictions included hundreds of candidateonco- and tumor-suppressor lncRNAs (cancerlncRNAs) whose somatic alterations account for thedysregulation of dozens of cancer genes and path-ways in each of 14 tumor contexts. To demonstrateproof of concept, we showed that perturbations tar-geting OIP5-AS1 (an inferred tumor suppressor) andTUG1 and WT1-AS (inferred onco-lncRNAs) dysre-gulated cancer genes and altered proliferation ofbreast and gynecologic cancer cells. Our analysis in-dicates that, although most lncRNAs are dysregu-lated in a tumor-specific manner, some, includingOIP5-AS1, TUG1, NEAT1, MEG3, and TSIX, synergis-tically dysregulate cancer pathways in multiple tumorcontexts

Pan-cancer Alterations of the MYC Oncogene and Its Proximal Network across the Cancer Genome Atlas

Although theMYConcogene has been implicated incancer, a systematic assessment of alterations ofMYC, related transcription factors, and co-regulatoryproteins, forming the proximal MYC network (PMN),across human cancers is lacking. Using computa-tional approaches, we define genomic and proteo-mic features associated with MYC and the PMNacross the 33 cancers of The Cancer Genome Atlas.Pan-cancer, 28% of all samples had at least one ofthe MYC paralogs amplified. In contrast, the MYCantagonists MGA and MNT were the most frequentlymutated or deleted members, proposing a roleas tumor suppressors.MYCalterations were mutu-ally exclusive withPIK3CA,PTEN,APC,orBRAFalterations, suggesting that MYC is a distinct onco-genic driver. Expression analysis revealed MYC-associated pathways in tumor subtypes, such asimmune response and growth factor signaling; chro-matin, translation, and DNA replication/repair wereconserved pan-cancer. This analysis reveals insightsinto MYC biology and is a reference for biomarkersand therapeutics for cancers with alterations ofMYC or the PMN