EvoPrompting: Language Models for Code-Level Neural Architecture Search

Given the recent impressive accomplishments of language models (LMs) for code generation, we explore the use of LMs as adaptive mutation and crossover operators for an evolutionary neural architecture search (NAS) algorithm. While NAS still proves too difficult a task for LMs to succeed at solely through prompting, we find that the combination of evolutionary prompt engineering with soft prompt-tuning, a method we term EvoPrompting, consistently finds diverse and high performing models. We first demonstrate that EvoPrompting is effective on the computationally efficient MNIST-1D dataset, where EvoPrompting produces convolutional architecture variants that outperform both those designed by human experts and naive few-shot prompting in terms of accuracy and model size. We then apply our method to searching for graph neural networks on the CLRS Algorithmic Reasoning Benchmark, where EvoPrompting is able to design novel architectures that outperform current state-of-the-art models on 21 out of 30 algorithmic reasoning tasks while maintaining similar model size. EvoPrompting is successful at designing accurate and efficient neural network architectures across a variety of machine learning tasks, while also being general enough for easy adaptation to other tasks beyond neural network design

Large Language Models Can Be Easily Distracted by Irrelevant Context

Large language models have achieved impressive performance on various natural language processing tasks. However, so far they have been evaluated primarily on benchmarks where all information in the input context is relevant for solving the task. In this work, we investigate the distractibility of large language models, i.e., how the model problem-solving accuracy can be influenced by irrelevant context. In particular, we introduce Grade-School Math with Irrelevant Context (GSM-IC), an arithmetic reasoning dataset with irrelevant information in the problem description. We use this benchmark to measure the distractibility of cutting-edge prompting techniques for large language models, and find that the model performance is dramatically decreased when irrelevant information is included. We also identify several approaches for mitigating this deficiency, such as decoding with self-consistency and adding to the prompt an instruction that tells the language model to ignore the irrelevant information

Cardiopulmonary Effects of Hemorrhagic Shock in Splenic Autotransplanted Pigs: A New Surgical Model

The spleen is an important organ for hemodynamic compensation during hemorrhagic shock. The aim of the study was to compare the hemodynamic and metabolic responses of sham-operated pigs with intact spleen, splenectomized pigs, and splenic autotransplanted pigs during hemorrhagic shock. Hemorrhagic shock was induced by 30% total blood volume bleed in sham-operated, splenectomized and splenic autotransplanted pigs (n=20). Cardiopulmonary and metabolic variables were measured before, immediately after, and at 20, 60 and 100 minutes after hemorrhage. Upon hemorrhagic shock induction, body temperature, mean arterial pressure, mean pulmonary arterial pressure, cardiac output, cardiac index and oxygen delivery decreased, while lactate and shock index increased. Hemoglobin and hematocrit were significantly lower in the splenectomized and splenic autotransplant groups as compared with the control group at 60 and 100 minutes after hemorrhage (p<0.05). Unlike intact spleen, splenic autotransplant could not improve hemodynamic parameters in hemorrhagic shock in pigs. In comparison to mice, rats or dogs, this species could be an interesting investigation model to test new surgical procedures during splenic related hemorrhagic shock, with potential applications in human medicine

Clinical Application of Mesenchymal Stem Cells and Novel Supportive Therapies for Oral Bone Regeneration

This work has been also recommended by the PACT (Platelet and Advanced Cell Therapies) Forum Civitatis of the POSEIDO Academic Consortium (Periodontology, Oral Surgery, Esthetic and Implant Dentistry Organization).Bone regeneration is often needed prior to dental implant treatment due to the lack of adequate quantity and quality of the bone after infectious diseases, trauma, tumor, or congenital conditions. In these situations, cell transplantation technologies may help to overcome the limitations of autografts, xenografts, allografts, and alloplastic materials. A database search was conducted to include human clinical trials (randomized or controlled) and case reports/series describing the clinical use of mesenchymal stem cells (MSCs) in the oral cavity for bone regeneration only specifically excluding periodontal regeneration. Additionally, novel advances in related technologies are also described. 190 records were identified. 51 articles were selected for full-text assessment, and only 28 met the inclusion criteria: 9 case series, 10 case reports, and 9 randomized controlled clinical trials. Collectively, they evaluate the use of MSCs in a total of 290 patients in 342 interventions. The current published literature is very diverse in methodology and measurement of outcomes. Moreover, the clinical significance is limited. Therefore, the use of these techniques should be further studied in more challenging clinical scenarios with well-designed and standardized RCTs, potentially in combination with new scaffolding techniques and bioactive molecules to improve the final outcomes.The authors of this paper were partially supported by the Talentia Scholarship Program (Junta de Andalucía, Spain) (MPM), the International Team for Implantology through the ITI Scholarship Program (AL), and the Research Groups #CTS-138 and #CTS-583 (Junta de Andalucía, Spain) (All)

Epitelizacija i kontrakcija rane nakon biopsije kože u kunića: matematički model zaraštavanja i remodelirajući indeksi

The objective of this study was to develop a standard operating procedure for the analysis of skin wound healing using histomorphometrical measurements and mathematical data analyses. The mathematical model is derived from observations of normal cutaneous healing in the rabbit. It is designed to allow a simple scoring of the major steps of healing and remodelling. Full-thickness punch biopsies were performed on the skin of the back of New Zealand-white rabbits and healing was analyzed by histopathological examination after 2, 5, 9 and 14 days, using different staining techniques. Histomorphological measurements were also made. The thickness of the epidermis and neo-epidermis were compared. Several indices relative to wound severity and contraction were computed in an attempt to defi ne a global healing index. A remodelling index was calculated based on a colorimetric analysis with Mallory Trichrome staining and hair migration. The changes in indeks values seemed to correlate with the histopathological analysis. No material flaws appeared when this model was applied to the natural healing process. This model was developed for scoring and accurate comparative evaluation of the effects of various treatments, biomaterials or pharmacological preparations on soft tissue healing and remodelling in rabbits. Although the healing of cutaneous wounds in rabbits differs from that in humans, this model may still be relevant for screening new wound healing preparations.Cilj ovog istraživanja je razvijanje osnovne metode za analizu zaraštavanja kože služeći se histomorfometrijskim mjerenjima i matematičkom analizom podataka. Matematički je model nastao promatranjem fi ziološkoga zarastanja kože u kunića. Model je razvijen za jednostavno mjerenje osnovnih faza zarastanja i remodeliranja rane. Potpuna biopsija kože provedena je na leđnoj koži novozelandskih bijelih kunića te je analiza zarastanja promatrana histopatološki nakon drugoga, petoga, devetoga i četrnaestoga dana rabeći različite metode bojenja. Također su izvršena histomorfološka mjerenja. Uspoređene su vrijednosti debljine fi ziološkoga i novonastaloga epidermisa. Nekoliko indeksa povezanih sa zarastanjem i kontrakcijom kože pribrajani su s pokušajem utvrđivanja potpunoga indeksa zaraštavanja. Kolometrijska analiza s Mallory trichrome bojenjem korištena je za izračun remodelirajućega indeksa i promatranja migracije dlačnoga folikula. Promjene u vrijednosti indeksa mogu se povezati s histopatološkom analizom. Prirodni proces zarastanja promatran je bez utjecaja čimbenika koji mogu doprinijeti ishodu samoga zarastanja. Taj je model razvijen kako bi se moglo promatrati i uspoređivati različita liječenja, biomaterijali i farmakološki pripravci za zarastanje i remodeliranje mekoga tkiva u kunića. Unatoč razlici u zarastanju kože kunića i čovjeka, ovaj model može biti koristan za promatranje novih pripravaka za zaraštavanje rana

Rethinking Attention with Performers

We introduce Performers, Transformer architectures which can estimate regular (softmax) full-rank-attention Transformers with provable accuracy, but using only linear (as opposed to quadratic) space and time complexity, without relying on any priors such as sparsity or low-rankness. To approximate softmax attention-kernels, Performers use a novel Fast Attention Via positive Orthogonal Random features approach (FAVOR+), which may be of independent interest for scalable kernel methods. FAVOR+ can be also used to efficiently model kernelizable attention mechanisms beyond softmax. This representational power is crucial to accurately compare softmax with other kernels for the first time on large-scale tasks, beyond the reach of regular Transformers, and investigate optimal attention-kernels. Performers are linear architectures fully compatible with regular Transformers and with strong theoretical guarantees: unbiased or nearly-unbiased estimation of the attention matrix, uniform convergence and low estimation variance. We tested Performers on a rich set of tasks stretching from pixel-prediction through text models to protein sequence modeling. We demonstrate competitive results with other examined efficient sparse and dense attention methods, showcasing effectiveness of the novel attention-learning paradigm leveraged by Performers.Comment: Published as a conference paper + oral presentation at ICLR 2021. 38 pages. See https://github.com/google-research/google-research/tree/master/protein_lm for protein language model code, and https://github.com/google-research/google-research/tree/master/performer for Performer code. See https://ai.googleblog.com/2020/10/rethinking-attention-with-performers.html for Google AI Blo

The state of the Martian climate

60°N was +2.0°C, relative to the 1981–2010 average value (Fig. 5.1). This marks a new high for the record. The average annual surface air temperature (SAT) anomaly for 2016 for land stations north of starting in 1900, and is a significant increase over the previous highest value of +1.2°C, which was observed in 2007, 2011, and 2015. Average global annual temperatures also showed record values in 2015 and 2016. Currently, the Arctic is warming at more than twice the rate of lower latitudes