129 research outputs found
EvoPrompting: Language Models for Code-Level Neural Architecture Search
Given the recent impressive accomplishments of language models (LMs) for code
generation, we explore the use of LMs as adaptive mutation and crossover
operators for an evolutionary neural architecture search (NAS) algorithm. While
NAS still proves too difficult a task for LMs to succeed at solely through
prompting, we find that the combination of evolutionary prompt engineering with
soft prompt-tuning, a method we term EvoPrompting, consistently finds diverse
and high performing models. We first demonstrate that EvoPrompting is effective
on the computationally efficient MNIST-1D dataset, where EvoPrompting produces
convolutional architecture variants that outperform both those designed by
human experts and naive few-shot prompting in terms of accuracy and model size.
We then apply our method to searching for graph neural networks on the CLRS
Algorithmic Reasoning Benchmark, where EvoPrompting is able to design novel
architectures that outperform current state-of-the-art models on 21 out of 30
algorithmic reasoning tasks while maintaining similar model size. EvoPrompting
is successful at designing accurate and efficient neural network architectures
across a variety of machine learning tasks, while also being general enough for
easy adaptation to other tasks beyond neural network design
Large Language Models Can Be Easily Distracted by Irrelevant Context
Large language models have achieved impressive performance on various natural
language processing tasks. However, so far they have been evaluated primarily
on benchmarks where all information in the input context is relevant for
solving the task. In this work, we investigate the distractibility of large
language models, i.e., how the model problem-solving accuracy can be influenced
by irrelevant context. In particular, we introduce Grade-School Math with
Irrelevant Context (GSM-IC), an arithmetic reasoning dataset with irrelevant
information in the problem description. We use this benchmark to measure the
distractibility of cutting-edge prompting techniques for large language models,
and find that the model performance is dramatically decreased when irrelevant
information is included. We also identify several approaches for mitigating
this deficiency, such as decoding with self-consistency and adding to the
prompt an instruction that tells the language model to ignore the irrelevant
information
Cardiopulmonary Effects of Hemorrhagic Shock in Splenic Autotransplanted Pigs: A New Surgical Model
The spleen is an important organ for hemodynamic compensation during hemorrhagic shock. The aim of the study was to compare the hemodynamic and metabolic responses of sham-operated pigs with intact spleen, splenectomized pigs, and splenic autotransplanted pigs during hemorrhagic shock. Hemorrhagic shock was induced by 30% total blood volume bleed in sham-operated, splenectomized and splenic autotransplanted pigs (n=20). Cardiopulmonary and metabolic variables were measured before, immediately after, and at 20, 60 and 100 minutes after hemorrhage. Upon hemorrhagic shock induction, body temperature, mean arterial pressure, mean pulmonary arterial pressure, cardiac output, cardiac index and oxygen delivery decreased, while lactate and shock index increased. Hemoglobin and hematocrit were significantly lower in the splenectomized and splenic autotransplant groups as compared with the control group at 60 and 100 minutes after hemorrhage (p<0.05). Unlike intact spleen, splenic autotransplant could not improve hemodynamic parameters in hemorrhagic shock in pigs. In comparison to mice, rats or dogs, this species could be an interesting investigation model to test new surgical procedures during splenic related hemorrhagic shock, with potential applications in human medicine
Clinical Application of Mesenchymal Stem Cells and Novel Supportive Therapies for Oral Bone Regeneration
This work has been also recommended by the PACT (Platelet and Advanced Cell Therapies) Forum Civitatis of the POSEIDO Academic Consortium (Periodontology, Oral Surgery, Esthetic and Implant Dentistry Organization).Bone regeneration is often needed prior to dental implant treatment due to the lack of adequate quantity and quality of the bone after infectious diseases, trauma, tumor, or congenital conditions. In these situations, cell transplantation technologies may help to overcome the limitations of autografts, xenografts, allografts, and alloplastic materials. A database search was conducted to include human clinical trials (randomized or controlled) and case reports/series describing the clinical use of mesenchymal stem cells (MSCs) in the oral cavity for bone regeneration only specifically excluding periodontal regeneration. Additionally, novel advances in related technologies are also described. 190 records were identified. 51 articles were selected for full-text assessment, and only 28 met the inclusion criteria: 9 case series, 10 case reports, and 9 randomized controlled clinical trials. Collectively, they evaluate the use of MSCs in a total of 290 patients in 342 interventions. The current published literature is very diverse in methodology and measurement of outcomes. Moreover, the clinical significance is limited. Therefore, the use of these techniques should be further studied in more challenging clinical scenarios with well-designed and standardized RCTs, potentially in combination with new scaffolding techniques and bioactive molecules to improve the final outcomes.The authors of this paper were partially supported by the Talentia Scholarship Program (Junta de Andalucía, Spain) (MPM), the International Team for Implantology through the ITI Scholarship Program (AL), and the Research Groups #CTS-138 and #CTS-583 (Junta de Andalucía, Spain) (All)
Epitelizacija i kontrakcija rane nakon biopsije kože u kunića: matematički model zaraštavanja i remodelirajući indeksi
The objective of this study was to develop a standard operating procedure for the analysis of skin wound healing using histomorphometrical measurements and mathematical data analyses. The mathematical model is derived from observations of normal cutaneous healing in the rabbit. It is designed to allow a simple scoring of the major steps of healing and remodelling. Full-thickness punch biopsies were performed on the skin of the back of New Zealand-white rabbits and healing was analyzed by histopathological examination after 2, 5, 9 and 14 days, using different staining techniques. Histomorphological measurements were also made. The thickness of the epidermis and neo-epidermis were compared. Several indices relative to wound severity and contraction were computed in an attempt to defi ne a global healing index. A remodelling index was calculated based on a colorimetric analysis with Mallory Trichrome staining and hair migration. The changes in indeks values seemed to correlate with the histopathological analysis. No material flaws appeared when this model was applied to the natural healing process. This model was developed for scoring and accurate comparative evaluation of the effects of various treatments, biomaterials or pharmacological preparations on soft tissue healing and remodelling in rabbits. Although the healing of cutaneous wounds in rabbits differs from that in humans, this model may still be relevant for screening new wound healing preparations.Cilj ovog istraživanja je razvijanje osnovne metode za analizu zaraštavanja kože služeći se histomorfometrijskim mjerenjima i matematičkom analizom podataka. Matematički je model nastao promatranjem fi ziološkoga zarastanja kože u kunića. Model je razvijen za jednostavno mjerenje osnovnih faza zarastanja i remodeliranja rane. Potpuna biopsija kože provedena je na leđnoj koži novozelandskih bijelih kunića te je analiza zarastanja promatrana histopatološki nakon drugoga, petoga, devetoga i četrnaestoga dana rabeći različite metode bojenja. Također su izvršena histomorfološka mjerenja. Uspoređene su vrijednosti debljine fi ziološkoga i novonastaloga epidermisa. Nekoliko indeksa povezanih sa zarastanjem i kontrakcijom
kože pribrajani su s pokušajem utvrđivanja potpunoga indeksa zaraštavanja. Kolometrijska analiza s Mallory trichrome bojenjem korištena je za izračun remodelirajućega indeksa i promatranja migracije dlačnoga folikula. Promjene u vrijednosti indeksa mogu se povezati s histopatološkom analizom. Prirodni proces zarastanja promatran je bez utjecaja čimbenika koji mogu doprinijeti ishodu samoga zarastanja. Taj je model razvijen kako bi se moglo promatrati i uspoređivati različita liječenja, biomaterijali i farmakološki pripravci za zarastanje i remodeliranje mekoga tkiva u kunića. Unatoč razlici u zarastanju kože kunića i čovjeka, ovaj model može biti koristan za promatranje novih pripravaka za zaraštavanje rana
Rethinking Attention with Performers
We introduce Performers, Transformer architectures which can estimate regular
(softmax) full-rank-attention Transformers with provable accuracy, but using
only linear (as opposed to quadratic) space and time complexity, without
relying on any priors such as sparsity or low-rankness. To approximate softmax
attention-kernels, Performers use a novel Fast Attention Via positive
Orthogonal Random features approach (FAVOR+), which may be of independent
interest for scalable kernel methods. FAVOR+ can be also used to efficiently
model kernelizable attention mechanisms beyond softmax. This representational
power is crucial to accurately compare softmax with other kernels for the first
time on large-scale tasks, beyond the reach of regular Transformers, and
investigate optimal attention-kernels. Performers are linear architectures
fully compatible with regular Transformers and with strong theoretical
guarantees: unbiased or nearly-unbiased estimation of the attention matrix,
uniform convergence and low estimation variance. We tested Performers on a rich
set of tasks stretching from pixel-prediction through text models to protein
sequence modeling. We demonstrate competitive results with other examined
efficient sparse and dense attention methods, showcasing effectiveness of the
novel attention-learning paradigm leveraged by Performers.Comment: Published as a conference paper + oral presentation at ICLR 2021. 38
pages. See
https://github.com/google-research/google-research/tree/master/protein_lm for
protein language model code, and
https://github.com/google-research/google-research/tree/master/performer for
Performer code. See
https://ai.googleblog.com/2020/10/rethinking-attention-with-performers.html
for Google AI Blo
The state of the Martian climate
60°N was +2.0°C, relative to the 1981–2010 average value (Fig. 5.1). This marks a new high for the record. The average annual surface air temperature (SAT) anomaly for 2016 for land stations north of starting in 1900, and is a significant increase over the previous highest value of +1.2°C, which was observed in 2007, 2011, and 2015. Average global annual temperatures also showed record values in 2015 and 2016. Currently, the Arctic is warming at more than twice the rate of lower latitudes
- …