4,468 research outputs found
Fluid and Electrolyte Management of Very Low Birth Weight Infants
Recent advances in medical knowledge and technology have markedly improved the survival rates of very low birth weight infants. Optimizing the neuro-developmental outcomes of these survivors has become an important priority in neonatal care, which includes appropriate management for achieving fluid and electrolyte balance. This review focuses on the principles of providing maintenance fluid to these infants, including careful assessment to avoid excessive fluid administration that may increase the risk of such neonatal morbidities as necrotizing enterocolitis, patent ductus arteriosus, and bronchopulmonary dysplasia (BPD). The review also describes the principles of fluid and electrolyte management of infants with BPD, which includes the strategy of providing adequate nutrition to promote normal growth
Transformer-Based Language Model Surprisal Predicts Human Reading Times Best with About Two Billion Training Tokens
Recent psycholinguistic studies have drawn conflicting conclusions about the
relationship between the quality of a language model and the ability of its
surprisal estimates to predict human reading times, which has been speculated
to be due to the large gap in both the amount of training data and model
capacity across studies. The current work aims to consolidate these findings by
evaluating surprisal estimates from Transformer-based language model variants
that vary systematically in the amount of training data and model capacity on
their ability to predict human reading times. The results show that surprisal
estimates from most variants with contemporary model capacities provide the
best fit after seeing about two billion training tokens, after which they begin
to diverge from humanlike expectations. Additionally, newly-trained smaller
model variants reveal a 'tipping point' at convergence, after which the
decrease in language model perplexity begins to result in poorer fits to human
reading times. These results suggest that the massive amount of training data
is mainly responsible for the poorer fit achieved by surprisal from larger
pre-trained language models, and that a certain degree of model capacity is
necessary for Transformer-based language models to capture humanlike
expectations.Comment: Findings of the Association for Computational Linguistics: EMNLP 202
Token-wise Decomposition of Autoregressive Language Model Hidden States for Analyzing Model Predictions
While there is much recent interest in studying why Transformer-based large
language models make predictions the way they do, the complex computations
performed within each layer have made their behavior somewhat opaque. To
mitigate this opacity, this work presents a linear decomposition of final
hidden states from autoregressive language models based on each initial input
token, which is exact for virtually all contemporary Transformer architectures.
This decomposition allows the definition of probability distributions that
ablate the contribution of specific input tokens, which can be used to analyze
their influence on model probabilities over a sequence of upcoming words with
only one forward pass from the model. Using the change in next-word probability
as a measure of importance, this work first examines which context words make
the biggest contribution to language model predictions. Regression experiments
suggest that Transformer-based language models rely primarily on collocational
associations, followed by linguistic factors such as syntactic dependencies and
coreference relationships in making next-word predictions. Additionally,
analyses using these measures to predict syntactic dependencies and coreferent
mention spans show that collocational association and repetitions of the same
token largely explain the language models' predictions on these tasks.Comment: ACL 202
Sustainable farming with native rocks: the transition without revolution.
The development process which humanity passed through favored a series of conquests, reflected in the better quality of life and longevity, however, it also provoked upsets and severe transformation in the environment and in the human food security. Such process is driving the ecosystems to be homogeneous, and, therefore,the nutrients� supply, via nourishment. To change this panorama, the present work discusses the gains of incorporating the stonemeal technique as a strategic alternative to give back the essential fertile characteristics to the soils. This technology has the function of facilitating the rejuvenation of the soils and increasing the availability of the necessary nutrients to the full development of the plants which is a basic input for the proliferation of life in all its dimensions
Frequency Explains the Inverse Correlation of Large Language Models' Size, Training Data Amount, and Surprisal's Fit to Reading Times
Recent studies have shown that as Transformer-based language models become
larger and are trained on very large amounts of data, the fit of their
surprisal estimates to naturalistic human reading times degrades. The current
work presents a series of analyses showing that word frequency is a key
explanatory factor underlying these two trends. First, residual errors from
four language model families on four corpora show that the inverse correlation
between model size and fit to reading times is the strongest on the subset of
least frequent words, which is driven by excessively accurate predictions of
larger model variants. Additionally, training dynamics reveal that during later
training steps, all model variants learn to predict rare words and that larger
model variants do so more accurately, which explains the detrimental effect of
both training data amount and model size on fit to reading times. Finally, a
feature attribution analysis demonstrates that larger model variants are able
to accurately predict rare words based on both an effectively longer context
window size as well as stronger local associations compared to smaller model
variants. Taken together, these results indicate that Transformer-based
language models' surprisal estimates diverge from human-like expectations due
to the superhumanly complex associations they learn for predicting rare words.Comment: EACL 202
Shifting Tides: The Evolution of Racial Inequality in Higher Education from the 1980s through the 2010s
Amid the proliferation of state-level bans on race-based affirmative action in higher education, the U.S. Supreme Court’s decision on June 29, 2023, dismantled race-conscious college admission policies, intensifying concerns about the persistence and potential increase of racial inequality in higher education. The authors analyze four restricted-use national survey datasets to investigate racial disparities in college attendance outcomes from the 1980s through the 2010s. Although college entrance rates increased for all racial groups, Black and Hispanic youth became increasingly less likely than their White peers to attend four-year selective colleges. In the 2010s cohort, Black and Hispanic youth were 8 and 7 percentage points, respectively, less likely than their White counterparts to secure admission to fouryear selective colleges, even after controlling for parents’ income, education, and other family background variables. The findings underscore the urgent need for proactive policy interventions to address the widening racial inequality in attending selective postsecondary institutions
Recommended from our members
Consumer Preference for Sustainable Tourism Certifications: A Choice Modeling Approach
Certifying tourism businesses as sustainable has been proposed as a means to connect the academic goals of ecotourism to the industry. One obstacle that impedes the development of widely recognized sustainable tourism certifications is the lack of knowledge regarding consumer preferences for these certifications. A better understanding of consumer desires is needed in order to create sustainable tourism certifications that will generate price premiums for certified businesses and thus accelerate the growth of certified businesses. This study uses a stated-preference choice modeling approach to examine consumer preference for sustainable tourism certifications amongst international tourists in Tanzania. Results suggest that certifications that emphasize environmental sustainability are more important to tourists than certifications that emphasize cultural or economic sustainability. Additionally, the more stringent certifications provide relatively little increase in consumer utility. The implications of the findings on the development of sustainable tourism certifications are discussed
Partially coherent ambiguity functions for depth-variant point spread function design
The ambiguity function (AF) provides a convenient way to model how a camera with a modified aperture responds to defocus. We use the AF to design an optimal aperture distribution, which creates a depth-variant point spread function (PSF) from a sparse set of desired intensity patterns at different focal depths. Prior knowledge of the coherence state of the light is used to constrain the optimization in the mutual intensity domain. We use an assumption of spatially coherent light to design a fixed-pattern aperture mask. The concept of a dynamic aperture mask that displays several aperture patterns during one image exposure is also suggested, which is modeled under an assumption of partially coherent light. Parallels are drawn between the optimal aperture functions for this dynamic mask and the eigenmodes of a coherent mode decomposition. We demonstrate how the space of design for a 3D intensity distribution of light using partially coherent assumptions is less constrained than under coherent light assumptions.United States. Air Force Office of Scientific Research (National Defense Science and Engineering Graduate (NDSEG) fellowship)United States. Defense Advanced Research Projects Agency (DARPA Young Faculty Award)Alfred P. Sloan Foundation (Research Fellowship
- …