337 research outputs found
Analyzing Transformer Dynamics as Movement through Embedding Space
Transformer language models exhibit intelligent behaviors such as
understanding natural language, recognizing patterns, acquiring knowledge,
reasoning, planning, reflecting and using tools. This paper explores how their
underlying mechanics give rise to intelligent behaviors. We adopt a systems
approach to analyze Transformers in detail and develop a mathematical framework
that frames their dynamics as movement through embedding space. This novel
perspective provides a principled way of thinking about the problem and reveals
important insights related to the emergence of intelligence:
1. At its core the Transformer is a Embedding Space walker, mapping
intelligent behavior to trajectories in this vector space.
2. At each step of the walk, it composes context into a single composite
vector whose location in Embedding Space defines the next step.
3. No learning actually occurs during decoding; in-context learning and
generalization are simply the result of different contexts composing into
different vectors.
4. Ultimately the knowledge, intelligence and skills exhibited by the model
are embodied in the organization of vectors in Embedding Space rather than in
specific neurons or layers. These abilities are properties of this
organization.
5. Attention's contribution boils down to the association-bias it lends to
vector composition and which influences the aforementioned organization.
However, more investigation is needed to ascertain its significance.
6. The entire model is composed from two principal operations: data
independent filtering and data dependent aggregation. This generalization
unifies Transformers with other sequence models and across modalities.
Building upon this foundation we formalize and test a semantic space theory
which posits that embedding vectors represent semantic concepts and find some
evidence of its validity
The use of the patient-centered medical home for children with medical complexity
Children with medical complexity (CMC) comprise a small amount of total pediatric patients but contribute to a large portion of total pediatric spending and health care utilization. As a result, there is great interest in creating innovations in their health care delivery systems to improve their quality of care and contain costs. One unique aspect of the health care of CMC is its fragmented nature. Since CMC commonly have multiple co-morbidities, they often receive care from several different specialists at any given time. The patient-centered medical home (PCMH) was previously suggested as a possible tool to improve the fragmented care of CMC. Although definitions of the medical home vary, common themes are its function to better coordinate and integrate the care of patients. Current studies suggest that many CMC do not make regular primary care visits or receive care that fulfills a majority of the components of a medical home. In addition, according to the limited studies that exist which examine CMC and other children with special health care needs (CSHCN), primary care and medical home usage amongst these populations may reduce the occurrence of preventable medical events like hospital readmissions or emergency department visits. Therefore, further research and work should be conducted to examine the feasibility and actions that must be conducted in order to increase the prevalence of these medical home programs amongst CMC
Risk-sensitive Inverse Reinforcement Learning via Semi- and Non-Parametric Methods
The literature on Inverse Reinforcement Learning (IRL) typically assumes that
humans take actions in order to minimize the expected value of a cost function,
i.e., that humans are risk neutral. Yet, in practice, humans are often far from
being risk neutral. To fill this gap, the objective of this paper is to devise
a framework for risk-sensitive IRL in order to explicitly account for a human's
risk sensitivity. To this end, we propose a flexible class of models based on
coherent risk measures, which allow us to capture an entire spectrum of risk
preferences from risk-neutral to worst-case. We propose efficient
non-parametric algorithms based on linear programming and semi-parametric
algorithms based on maximum likelihood for inferring a human's underlying risk
measure and cost function for a rich class of static and dynamic
decision-making settings. The resulting approach is demonstrated on a simulated
driving game with ten human participants. Our method is able to infer and mimic
a wide range of qualitatively different driving styles from highly risk-averse
to risk-neutral in a data-efficient manner. Moreover, comparisons of the
Risk-Sensitive (RS) IRL approach with a risk-neutral model show that the RS-IRL
framework more accurately captures observed participant behavior both
qualitatively and quantitatively, especially in scenarios where catastrophic
outcomes such as collisions can occur.Comment: Submitted to International Journal of Robotics Research; Revision 1:
(i) Clarified minor technical points; (ii) Revised proof for Theorem 3 to
hold under weaker assumptions; (iii) Added additional figures and expanded
discussions to improve readabilit
Micro-albuminuria in non-diabetic acute ischaemic stroke: prevalence and its co-relation with stroke severity
Background: Microalbuminuria is not only a predictor of subsequent kidney disease, but also an indicator of generalised endothelial injury and a manifestation of endothelial dysfunction. The present study is aimed to determine the prevalence of microalbuminuria among non–diabetic ischaemic stroke patients and find its correlation with ischaemic stroke which eventually will aid us in coming up with potent strategies to provide better prevention and cure.Methods: The present study was conducted in Department of Medicine in collaboration with Department of Biochemistry and Department of Radiology, Guru Nanak Dev Hospital, Amritsar, Punjab, India after taking approval from institutional thesis and ethical committee. The study included 60 patients (30 Cases + 30 Controls) in age group 20-80 years diagnosed as stroke and haemorrhage ruled out by NCCT Brain/MRI Brain at admission. Cases were patients with history of hypertension with acute ischaemic stroke. Controls were age and sex matched patients with no history of hypertension with acute ischaemic stroke. The microalbuminuria was assayed by immunoturbimetry. The stroke severity was assessed by NIH Stroke Severity scale. P value less than 0.05 was considered the level of significance.Results: The overall prevalence of microalbuminuria in acute ischaemic stroke patients was 41.67%. When comparing NIH SS (National Institutes of Health Stroke Scale) score with the levels of albumin in urine, there was a significant positive correlation with urinary albumin levels and stroke severity in the patients having urinary albumin levels in microalbuminuria range both in Case group and Control group with P value less than 0.05.Conclusions: Urine albumin excretion had a positive correlation with the NIH SS Score of the patient in acute ischemic stroke. Those with a higher NIH SS Score had a higher rate of urine albumin excretion and vice versa. Therefore, measurement of microalbuminuria may help to assess those who are at increased risk of severe stroke and may require a more aggressive management
- …