Laughing Hyena Distillery: Extracting Compact Recurrences From Convolutions

Recent advances in attention-free sequence models rely on convolutions as alternatives to the attention operator at the core of Transformers. In particular, long convolution sequence models have achieved state-of-the-art performance in many domains, but incur a significant cost during auto-regressive inference workloads -- naively requiring a full pass (or caching of activations) over the input sequence for each generated token -- similarly to attention-based models. In this paper, we seek to enable $\mathcal O(1)$ compute and memory cost per token in any pre-trained long convolution architecture to reduce memory footprint and increase throughput during generation. Concretely, our methods consist in extracting low-dimensional linear state-space models from each convolution layer, building upon rational interpolation and model-order reduction techniques. We further introduce architectural improvements to convolution-based layers such as Hyena: by weight-tying the filters across channels into heads, we achieve higher pre-training quality and reduce the number of filters to be distilled. The resulting model achieves 10x higher throughput than Transformers and 1.5x higher than Hyena at 1.3B parameters, without any loss in quality after distillation

Generic and disease-specific health related quality of life in non-cirrhotic, cirrhotic and transplanted liver patients: a cross-sectional study

BACKGROUND: Studies on Health Related Quality of Life (HRQoL) of chronic liver patients were performed in clinical populations. These studies included various disease stages but small variations in aetiology and no transplanted patients. We performed a large HRQoL study in non-cirrhotic, cirrhotic and transplanted liver patients with sufficient variety in aetiology. We compared the generic HRQoL and fatigue between liver patients and healthy controls and compared the disease-specific and generic HRQoL and fatigue between non-cirrhotic, cirrhotic and transplanted liver patients, corrected for aetiology. METHODS: Members of the Dutch liver patient association received the Short Form-36, the Liver Disease Symptom Index and the Multidimensional Fatigue Index-20. Based on reported clinical characteristics we classified respondents (n = 1175) as non-cirrhotic, compensated cirrhotic, decompensated cirrhotic or transplants. We used linear, ordinal and logistic regression to compare the HRQoL between groups. RESULTS: All liver patients showed a significantly worse generic HRQoL and fatigue than healthy controls. Decompensated cirrhotic patients showed a significantly worse disease-specific and generic HRQoL and fatigue than non-cirrhotic patients, while HRQoL differences between non-cirrhotic and compensated cirrhotic patients were predominantly insignificant. Transplanted patients showed a better generic HRQoL, less fatigue and lower probabilities of severe symptoms than non-cirrhotic patients, but almost equal probabilities of symptom hindrance. CONCLUSIONS: HRQoL in chronic liver patients depends on disease stage and transplant history. Non-cirrhotic and compensated cirrhotic patients have a similar HRQoL. Decompensated patients show the worst HRQoL, while transplanted patients show a significantly better HRQoL than cirrhotic and non-cirrhotic patients

The Relationship Between HR Practices and Firm Performance: Examining Causal Order

Significant research attention has been devoted to examining the relationship between HR practices and firm performance, and the research support has assumed HR as the causal variable. Using data from 45 business units (with 62 data points), this study examines how measures of HR practices correlate with past, concurrent, and future operational performance measures. The results indicate that correlations with performance measures at all three times are both high and invariant, and that controlling for past or concurrent performance virtually eliminates the correlation of HR with future performance. Implications are discussed

A rainfall model for drought risk analysis in south-east UK

Drought risk assessment ideally requires long-term rainfall records especially where inter-annual droughts are of potential concern, and spatially consistent estimates of rainfall to support regional and inter-regional scale assessments. This paper addresses these challenges by developing a spatially consistent stochastic model of monthly rainfall for south-east UK. Conditioned on 50 gauged sites, the model infills the historic record from 1855-2011 in both space and time, and extends the record by synthesising droughts which are consistent with the observed rainfall statistics. The long record length allows more insight into the variability of rainfall and potentially a stronger basis for risk assessment than is generally possible. It is shown that, although localised biases exist in both space and time, the model results are generally consistent with the observed record including for a range of inter-annual droughts and spatial statistics. Simulations show that some of the most severe inter-annual droughts on the record may recur, despite a trend towards generally wetter winters

Impacts of urbanisation on the native avifauna of Perth, Western Australia

Urban development either eliminates, or severely fragments, native vegetation, and therefore alters the distribution and abundance of species that depend on it for habitat. We assessed the impact of urban development on bird communities at 121 sites in and around Perth, Western Australia. Based on data from community surveys, at least 83 % of 65 landbirds were found to be dependent, in some way, on the presence of native vegetation. For three groups of species defined by specific patterns of habitat use (bushland birds), there were sufficient data to show that species occurrences declined as the landscape changed from variegated to fragmented to relictual, according to the percentage of vegetation cover remaining. For three other groups (urban birds) species occurrences were either unrelated to the amount of vegetation cover, or increased as vegetation cover declined. In order to maximise the chances of retaining avian diversity when planning for broad-scale changes in land-use (i.e. clearing native vegetation for housing or industrial development), land planners should aim for a mosaic of variegated urban landscapes (\u3e60 % vegetation retention) set amongst the fragmented and relictual urban landscapes (% vegetation retention) that are characteristic of most cities and their suburbs. Management actions for conserving remnant biota within fragmented urban landscapes should concentrate on maintaining the integrity and quality of remnant native vegetation, and aim at building awareness among the general public of the conservation value of remnant native vegetation

Co-prescription of medication for bipolar disorder and diabetes mellitus : a nationwide population based study with focus on gender differences

BackgroundStudies have shown a correlation between bipolar disorder and diabetes mellitus. It is unclear if this correlation is a part of common pathophysiological pathways, or if medication for bipolar disorder has negative effects on blood sugar regulation.MethodsThe Norwegian prescription database was analyzed. Prescriptions for lithium, lamotrigine, carbamazepine and valproate were used as proxies for bipolar disorder. Prescriptions for insulin and oral anti-diabetic agents were used as proxies for diabetes mellitus. We explored the association between medication for bipolar disorder and diabetes medication by logistic regressionResultsWe found a strong association between concomitant use of medication to treat diabetes mellitus and mood stabilizers for the treatment of bipolar disorder. Females had a 30% higher risk compared to men of being treated for both disorders. Persons using oral anti-diabetic agents had higher odds of receiving valproate than either lithium or lamotrigine. Use of insulin as monotherapy seemed to have lower odds than oral anti-diabetic agents of co-prescription of mood stabilizers, compared to the general population.ConclusionsThis study showed a strong association between the use of mood stabilizers and anti-diabetic agents. The association was stronger among women than men

The deuteron: structure and form factors

A brief review of the history of the discovery of the deuteron in provided. The current status of both experiment and theory for the elastic electron scattering is then presented.Comment: 80 pages, 33 figures, submited to Advances in Nuclear Physic

Improving the development, monitoring and reporting of stroke rehabilitation research: consensus-based core recommendations from the Stroke Recovery and Rehabilitation Roundtable (SRRR)

Recent reviews have demonstrated that the quality of stroke rehabilitation research has continued to improve over the last four decades but despite this progress there are still many barriers in moving the field forward. Rigorous development, monitoring and complete reporting of interventions in stroke trials are essential in providing rehabilitation evidence that is robust, meaningful and implementable. An international partnership of stroke rehabilitation experts committed to develop consensus-based core recommendations with a remit of addressing the issues identified as limiting stroke rehabilitation research in the areas of developing, monitoring and reporting stroke rehabilitation interventions. Work exploring each of the three areas took place via multiple teleconferences and a two-day meeting in Philadelphia in May 2016. A total of 15 recommendations were made. To validate the need for the recommendations the group reviewed all stroke rehabilitation trials published in 2015 (n=182 papers). Our review highlighted that the majority of publications did not clearly describe how interventions were developed or monitored during the trial. In particular, under-reporting of the theoretical rationale for the intervention and the components of the intervention calls into question many interventions that have been evaluated for efficacy. More trials were found to have addressed the reporting of interventions recommendations than those related to development or monitoring. Nonetheless the majority of reporting recommendations were still not adequately described. To progress the field of stroke rehabilitation research and to ensure stroke patients receive optimal evidence based clinical care we urge the research community to endorse and adopt our recommendations

Supporting self-management after attending a structured education programme: a qualitative longitudinal investigation of type 1 diabetes patients’ experiences and views

Background: Structured education programmes for patients with diabetes and other chronic conditions are being widely adopted. However, follow-up studies suggest that course graduates may struggle to sustain the self-care practices taught on their courses over time. This study explored the support needs of patients with type 1 diabetes after attending a structured education programme promoting an empowerment approach and training in use of flexible intensive insulin therapy, a regimen now widely advocated and used to manage this condition. The objective was to inform future support offered to course graduates. Methods: Repeat, in-depth interviews with 30 type 1 diabetes patients after attending Dose Adjustment for Normal Eating (DAFNE) courses in the UK, and six and 12 months later. Data were analysed using an inductive, thematic approach. Results: While the flexible intensive insulin treatment approach taught on DAFNE courses was seen as a logical and effective way of managing one’s diabetes, it was also considered more technically complex than other insulin regimens. To sustain effective disease self-management using flexible intensive insulin treatment over time, patients often expected, and needed, on-going input and support from health care professionals trained in the approach. This included: help determining insulin dose adjustments; reassurance; and, opportunities to trouble-shoot issues of concern. While some benefits were identified to receiving follow-up support in a group setting, most patients stated a preference or need for tailored and individualised support from appropriately-trained clinicians, accessible on an ‘as and when needed’ basis. Conclusions: Our findings highlight potential limitations to group-based forms of follow-up support for sustaining diabetes self-management. To maintain the clinical benefits of structured education for patients with type 1 diabetes over time, course graduates may benefit from and prefer ongoing, one-to-one support from health care professionals trained in the programme’s practices and principles. This support should be tailored and personalised to reflect patients’ specific and unique experiences of applying their education and training in the context of their everyday lives, and could be the subject of future research

Studying the Underlying Event in Drell-Yan and High Transverse Momentum Jet Production at the Tevatron

We study the underlying event in proton-antiproton collisions by examining the behavior of charged particles (transverse momentum pT > 0.5 GeV/c, pseudorapidity |\eta| < 1) produced in association with large transverse momentum jets (~2.2 fb-1) or with Drell-Yan lepton-pairs (~2.7 fb-1) in the Z-boson mass region (70 < M(pair) < 110 GeV/c2) as measured by CDF at 1.96 TeV center-of-mass energy. We use the direction of the lepton-pair (in Drell-Yan production) or the leading jet (in high-pT jet production) in each event to define three regions of \eta-\phi space; toward, away, and transverse, where \phi is the azimuthal scattering angle. For Drell-Yan production (excluding the leptons) both the toward and transverse regions are very sensitive to the underlying event. In high-pT jet production the transverse region is very sensitive to the underlying event and is separated into a MAX and MIN transverse region, which helps separate the hard component (initial and final-state radiation) from the beam-beam remnant and multiple parton interaction components of the scattering. The data are corrected to the particle level to remove detector effects and are then compared with several QCD Monte-Carlo models. The goal of this analysis is to provide data that can be used to test and improve the QCD Monte-Carlo models of the underlying event that are used to simulate hadron-hadron collisions.Comment: Submitted to Phys.Rev.