Deep reinforcement learning from human preferences

For sophisticated reinforcement learning (RL) systems to interact usefully with real-world environments, we need to communicate complex goals to these systems. In this work, we explore goals defined in terms of (non-expert) human preferences between pairs of trajectory segments. We show that this approach can effectively solve complex RL tasks without access to the reward function, including Atari games and simulated robot locomotion, while providing feedback on less than one percent of our agent's interactions with the environment. This reduces the cost of human oversight far enough that it can be practically applied to state-of-the-art RL systems. To demonstrate the flexibility of our approach, we show that we can successfully train complex novel behaviors with about an hour of human time. These behaviors and environments are considerably more complex than any that have been previously learned from human feedback

Smart technologies for effective reconfiguration: the FASTER approach

Current and future computing systems increasingly require that their functionality stays flexible after the system is operational, in order to cope with changing user requirements and improvements in system features, i.e. changing protocols and data-coding standards, evolving demands for support of different user applications, and newly emerging applications in communication, computing and consumer electronics. Therefore, extending the functionality and the lifetime of products requires the addition of new functionality to track and satisfy the customers needs and market and technology trends. Many contemporary products along with the software part incorporate hardware accelerators for reasons of performance and power efficiency. While adaptivity of software is straightforward, adaptation of the hardware to changing requirements constitutes a challenging problem requiring delicate solutions. The FASTER (Facilitating Analysis and Synthesis Technologies for Effective Reconfiguration) project aims at introducing a complete methodology to allow designers to easily implement a system specification on a platform which includes a general purpose processor combined with multiple accelerators running on an FPGA, taking as input a high-level description and fully exploiting, both at design time and at run time, the capabilities of partial dynamic reconfiguration. The goal is that for selected application domains, the FASTER toolchain will be able to reduce the design and verification time of complex reconfigurable systems providing additional novel verification features that are not available in existing tool flows

Assessment of neural networks training strategies for histomorphometric analysis of synchrotron radiation medical images

Abstract Micro-computed tomography (μCT) obtained by synchrotron radiation (SR) enables magnified images with a high space resolution that might be used as a non-invasive and non-destructive technique for the quantitative analysis of medical images, in particular the histomorphometry (HMM) of bony mass. In the preprocessing of such images, conventional operations such as binarization and morphological filtering are used before calculating the stereological parameters related, for example, to the trabecular bone microarchitecture. However, there is no standardization of methods for HMM based on μCT images, especially the ones obtained with SR X-ray. Notwithstanding the several uses of artificial neural networks (ANNs) in medical imaging, their application to the HMM of SR-μCT medical images is still incipient, despite the potential of both techniques. The contribution of this paper is the assessment and comparison of well-known training algorithms as well as the proposal of training strategies (combinations of training algorithms, sub-image kernel and symmetry information) for feed-forward ANNs in the task of bone pixels recognition in SR-μCT medical images. For a quantitative comparison, the results of a cross validation and a statistical analysis of the results for 36 training strategies are presented. The ANNs demonstrated both very low mean square errors in the validation, and good quality segmentation of the image of interest for application to HMM in SR-μCT medical images

Canagliflozin and renal outcomes in type 2 diabetes and nephropathy

BACKGROUND Type 2 diabetes mellitus is the leading cause of kidney failure worldwide, but few effective long-term treatments are available. In cardiovascular trials of inhibitors of sodium–glucose cotransporter 2 (SGLT2), exploratory results have suggested that such drugs may improve renal outcomes in patients with type 2 diabetes. METHODS In this double-blind, randomized trial, we assigned patients with type 2 diabetes and albuminuric chronic kidney disease to receive canagliflozin, an oral SGLT2 inhibitor, at a dose of 100 mg daily or placebo. All the patients had an estimated glomerular filtration rate (GFR) of 30 to <90 ml per minute per 1.73 m2 of body-surface area and albuminuria (ratio of albumin [mg] to creatinine [g], >300 to 5000) and were treated with renin–angiotensin system blockade. The primary outcome was a composite of end-stage kidney disease (dialysis, transplantation, or a sustained estimated GFR of <15 ml per minute per 1.73 m2), a doubling of the serum creatinine level, or death from renal or cardiovascular causes. Prespecified secondary outcomes were tested hierarchically. RESULTS The trial was stopped early after a planned interim analysis on the recommendation of the data and safety monitoring committee. At that time, 4401 patients had undergone randomization, with a median follow-up of 2.62 years. The relative risk of the primary outcome was 30% lower in the canagliflozin group than in the placebo group, with event rates of 43.2 and 61.2 per 1000 patient-years, respectively (hazard ratio, 0.70; 95% confidence interval [CI], 0.59 to 0.82; P=0.00001). The relative risk of the renal-specific composite of end-stage kidney disease, a doubling of the creatinine level, or death from renal causes was lower by 34% (hazard ratio, 0.66; 95% CI, 0.53 to 0.81; P<0.001), and the relative risk of end-stage kidney disease was lower by 32% (hazard ratio, 0.68; 95% CI, 0.54 to 0.86; P=0.002). The canagliflozin group also had a lower risk of cardiovascular death, myocardial infarction, or stroke (hazard ratio, 0.80; 95% CI, 0.67 to 0.95; P=0.01) and hospitalization for heart failure (hazard ratio, 0.61; 95% CI, 0.47 to 0.80; P<0.001). There were no significant differences in rates of amputation or fracture. CONCLUSIONS In patients with type 2 diabetes and kidney disease, the risk of kidney failure and cardiovascular events was lower in the canagliflozin group than in the placebo group at a median follow-up of 2.62 years

Canagliflozin and Renal Outcomes in Type 2 Diabetes and Nephropathy

BACKGROUND Type 2 diabetes mellitus is the leading cause of kidney failure worldwide, but few effective long-term treatments are available. In cardiovascular trials of inhibitors of sodium–glucose cotransporter 2 (SGLT2), exploratory results have suggested that such drugs may improve renal outcomes in patients with type 2 diabetes. METHODS In this double-blind, randomized trial, we assigned patients with type 2 diabetes and albuminuric chronic kidney disease to receive canagliflozin, an oral SGLT2 inhibitor, at a dose of 100 mg daily or placebo. All the patients had an estimated glomerular filtration rate (GFR) of 30 to 300 to 5000) and were treated with renin–angiotensin system blockade. The primary outcome was a composite of end-stage kidney disease (dialysis, transplantation, or a sustained estimated GFR of <15 ml per minute per 1.73 m 2), a doubling of the serum creatinine level, or death from renal or cardiovascular causes. Prespecified secondary outcomes were tested hierarchically. RESULTS The trial was stopped early after a planned interim analysis on the recommendation of the data and safety monitoring committee. At that time, 4401 patients had undergone randomization, with a median follow-up of 2.62 years. The relative risk of the primary outcome was 30% lower in the canagliflozin group than in the placebo group, with event rates of 43.2 and 61.2 per 1000 patient-years, respectively (hazard ratio, 0.70; 95% confidence interval [CI], 0.59 to 0.82; P=0.00001). The relative risk of the renal-specific composite of end-stage kidney disease, a doubling of the creatinine level, or death from renal causes was lower by 34% (hazard ratio, 0.66; 95% CI, 0.53 to 0.81; P<0.001), and the relative risk of end-stage kidney disease was lower by 32% (hazard ratio, 0.68; 95% CI, 0.54 to 0.86; P=0.002). The canagliflozin group also had a lower risk of cardiovascular death, myocardial infarction, or stroke (hazard ratio, 0.80; 95% CI, 0.67 to 0.95; P=0.01) and hospitalization for heart failure (hazard ratio, 0.61; 95% CI, 0.47 to 0.80; P<0.001). There were no significant differences in rates of amputation or fracture. CONCLUSIONS In patients with type 2 diabetes and kidney disease, the risk of kidney failure and cardiovascular events was lower in the canagliflozin group than in the placebo group at a median follow-up of 2.62 years

A multi-domain gem-grade Brazilian apatite

A gem-grade apatite from Brazil of general composition (Ca, Na)10[(P, Si, S)O4]6(F, Cl, OH)2 has been studied using single-crystal X-ray and neutron diffraction together with synchrotron powder X-ray diffraction. Earlier electron microscopy studies had shown the nominally single-phase apatite contains an abundant fluorapatite (F-Ap) host, together with chloro-hydroxylapatites (Cl/OH-Ap) guest phases that encapsulate hydroxylellestadite (OH-El) nanocrystals. While the latter features appear as small (200-400 nm) chemically distinct regions by transmission electron microscopy, and can be identified as separate phases by synchrotron powder X-ray diffraction, these could not be detected by singlecrystal X-ray and neutron analysis. The observations using neutron, X-ray and electron probes are however consistent and complementary. After refinement in the space group P63/m the tunnel anions F-are fixed at z = 1/4 along , while the anions Cl- and OH- are disordered, with the suggestion that O-H···O-H··· hydrogen-bonded chains form in localized regions, such that no net poling results. The major cations are located in the 4f AFO6 metaprism (Ca+Na), 6h ATO6X tunnel site (Ca only), and 6h BO4 tetrahedron (P+Si+S). The structural intricacy of this gem stone provides further evidence that apatite microstructures display a nano-phase separation that is generally unrecognized, with the implication that such complexity may impact upon the functionality of technological analogues