Extracting Reward Functions from Diffusion Models

Diffusion models have achieved remarkable results in image generation, and have similarly been used to learn high-performing policies in sequential decision-making tasks. Decision-making diffusion models can be trained on lower-quality data, and then be steered with a reward function to generate near-optimal trajectories. We consider the problem of extracting a reward function by comparing a decision-making diffusion model that models low-reward behavior and one that models high-reward behavior; a setting related to inverse reinforcement learning. We first define the notion of a relative reward function of two diffusion models and show conditions under which it exists and is unique. We then devise a practical learning algorithm for extracting it by aligning the gradients of a reward function -- parametrized by a neural network -- to the difference in outputs of both diffusion models. Our method finds correct reward functions in navigation environments, and we demonstrate that steering the base model with the learned reward functions results in significantly increased performance in standard locomotion benchmarks. Finally, we demonstrate that our approach generalizes beyond sequential decision-making by learning a reward-like function from two large-scale image generation diffusion models. The extracted reward function successfully assigns lower rewards to harmful images

Influenza della risposta sismica locale sul non sincronismo del moto in superficie: validazione empirica delle indicazioni normative

Nella progettazione di opere con significativo sviluppo longitudinale delle fondazioni, il moto sismico può avere caratteristiche variabili lungo lo sviluppo della struttura. Le differenze possono essere causate dalla perdita di sincronismo nella propagazione delle onde, dalle disomogeneità e discontinuità del sottosuolo e dalla diversa risposta locale del terreno. Secondo le prescrizioni delle Norme Tecniche per le Costruzioni (D.M. 14/1/2008), la valutazione degli effetti del moto asincrono indotti sulle strutture può essere effettuata combinando gli effetti dinamici con quelli pseudo-statici causati dallo spostamento relativo massimo tra due punti della struttura. Questi ultimi si possono valutare attraverso una relazione che deriva dalla soluzione di un modello analitico di propagazione delle onde sismiche (Nuti & Vanzi, 2005). Obiettivo di questo studio è verificare le regole presenti nelle NTC attraverso il confronto con valori di spostamento relativo derivanti da registrazioni accelerometriche di eventi sismici reali. L’effetto combinato dei fattori sopra menzionati è stato valutato empiricamente attraverso l’analisi del moto sismico in superficie registrato da coppie di stazioni ac-celerometriche, ubicate a distanze dell’ordine delle centinaia di metri, su terreni con caratteristiche lito-stratigrafiche corrispondenti ad un sito di riferimento (classe A della Normativa) e un sito con amplificazio-ne stratigrafica (classi B, C, D). Gli spostamenti relativi, calcolati secondo tre possibili approcci, mostrano che la formulazione analitica suggerita dalla Normativa è appropriata per descrivere la dipendenza degli spo-stamenti relativi dall’accelerazione di riferimento. Gli spostamenti calcolati secondo le prescrizioni di Nor-mativa, inoltre, risultano mediamente maggiori di quelli misurati sperimentalmente

Developments in carbohydrate-based metzincin inhibitors

Matrix metalloproteinases (MMPs) and A disintegrin and Metalloproteinase (ADAMs) are zinc-dependent endopeptidases belonging to the metzincin superfamily. Upregulation of metzincin activity is a major feature in many serious pathologies such as cancer, inflammations, and infections. In the last decades, many classes of small molecules have been developed directed to inhibit these enzymes. The principal shortcomings that have hindered clinical development of metzincin inhibitors are low selectivity for the target enzyme, poor water solubility, and long-term toxicity. Over the last 15 years, a novel approach to improve solubility and bioavailability of metzincin inhibitors has been the synthesis of carbohydrate-based compounds. This strategy consists of linking a hydrophilic sugar moiety to an aromatic lipophilic scaffold. This review aims to describe the development of sugar-based and azasugar-based derivatives as metzincin inhibitors and their activity in several pathological models

Commissioning of a Novel Test Apparatus for the Identification of the Dynamic Coefficients of Large Tilting Pad Journal Bearings

This paper describes the commissioning of a novel test bench for the static and dynamic characterization of large tilting pad journal bearings, realized within a collaboration of the Department of Civil and Industrial Engineering of the University of Pisa, BHGE and AM Testing. The adopted test bench configuration has the test article (TA) floating at the mid-span of a rotor supported by two rolling bearings. The TA is statically loaded vertically upwards by a hydraulic actuator and excited dynamically by two orthogonal hydraulic actuators with multiple frequency sinusoidal forces. The test rig is capable of testing bearings with a diameter from 150 to 300 mm. It includes very complex mechanical, hydraulic, electrical and electronic components, and needs, for the whole plant, about 1 MW of electric power. The commissioning of the testing system involved several aspects and presented various issues. This work focuses on measuring systems and data acquisition of high-frequency data (forces, accelerations and relative displacements) and on data processing for the identification of the bearing dynamic coefficients. The identification procedure is based on the linearity assumption and the principle of superposition, operating in the frequency domain with the fast Fourier transforms of the applied forces and displacement signals. First results, referred to a 4-pad bearing, are in satisfactory agreement with theoretical ones

Molecular dysfunction and phenotypic derangement in diabetic cardiomyopathy

The high incidence and poor prognosis of heart failure (HF) patients affected with diabetes (DM) is in part related to a specific cardiac remodeling currently recognized as diabetic cardiomyopathy (DCM). This cardiac frame occurs regardless of the presence of coronary artery diseases (CAD) and it can account for 15-20% of the total diabetic population. The pathogenesis of DCM remains controversial, and several molecular and cellular alterations including myocardial hypertrophy, interstitial fibrosis, oxidative stress and vascular inflammation, have been postulated. The main cardio-vascular alterations associated with hyperglycemia comprise endothelial dysfunction, adverse effects of circulating free fatty acids (FFA) and increased systemic inflammation. High glucose concentrations lead to a loss of mitochondrial networks, increased reactive oxygen species (ROS), endothelial nitric oxide synthase (eNOS) activation and a reduction in cGMP production related to protein kinase G (PKG) activity. Current mechanisms enhance the collagen deposition with subsequent increased myocardial stiffness. Several concerns regarding the exact role of DCM in HF development such as having an appearance as either dilated or as a concentric phenotype and whether diabetes could be considered a causal factor or a comorbidity in HF, remain to be clarified. In this review, we sought to explain the different DCM subtypes and the underlying pathophysiological mechanisms. Therefore, the traditional and new molecular and signal alterations and their relationship with macroscopic structural abnormalities are described

Generation of non-synchronous accelerograms for evaluate the seismic bridge response, including local site amplification.

Non-synchronous seismic actions particularly affect the behaviour of infrastructures with significant longitudinal extension, as bridges, interacting with the soil at surface or below ground level. Some authors state that non synchronism may increase by a large amount the structural response. Several acceleration records relative to different points of the ground with different soil profiles at distances meaningful for bridge analyses, are not available in data banks. The objective of this work is the generation of arrays of asynchronous signals at different points in space, starting from natural accelerograms related to a given seismic event, to increase the number of the available data. The computer code GAS has been modified to use natural accelerograms. The procedure has been applied to a real case, L’Aquila main-shock, for which records in different points of the free field are known

The effect of zoledronic acid on serum osteoprotegerin in early stage multiple myeloma

We evaluated the effect of zoledronic acid (ZA) on serum levels of osteoprotegerin (OPG) and the ligand for receptor activator of nuclear factor kappaB (RANKL) in patients with smoldering myeloma. In treated subjects we found an increase of OPG accounting for an effect of ZA on osteoblast and/or bone marrow stromal cells together with the direct effect on osteoclasts

Seismic Response of Viaducts Accounting for Soil-Structure Interaction

A research was recently granted by the Italian Government to develop a comprehensive procedure to account for spatial variability of ground motion as well as soil-structure interaction in assessing the behaviour of bridges. This paper reports on the work-package relevant to the effects of soil-structure interaction. In the first section, a methodology to include the effects of soil-structure interaction in the nonlinear response of bridges is presented. Kinematic interaction analysis is performed in the frequency domain by means of a procedure accounting for radiation damping, soil-pile and pile-to-pile interaction; the non-linear inertial interaction analysis is performed in the time domain by using a finite element model of the superstructure. Suitable lumped parameter models are implemented to reproduce the frequency-dependent compliance of soil-foundation systems. In the second section, some results of nonlinear dynamic analyses performed on some bridges designed on soft soils by means of a direct displacement approach are presented