Learning Multi-Task Transferable Rewards via Variational Inverse Reinforcement Learning

Many robotic tasks are composed of a lot of temporally correlated sub-tasks in a highly complex environment. It is important to discover situational intentions and proper actions by deliberating on temporal abstractions to solve problems effectively. To understand the intention separated from changing task dynamics, we extend an empowerment-based regularization technique to situations with multiple tasks based on the framework of a generative adversarial network. Under the multitask environments with unknown dynamics, we focus on learning a reward and policy from the unlabeled expert examples. In this study, we define situational empowerment as the maximum of mutual information representing how an action conditioned on both a certain state and sub-task affects the future. Our proposed method derives the variational lower bound of the situational mutual information to optimize it. We simultaneously learn the transferable multi-task reward function and policy by adding an induced term to the objective function. By doing so, the multi-task reward function helps to learn a robust policy for environmental change. We validate the advantages of our approach on multi-task learning and multi-task transfer learning. We demonstrate our proposed method has the robustness of both randomness and changing task dynamics. Finally, we prove that our method has significantly better performance and data efficiency than existing imitation learning methods on various benchmarks.Comment: Accepted in ICRA 202

Vulnerability assessment for the hazards of crosswinds when vehicles cross a bridge deck

AbstractA new procedure to assess the crosswind hazard of operating a vehicle over a bridge deck has been developed using a probabilistic approach that utilizes long-term wind data at bridge sites as well as the aerodynamic properties of bridge decks and vehicles. The proposed procedure for safety assessment considers the probabilities of two accident types: sideslip and overturning. The vulnerability of vehicles to crosswinds is represented by the number of days for traffic control that would be required to secure vehicle safety over a period of one year. The distribution of wind speed over a bridge deck was estimated from a section model wind tunnel test. A sea-crossing bridge was selected as an example, and a series of case studies were performed to identify the influential factors affecting vehicle vulnerability to crosswinds: vehicle type and loaded weight, the position of a running vehicle over a bridge deck, the bridge alignment relative to the dominant wind direction, and vehicle speed

Macrophage inflammatory protein-related protein-2, a novel CC chemokine, can regulate preadipocyte migration and adipocyte differentiation

AbstractAdipocytes not only store energy, but also secrete biologically active molecules called adipocytokines, which play a pivotal role in adipocyte-related pathological processes such as diabetes and cardiovascular disease. Recent studies have shown that preadipocyte/adipocyte expresses chemokines (e.g. monocyte chemoattractant protein-1, macrophage inflammatory protein-1 alpha) which alter adipocyte function, indicating the involvement of chemokines in adipocyte-related pathologies. The current study investigated the potential of macrophage inflammatory protein-related protein-2 (MRP-2), a novel CC chemokine, to modulate preadipocyte trafficking and adipocyte differentiation. MRP-2 and its receptors were highly expressed in preadipocytes and differentiated adipocytes as well as in the mouse fat pad. Chemotaxis assays revealed that MRP-2 was a specific chemotactic regulator in preadipocyte migration. The levels of MRP-2 expression in adipose tissue were enhanced in obese mice compared to lean mice. MRP-2 secretion by preadipocytes was suppressed during differentiation. MRP-2 suppressed the expression of adipocyte differentiation markers such as adipocyte fatty acid-binding protein and glycerol-3 phosphate dehydrogenase. Taken together, our data suggest that MRP-2 plays a role in the regulation of preadipocyte migration and adipocyte differentiation during adipose tissue development. MRP-2 may be another adipocytokine, which can be involved in the adipocyte-related pathological process

Accounting Conservatism, Changes In Real Investment, And Analysts Earnings Forecasts

This study examines whether sell-side analysts fully incorporate into their earnings forecasts the joint effects between accounting conservatism and changes in real investment on the quality of current earnings. Our results indicate that sell-side analysts do not fully incorporate such effects when they forecast future earnings so that they overestimate (underestimate) future earnings when current earnings are inflated (depressed) by those effects. Thus, we conclude that sell-side analysts do not recognize fully the joint effects between accounting conservatism and real activity on the earnings quality and that they need to mitigate their bias to enhance market efficiency by providing investors with a good benchmark for their earnings expectation

Control Method Of Circulating Refrigerant Amount For Heat Pump System

A heat pump system requires proper refrigerant charge amount. Once refrigerant is charged into a heat pump system, its charge amount is fixed. For this reason, prediction of optimal refrigerant charge amount is very important in order to yield best performance. Too low charge amount degrades capacity of heat pump. On the other hand, excessive charge amount decreases coefficient of performance (COP). The optimal value of refrigerant charge amount highly depends on secondary fluid temperature conditions. Consequently, fixed charge amount of refrigerant in heat pump shows the best performance only at certain temperature condition. Several ideas have revealed to change charge amount of the heat pump system. One is to have an additional reservoir to store or release refrigerant which is attached to a heat pump system. This method may seem simple but to measure exact amount of refrigerant in reservoir, additional pressure transducer, temperature measurement device, level sensor and other apparatus are required that increase the cost of heat pump. Another idea is to have reservoir between condenser outlet and expansion device. Rajapaksha and Suen (2004) showed that existence of reservoir at this point helps improve capacity while reducing the system COP. In this study, a new method for refrigerant charge amount control technique is presented. It has very simple control logic and requires only a few additional cost factors; several valves and additional tubes are only required. This method is based on different refrigerant phase distribution at each point of inlet and outlet of components in heat pump system. In a simple cycle heat pump system, refrigerant at condenser outlet (before expansion device) is in a subcooled liquid state at high pressure, while refrigerant is in a superheated vapor state at evaporator outlet (before compressor inlet) at low pressure. This technique regulates refrigerant charge by holding some volume of refrigerant in the connecting tube of considerable volume installed between the condenser outlet and the evaporator outlet. Using several solenoid valves (on/off) desired amount of refrigerant can be stored into a volume provided by a connecting tube. This connected volume is referred as ‘stagnation volume’ (Vstag). When one of this installed valve is closed and the rest of the valves are open, certain amount of refrigerant is stored in the stagnation volume (Vstag) while operating heat pump system. If closed valve is adjacent to condenser outlet, charge amount to the heat pump system increases while the charge is reduced when the valve adjacent to evaporator outlet is closed. This method is numerically verified and there are very little variation of COP. Therefore, heat pump can be operated at optimized circulating amount of refrigerant in spite of the secondary fluid temperature variation during heating or cooling operation

Bucillamine prevents cisplatin-induced ototoxicity through induction of glutathione and antioxidant genes.

Bucillamine is used for the treatment of rheumatoid arthritis. This study investigated the protective effects of bucillamine against cisplatin-induced damage in auditory cells, the organ of Corti from postnatal rats (P2) and adult Balb/C mice. Cisplatin increases the catalytic activity of caspase-3 and caspase-8 proteases and the production of free radicals, which were significantly suppressed by pretreatment with bucillamine. Bucillamine induces the intranuclear translocation of Nrf2 and thereby increases the expression of γ-glutamylcysteine synthetase (γ-GCS) and glutathione synthetase (GSS), which further induces intracellular antioxidant glutathione (GSH), heme oxygenase 1 (HO-1) and superoxide dismutase 2 (SOD2). However, knockdown studies of HO-1 and SOD2 suggest that the protective effect of bucillamine against cisplatin is independent of the enzymatic activity of HO-1 and SOD. Furthermore, pretreatment with bucillamine protects sensory hair cells on organ of Corti explants from cisplatin-induced cytotoxicity concomitantly with inhibition of caspase-3 activation. The auditory-brainstem-evoked response of cisplatin-injected mice shows marked increases in hearing threshold shifts, which was markedly suppressed by pretreatment with bucillamine in vivo. Taken together, bucillamine protects sensory hair cells from cisplatin through a scavenging effect on itself, as well as the induction of intracellular GSH

Memory-Efficient Fine-Tuning of Compressed Large Language Models via sub-4-bit Integer Quantization

Large language models (LLMs) face the challenges in fine-tuning and deployment due to their high memory demands and computational costs. While parameter-efficient fine-tuning (PEFT) methods aim to reduce the memory usage of the optimizer state during fine-tuning, the inherent size of pre-trained LLM weights continues to be a pressing concern. Even though quantization techniques are widely proposed to ease memory demands and accelerate LLM inference, most of these techniques are geared towards the deployment phase. To bridge this gap, this paper presents Parameter-Efficient and Quantization-aware Adaptation (PEQA) - a simple yet effective method that combines the advantages of PEFT with quantized LLMs. By updating solely the quantization scales, PEQA can be directly applied to quantized LLMs, ensuring seamless task transitions. Parallel to existing PEFT methods, PEQA significantly reduces the memory overhead associated with the optimizer state. Furthermore, it leverages the advantages of quantization to substantially reduce model sizes. Even after fine-tuning, the quantization structure of a PEQA-tuned LLM remains intact, allowing for accelerated inference on the deployment stage. We employ PEQA-tuning for task-specific adaptation on LLMs with up to 65 billion parameters. To assess the logical reasoning and language comprehension of PEQA-tuned LLMs, we fine-tune low-bit quantized LLMs using a instruction dataset. Our results show that even when LLMs are quantized to below 4-bit precision, their capabilities in language modeling, few-shot in-context learning, and comprehension can be resiliently restored to (or even improved over) their full-precision original performances with PEQA.Comment: Published at NeurIPS 2023. Camera-ready versio

Evaluation of Left Atrial Volumes Using Multidetector Computed Tomography: Comparison with Echocardiography

OBJECTIVE: To prospectively assess the relationship between the two different measurement methods for the evaluation of left atrial (LA) volume using cardiac multidetector computed tomography (MDCT) and to compare the results between cardiac MDCT and echocardiography. MATERIALS AND METHODS: Thirty-five patients (20 men, 15 women; mean age, 60 years) underwent cardiac MDCT angiography for coronary artery disease. The LA volumes were measured using two different methods: the two dimensional (2D) length-based (LB) method measured along the three-orthogonal planes of the LA and the 3D volumetric threshold-based (VTB) method measured according to the threshold 3D segmentation of the LA. The results obtained by cardiac MDCT were compared with those obtained by echocardiography. RESULTS: The LA end-systolic and end-diastolic volumes (LAESV and LAEDV) measured by the 2D-LB method correlated well with those measured by the 3D-VTB method using cardiac MDCT (r = 0.763, r = 0.786, p = 0.001). However, there was a significant difference in the LAESVs between the two measurement methods using cardiac MDCT (p < 0.05). The LAESV measured by cardiac MDCT correlated well with measurements by echocardiography (r = 0.864, p = 0.001), however with a significant difference (p < 0.01) in their volumes. The cardiac MDCT overestimated the LAESV by 22% compared to measurements by echocardiography. CONCLUSION: A significant correlation was found between the two different measurement methods for evaluating LA volumes by cardiac MDCT. Further, cardiac MDCT correlates well with echocardiography in evaluating the LA volume. However, there are significant differences in the LAESV between the two measurement methods using cardiac MDCT and between cardiac MDCT and echocardiographyope