Safety-aware apprenticeship learning

It is well acknowledged in the AI community that finding a good reward function for reinforcement learning is extremely challenging. Apprenticeship learning (AL) is a class of “learning from demonstration” techniques where the reward function of a Markov Decision Process (MDP) is unknown to the learning agent and the agent uses inverse reinforcement learning (IRL) methods to recover expert policy from a set of expert demonstrations. However, as the agent learns exclusively from observations, given a constraint on the probability of the agent running into unwanted situations, there is no verification, nor guarantee, for the learnt policy on the satisfaction of the restriction. In this dissertation, we study the problem of how to guide AL to learn a policy that is inherently safe while still meeting its learning objective. By combining formal methods with imitation learning, a Counterexample-Guided Apprenticeship Learning algorithm is proposed. We consider a setting where the unknown reward function is assumed to be a linear combination of a set of state features, and the safety property is specified in Probabilistic Computation Tree Logic (PCTL). By embedding probabilistic model checking inside AL, we propose a novel counterexample-guided approach that can ensure both safety and performance of the learnt policy. This algorithm guarantees that given some formal safety specification defined by probabilistic temporal logic, the learnt policy shall satisfy this specification. We demonstrate the effectiveness of our approach on several challenging AL scenarios where safety is essential

Acoustic Frequency Multiplication and Pure Second Harmonic Generation of Phonons by Magnetic Transducers

We predict frequency multiplication of surface acoustic waves in dielectric substrates via the ferromagnetic resonance of adjacent magnetic transducers when driven by microwaves. We find pure second harmonic generation (SHG) without any linear and third harmonic components by a magnetic nanowire. The SHG and linear phonon pumping are switched by varying the saturated magnetization direction of the wire, or resolved directionally when pumped by magnetic nano-disc. We address the high efficiency of SHG with comparable magnitude to that of linear response, as well as unique non-reciprocal phonon transport that is remarkably distinct in different phonon harmonics. Such acoustic frequency comb driven by microwaves should bring unprecedented tunability for the miniaturized phononic and spintronic devices.Comment: 7 pages, 3 figure

Performance evaluation of a tidal current turbine with bidirectional symmetrical foils

As one might expect, tidal currents in terms of ebb and flood tides are approximately bidirectional. A Horizontal Axial Tidal Turbine (HATT) with unidirectional foils has to be able to face the current directions in order to maximize current energy harvesting. There are two regular solutions to keep a HATT always facing the direction of the flow, which are transferred from wind turbine applications. One is to yaw the turbine around the supporting structure with a yaw mechanism. The other is to reverse the blade pitch angle through 180° with a pitch-adjusting mechanism. The above solutions are not cost-effective in marine applications due to the harsh marine environment and high cost of installation and maintenance. In order to avoid the above disadvantages, a turbine with bidirectional foils is presented in this paper. A bare turbine with bidirectional foils is characterized in that it has nearly the same energy conversion capability in both tidal current directions without using the yaw or pitch mechanism. Considering the working conditions of the bidirectional turbine in which the turbine is installed on a mono-pile, the effect of the mono-pile on the turbine’s performance is evaluated in this paper, especially when the turbine is downstream of the mono-pile. The paper was focused on the evaluation of the hydrodynamic performance of the bidirectional turbine. The hydrodynamic performance of the bare bidirectional turbine without any supporting structure was evaluated based on a steady-state computational fluid dynamics (CFD) model and model tests. Performance comparison has been made between the turbine with bidirectional foils and the turbine with NACA foils. The effect of the mono-pile on the performance of the bidirectional turbine was studied by using the steady-state and the transient CFD model. The steady-state CFD model was used to evaluate the effect of the mono-pile clearance, which is the distance between the mono-pile and the turbine on the performance of the turbine. The transient CFD model was used to determine the time-dependent characteristics of the turbine, such as time-dependent power and drag coefficients. The results show that the bare bidirectional turbine has nearly the same energy conversion capability in both tidal current directions. The performance of the bidirectional turbine is inferior to the turbine with NACA foils. At the designed tip speed ratio, the power coefficient of the turbine with NACA foils is 0.4498, which increases by 1.6% compared to the 0.4338 of the bidirectional turbine. The turbine’s performance decreases due to the introduction of the mono-pile, and the closer the turbine is to the mono-pile, the greater effect on the turbine’s performance the mono-pile has. At the designed clearance of 1.5 DS, the presence of a mono-pile decreases the peak Cp value by 1.82% and 3.17% to a value of 0.4156 and 0.4004 for the turbine located in the mono-pile upstream and downstream, respectively. The mono-pile can result in the fluctuation of the turbine’s performance. This fluctuation will detrimentally harm the life of the turbine as it will lead to increased wear and fatigue issues

Semiparametric efficient estimation of genetic relatedness with machine learning methods

In this paper, we propose semiparametric efficient estimators of genetic relatedness between two traits in a model-free framework. Most existing methods require specifying certain parametric models involving the traits and genetic variants. However, the bias due to model misspecification may yield misleading statistical results. Moreover, the semiparametric efficient bounds for estimators of genetic relatedness are still lacking. In this paper, we develop semiparametric efficient estimators with machine learning methods and construct valid confidence intervals for two important measures of genetic relatedness: genetic covariance and genetic correlation, allowing both continuous and discrete responses. Based on the derived efficient influence functions of genetic relatedness, we propose a consistent estimator of the genetic covariance as long as one of genetic values is consistently estimated. The data of two traits may be collected from the same group or different groups of individuals. Various numerical studies are performed to illustrate our introduced procedures. We also apply proposed procedures to analyze Carworth Farms White mice genome-wide association study data.Comment: 46pages,9 tables, 1 figur

Exploiting Spatial-Temporal Context for Interacting Hand Reconstruction on Monocular RGB Video

Reconstructing interacting hands from monocular RGB data is a challenging task, as it involves many interfering factors, e.g. self- and mutual occlusion and similar textures. Previous works only leverage information from a single RGB image without modeling their physically plausible relation, which leads to inferior reconstruction results. In this work, we are dedicated to explicitly exploiting spatial-temporal information to achieve better interacting hand reconstruction. On one hand, we leverage temporal context to complement insufficient information provided by the single frame, and design a novel temporal framework with a temporal constraint for interacting hand motion smoothness. On the other hand, we further propose an interpenetration detection module to produce kinetically plausible interacting hands without physical collisions. Extensive experiments are performed to validate the effectiveness of our proposed framework, which achieves new state-of-the-art performance on public benchmarks.Comment: 16 page

BEST: BERT Pre-Training for Sign Language Recognition with Coupling Tokenization

In this work, we are dedicated to leveraging the BERT pre-training success and modeling the domain-specific statistics to fertilize the sign language recognition~(SLR) model. Considering the dominance of hand and body in sign language expression, we organize them as pose triplet units and feed them into the Transformer backbone in a frame-wise manner. Pre-training is performed via reconstructing the masked triplet unit from the corrupted input sequence, which learns the hierarchical correlation context cues among internal and external triplet units. Notably, different from the highly semantic word token in BERT, the pose unit is a low-level signal originally located in continuous space, which prevents the direct adoption of the BERT cross-entropy objective. To this end, we bridge this semantic gap via coupling tokenization of the triplet unit. It adaptively extracts the discrete pseudo label from the pose triplet unit, which represents the semantic gesture/body state. After pre-training, we fine-tune the pre-trained encoder on the downstream SLR task, jointly with the newly added task-specific layer. Extensive experiments are conducted to validate the effectiveness of our proposed method, achieving new state-of-the-art performance on all four benchmarks with a notable gain.Comment: Accepted by AAAI 2023 (Oral

A Hybrid Wireless Image Transmission Scheme with Diffusion

We propose a hybrid joint source-channel coding (JSCC) scheme, in which the conventional digital communication scheme is complemented with a generative refinement component to improve the perceptual quality of the reconstruction. The input image is decomposed into two components: the first is a coarse compressed version, and is transmitted following the conventional separation based approach. An additional component is obtained through the diffusion process by adding independent Gaussian noise to the input image, and is transmitted using DeepJSCC. The decoder combines the two signals to produce a high quality reconstruction of the source. Experimental results show that the hybrid design provides bandwidth savings and enables graceful performance improvement as the channel quality improves