Method of Moments in Approximate Bayesian Inference: From Theory to Practice

With recent advances in approximate inference, Bayesian methods have proven successful in larger datasets and more complex models. The central problem in Bayesian inference is how to approximate intractable posteriors accurately and efficiently. Variational inference deals with this problem by projecting the posterior onto a simpler distribution space. The projection step in variational inference is usually done by minimizing Kullback–Leibler divergence, but alternative methods may sometimes yield faster and more accurate solutions. Moments are statistics to describe the shape of a probability distribution, and one can project the distribution by matching a set of moments. The idea of moment matching dates back to the method of moments (MM), a simple approach to estimate unknown parameters by enforcing the moments to match with estimation. While MM has been primarily studied in frequentist statistics, it can lend itself naturally to approximate Bayesian inference. This thesis aims to better understand how to apply MM in general-purpose Bayesian inference problems and the advantage of MM methods in Bayesian inference. We begin with the simplest model in machine learning and gradually extend to more complex and practical settings. The scope of our work spans from theory, methodology to applications. We first study a specific algorithm that uses MM in mixture posteriors, Bayesian Moment Matching (BMM). We prove consistency of BMM in a naive Bayes model and then propose an initializer to Boolean SAT solvers based on its extension to Bayesian networks. BMM is quite restrictive and can only be used with conjugate priors. We then propose a new algorithm, Multiple Moment Matching Inference (MMMI), a general-purpose approximate Bayesian inference algorithm based on the idea of MM, and demonstrate its competitive predictive performance on real-world datasets

Robotics Dexterous Grasping: The Methods Based on Point Cloud and Deep Learning

Dexterous manipulation, especially dexterous grasping, is a primitive and crucial ability of robots that allows the implementation of performing human-like behaviors. Deploying the ability on robots enables them to assist and substitute human to accomplish more complex tasks in daily life and industrial production. A comprehensive review of the methods based on point cloud and deep learning for robotics dexterous grasping from three perspectives is given in this paper. As a new category schemes of the mainstream methods, the proposed generation-evaluation framework is the core concept of the classification. The other two classifications based on learning modes and applications are also briefly described afterwards. This review aims to afford a guideline for robotics dexterous grasping researchers and developers

Progressive Transfer Learning for Dexterous In-Hand Manipulation with Multi-Fingered Anthropomorphic Hand

Dexterous in-hand manipulation for a multi-fingered anthropomorphic hand is extremely difficult because of the high-dimensional state and action spaces, rich contact patterns between the fingers and objects. Even though deep reinforcement learning has made moderate progress and demonstrated its strong potential for manipulation, it is still faced with certain challenges, such as large-scale data collection and high sample complexity. Especially, for some slight change scenes, it always needs to re-collect vast amounts of data and carry out numerous iterations of fine-tuning. Remarkably, humans can quickly transfer learned manipulation skills to different scenarios with little supervision. Inspired by human flexible transfer learning capability, we propose a novel dexterous in-hand manipulation progressive transfer learning framework (PTL) based on efficiently utilizing the collected trajectories and the source-trained dynamics model. This framework adopts progressive neural networks for dynamics model transfer learning on samples selected by a new samples selection method based on dynamics properties, rewards and scores of the trajectories. Experimental results on contact-rich anthropomorphic hand manipulation tasks show that our method can efficiently and effectively learn in-hand manipulation skills with a few online attempts and adjustment learning under the new scene. Compared to learning from scratch, our method can reduce training time costs by 95%.Comment: 12 pages, 7 figures, submitted to TNNL

CMMLU: Measuring massive multitask language understanding in Chinese

As the capabilities of large language models (LLMs) continue to advance, evaluating their performance becomes increasingly crucial and challenging. This paper aims to bridge this gap by introducing CMMLU, a comprehensive Chinese benchmark that covers various subjects, including natural science, social sciences, engineering, and humanities. We conduct a thorough evaluation of 18 advanced multilingual- and Chinese-oriented LLMs, assessing their performance across different subjects and settings. The results reveal that most existing LLMs struggle to achieve an average accuracy of 50%, even when provided with in-context examples and chain-of-thought prompts, whereas the random baseline stands at 25%. This highlights significant room for improvement in LLMs. Additionally, we conduct extensive experiments to identify factors impacting the models' performance and propose directions for enhancing LLMs. CMMLU fills the gap in evaluating the knowledge and reasoning capabilities of large language models within the Chinese context

Dataset Inference for Self-Supervised Models

Self-supervised models are increasingly prevalent in machine learning (ML) since they reduce the need for expensively labeled data. Because of their versatility in downstream applications, they are increasingly used as a service exposed via public APIs. At the same time, these encoder models are particularly vulnerable to model stealing attacks due to the high dimensionality of vector representations they output. Yet, encoders remain undefended: existing mitigation strategies for stealing attacks focus on supervised learning. We introduce a new dataset inference defense, which uses the private training set of the victim encoder model to attribute its ownership in the event of stealing. The intuition is that the log-likelihood of an encoder's output representations is higher on the victim's training data than on test data if it is stolen from the victim, but not if it is independently trained. We compute this log-likelihood using density estimation models. As part of our evaluation, we also propose measuring the fidelity of stolen encoders and quantifying the effectiveness of the theft detection without involving downstream tasks; instead, we leverage mutual information and distance measurements. Our extensive empirical results in the vision domain demonstrate that dataset inference is a promising direction for defending self-supervised models against model stealing.Comment: Accepted at NeurIPS 202

Detection and Localization of Faults in a Regional Power Grid

The structure of power flows in transmission grids is evolving and is likely to change significantly in the coming years due to the rapid growth of renewable energy generation that introduces randomness and bidirectional power flows. Another transformative aspect is the increasing penetration of various smart-meter technologies. Inexpensive measurement devices can be placed at practically any component of the grid. Using model data reflecting smart-meter measurements, we propose a two-stage procedure for detecting a fault in a regional power grid. In the first stage, a fault is detected in real time. In the second stage, the faulted line is identified with a negligible delay. The approach uses only the voltage modulus measured at buses (nodes of the grid) as the input. Our method does not require prior knowledge of the fault type. The method is fully implemented in  R. Pseudo code and complete mathematical formulas are provided

Human–robot object handover: Recent progress and future direction

Human–robot object handover is one of the most primitive and crucial capabilities in human–robot collaboration. It is of great significance to promote robots to truly enter human production and life scenarios and serve human in numerous tasks. Remarkable progressions in the field of human–robot object handover have been made by researchers. This article reviews the recent literature on human–robot object handover. To this end, we summarize the results from multiple dimensions, from the role played by the robot (receiver or giver), to the end-effector of the robot (parallel-jaw gripper or multi-finger hand), to the robot abilities (grasp strategy or motion planning). We also implement a human–robot object handover system for anthropomorphic hand to verify human–robot object handover pipeline. This review aims to provide researchers and developers with a guideline for designing human–robot object handover methods

Dynamic constitutive identification of concrete based on improved dung beetle algorithm to optimize long short-term memory model

Abstract In order to improve the accuracy of concrete dynamic principal identification, a concrete dynamic principal identification model based on Improved Dung Beetle Algorithm (IDBO) optimized Long Short-Term Memory (LSTM) network is proposed. Firstly, the apparent stress–strain curves of concrete containing damage evolution were measured by Split Hopkinson Pressure Bar (SHPB) test to decouple and separate the damage and rheology, and this system was modeled by using LSTM network. Secondly, for the problem of low convergence accuracy and easy to fall into local optimum of Dung Beetle Algorithm (DBO), the greedy lens imaging reverse learning initialization population strategy, the embedded curve adaptive weighting factor and the PID control optimal solution perturbation strategy are introduced, and the superiority of IDBO algorithm is proved through the comparison of optimization test with DBO, Harris Hawk Optimization Algorithm, Gray Wolf Algorithm, and Fruit Fly Algorithm and the combination of LSTM is built to construct the IDBO-LSTM dynamic homeostasis identification model. The final results show that the IDBO-LSTM model can recognize the concrete material damage without considering the damage; in the case of considering the damage, the IDBO-LSTM prediction curves basically match the SHPB test curves, which proves the feasibility and excellence of the proposed method

Decomposed Iterative Optimal Power Flow with Automatic Regionalization

The optimal power flow (OPF) problem plays an important role in power system operation and control. The problem is nonconvex and NP-hard, hence global optimality is not guaranteed and the complexity grows exponentially with the size of the system. Therefore, centralized optimization techniques are not suitable for large-scale systems and an efficient decomposed implementation of OPF is highly demanded. In this paper, we propose a novel and efficient method to decompose the entire system into multiple sub-systems based on automatic regionalization and acquire the OPF solution across sub-systems via a modified MATPOWER solver. The proposed method is implemented in a modified solver and tested on several IEEE Power System Test Cases. The performance is shown to be more appealing compared with the original solver

Application Research on Liner Tube without Variable Diameter of Shale Gas Plunger Gas Lift Wellhead

Given that there is variable diameter in the shale gas plunger gas lift wellhead, which causes that the plunger cannot run up and down the gas well wellhead, affects the effect of plunger drainage gas production, the paper carries out research on liner tube without variable diameter of the shale gas plunger gas lift wellhead and installation methods, makes the diameter of the gas well wellhead consistent with that of the oil pipe, ensure that the plunger can be open up and down the wellhead of the Christmas tree, facilitate the periodic inspection and replacement operation of the plunger, and reduces the cost of plunger salvage operations. The successful application of the process plan in the shale gas well ** has achieved good drainage gas production effect, truly solved the variable diameter problem of shale gas plunger gas lift wellhead, and achieved the goal of once and for all