38 research outputs found

    Reinforcement learning-based autonomous robot navigation and tracking

    Get PDF
    Autonomous navigation requires determining a collision-free path for a mobile robot using only partial observations of the environment. This capability is highly needed for a wide range of applications, such as search and rescue operations, surveillance, environmental monitoring, and domestic service robots. In many scenarios, an accurate global map is not available beforehand, posing significant challenges for a robot planning its path. This type of navigation is often referred to as Mapless Navigation, and such work is not limited to only Unmanned Ground Vehicle (UGV) but also other vehicles, such as Unmanned Aerial Vehicles (UAV) and more. This research aims to develop Reinforcement Learning (RL)-based methods for autonomous navigation for mobile robots, as well as effective tracking strategies for a UAV to follow a moving target. Mapless navigation usually assumes accurate localisation, which is unrealistic. In the real world, localisation methods, such as simultaneous localisation and mapping (SLAM), are needed. However, the localisation performance could deteriorate depending on the environment and observation quality. Therefore, To avoid de-teriorated localisation, this work introduces an RL-based navigation algorithm to enable mobile robots to navigate in unknown environments, while incorporating localisation performance in training the policy. Specifically, a localisation-related penalty is introduced in the reward space, ensuring localisation safety is taken into consideration during navigation. Different metrics are formulated to identify if the localisation performance starts to deteriorate in order to penalise the robot. As such, the navigation policy will not only optimise its paths in terms of travel distance and collision avoidance towards the goal but also avoid venturing into areas that pose challenges for localisation algorithms. The localisation-safe algorithm is further extended to UAV navigation, which uses image-based observations. Instead of deploying an end-to-end control pipeline, this work establishes a hierarchical control framework that leverages both the capabilities of neural networks for perception and the stability and safety guarantees of conventional controllers. The high-level controller in this hierarchical framework is a neural network policy with semantic image inputs, trained using RL algorithms with localisation-related rewards. The efficacy of the trained policy is demonstrated in real-world experiments for localisation-safe navigation, and, notably, it exhibits effectiveness without the need for retraining, thanks to the hierarchical control scheme and semantic inputs. Last, a tracking policy is introduced to enable a UAV to track a moving target. This study designs a reward space, enabling a vision-based UAV, which utilises depth images for perception, to follow a target within a safe and visible range. The objective is to maintain the mobile target at the centre of the drone camera’s image without being occluded by other objects and to avoid collisions with obstacles. It is observed that training such a policy from scratch may lead to local minima. To address this, a state-based teacher policy is trained to perform the tracking task, with environmental perception relying on direct access to state information, including position coordinates of obstacles, instead of depth images. An RL algorithm is then constructed to train the vision-based policy, incorporating behavioural guidance from the state-based teacher policy. This approach yields promising tracking performance

    Localisation-safe reinforcement learning for mapless navigation

    Get PDF
    Most reinforcement learning (RL)-based works for mapless point goal navigation tasks assume the availability of the robot ground-truth poses, which is unrealistic for real world applications. In this work, we remove such an assumption and deploy observation-based localisation algorithms, such as Lidar-based or visual odometry, for robot self-pose estimation. These algorithms, despite having widely achieved promising performance and being robust to various harsh environments, may fail to track robot locations under many scenarios, where observations perceived along robot trajectories are insufficient or ambiguous. Hence, using such localisation algorithms will introduce new unstudied problems for mapless navigation tasks. This work will propose a new RL-based algorithm, with which robots learn to navigate in a way that prevents localisation failures or getting trapped in local minimum regions. This ability can be learned by deploying two techniques suggested in this work: a reward metric to decide punishment on behaviours resulting in localisation failures; and a reconfigured state representation that consists of current observation and history trajectory information to transfer the problem from a partially observable Markov decision process (POMDP) to a Markov Decision Process (MDP) model to avoid local minimum

    Reinforcement learning-based mapless navigation with fail-safe localisation

    Get PDF
    Mapless navigation is the capability of a robot to navigate without knowing the map. Previous works assume the availability of accurate self-localisation, which is, however, usually unrealistic. In our work, we deploy simultaneous localisation and mapping (SLAM)-based self-localisation for mapless navigation. SLAM performance is prone to the quality of perceived features of the surroundings. This work presents a Reinforcement Learning (RL)-based mapless navigation algorithm, aiming to improve the robustness of robot localisation by encouraging the robot to learn to be aware of the quality of its surrounding features and avoid feature-poor environment, where localisation is less reliable. Particle filter (PF) is deployed for pose estimation in our work, although, in principle, any localisation algorithm should work with this framework. The aim of the work is two-fold: to train a robot to learn 1) to avoid collisions and also 2) to identify paths that optimise PF-based localisation, such that the robot will be unlikely to fail to localise itself, hence fail-safe SLAM. A simulation environment is tested in this work with different maps and randomised training conditions. The trained policy has demonstrated superior performance compared with standard mapless navigation without this optimised policy

    Apocynum venetum leaf extract alleviated doxorubicin-induced cardiotoxicity by regulating organic acid metabolism in gut microbiota

    Get PDF
    Apocynum venetum leaf is commonly utilized for its beneficial effects in reducing blood pressure, inducing sedation, promoting diuresis, anti-aging, and cardioprotection, which also exhibit positive effects on the gut microbiota. The gut microbiota plays a role as an endocrine organ by producing bioactive metabolites that can directly or indirectly impact host physiology, specifically cardiovascular diseases. In this study, main chemical components of A. venetum leaf extract (AVLE) were identified by LC-MS, and an orally administered AVLE was employed to treat mice with doxorubicin (Dox)-induced cardiotoxicity. The results showed that AVLE contained hyperoside and oganic acids. The pharmacological findings revealed that AVLE regulated the gut microbiota, resulting in a significant increase in the levels of two organic acids, indole-3-propionic acid (IPA) and acetic acid (AA). Both IPA and AA exhibited the ability to reduce BNP, CK, and LDH levels in mice with Dox-induced cardiotoxicity. Moreover, IPA demonstrated an improvement in Dox-induced cardiac injury by inhibiting apoptosis, while AA promoted increased secretion of ghrelin through the parasympathetic nervous system, subsequently reducing cardiac fibrosis by decreasing collagen I, collagen III, and activin A. Hence, our study demonstrates that AVLE exerts a beneficial cardioprotective effect by modulating the gut microbiota, providing a potential novel target for the treatment and prevention of Dox-induced cardiotoxicity

    Design of Disturbance Extended State Observer (D-ESO)-Based Constrained Full-State Model Predictive Controller for the Integrated Turbo-Shaft Engine/Rotor System

    No full text
    In the modern helicopter design and development process, constrained full-state control technology for turbo-shaft engine/rotor systems has always been a research hotspot in academia and industry. However, relevant references have pointed out that the traditional design method with an overly complex structure (Min-Max structure and schedule-based transient controller, i.e., M-M-STC) may not be able to meet the protection requirements of engine control systems under certain circumstances and can be too conservative under other conditions. In order to address the engine limit protection problem more efficiently, a constrained full-state model predictive controller (MPC) has been designed in this paper by incorporating a linear parameter varying (LPV) predictive model. Meanwhile, disturbance extended state observer (D-ESO) (which a sufficient convergence condition is deduced for) has also been proposed as the compensator of the LPV model to alleviate the MPC model mismatch problem. Finally, we run a group of comparison simulations with the traditional M-M-STC method to verify the effectiveness of this controller by taking compressor surge prevention problems as a case study, and the results indicate the validity of the proposed method

    Lightweight image super-resolution with a feature-refined network

    No full text
    In recent years, deep Convolutional Neural Networks (CNNs) have achieved impressive successes on the Single Image Super-Resolution task (SISR). However, it remains difficult to apply these CNN-based SISR methods in embedded devices due to their high memory and computational requirements. To alleviate this issue, we focus on lightweight SISR methods. The observed similarity between the feature maps in CNNs serves as inspiration to explore the design of a cost-efficient module to obtain feature maps whose representation ability is roughly equivalent to that of a conventional convolutional layer. We thus propose a shadow module applying simple linear transformations with a lower cost to generate similar feature maps. Based on this module, we design a Feature-Refined Block (FRB) to learn more representative features. Besides, we propose a Global Dense Feature Fusion (GDFF) structure to construct a Feature-Refined Network (FRN) with such FRBs for lightweight SISR. Extensive experimental results demonstrate the superior performance of the proposed FRN in comparison with the state-of-the-art lightweight SISR methods, while consuming relatively low memory and computation resources

    A deep recursive multi-scale feature fusion network for image super-resolution?

    No full text
    Recently, Convolutional Neural Networks (CNNs) have achieved great success in Single Image Super-Resolution (SISR). In particular, the recursive networks are now widely used. However, existing recursion-based SISR networks can only make use of multi-scale features in a layer-wise manner. In this paper, a Deep Recursive Multi-Scale Feature Fusion Network (DRMSFFN) is proposed to address this issue. Specifically, we propose a Recursive Multi-Scale Feature Fusion Block (RMSFFB) to make full use of multi-scale features. Besides, a Progressive Feature Fusion (PFF) technique is proposed to take advantage of the hierarchical features from the RMSFFB in a global manner. At the reconstruction stage, we use a deconvolutional layer to upscale the feature maps to the desired size. Extensive experimental results on benchmark datasets demonstrate the superiority of the proposed DRMSFFN in comparison with the state-of-the-art methods in both quantitative and qualitative evaluations
    corecore