Two-dimensional swarm formation in time-invariant external potential: Modeling, analysis, and control

Cluster formation has been observed in many organisms in nature. It has the desirable properties for designing energy efficient protocols for Wireless Sensor Networks (WSNs). In this paper, we present a new approach for energy efficient WSN protocols that investigates how the cluster formation of sensors responds to the external time-invariant energy potential. In this approach, the necessity for data transmission to the Base Station is eliminated, thereby conserving energy for WSNs. We define swarm formation topology and estimate the curvature of an external potential manifold by analyzing the change of the swarm formation in time. We also introduce a dynamic formation control algorithm for maintaining defined swarm formation topology in the external potential. Energy conservation is a crucial challenge in Wireless Sensor Networks (WSNs). As energy for data transmission is most costly, WSNs’ algorithms need to be designed in ways where data transmission, especially to the control center called the Base Station (BS), is minimized. Clustering is a possible mechanism to design energy efficient algorithms for WSNs. In this paper, we combine the idea of swarm intelligence with WSNs and design an algorithm that captures the environmental information through analyzing the change in sensor cluster formations (swarm formation) rather than gathering information directly through individual sensor measurements. In this approach, it is numerically clarified that the necessity for the BS is eliminated, and the formation is controllable based on the obtained information

Constrained Reinforcement Learning using Distributional Representation for Trustworthy Quadrotor UAV Tracking Control

Simultaneously accurate and reliable tracking control for quadrotors in complex dynamic environments is challenging. As aerodynamics derived from drag forces and moment variations are chaotic and difficult to precisely identify, most current quadrotor tracking systems treat them as simple `disturbances' in conventional control approaches. We propose a novel, interpretable trajectory tracker integrating a Distributional Reinforcement Learning disturbance estimator for unknown aerodynamic effects with a Stochastic Model Predictive Controller (SMPC). The proposed estimator `Constrained Distributional Reinforced disturbance estimator' (ConsDRED) accurately identifies uncertainties between true and estimated values of aerodynamic effects. Simplified Affine Disturbance Feedback is used for control parameterization to guarantee convexity, which we then integrate with a SMPC. We theoretically guarantee that ConsDRED achieves at least an optimal global convergence rate and a certain sublinear rate if constraints are violated with an error decreases as the width and the layer of neural network increase. To demonstrate practicality, we show convergent training in simulation and real-world experiments, and empirically verify that ConsDRED is less sensitive to hyperparameter settings compared with canonical constrained RL approaches. We demonstrate our system improves accumulative tracking errors by at least 70% compared with the recent art. Importantly, the proposed framework, ConsDRED-SMPC, balances the tradeoff between pursuing high performance and obeying conservative constraints for practical implementationsComment: 16 pages, 8 figures. arXiv admin note: substantial text overlap with arXiv:2205.0715

Safe Reinforcement Learning as Wasserstein Variational Inference: Formal Methods for Interpretability

Reinforcement Learning or optimal control can provide effective reasoning for sequential decision-making problems with variable dynamics. Such reasoning in practical implementation, however, poses a persistent challenge in interpreting the reward function and corresponding optimal policy. Consequently, formalizing the sequential decision-making problems as inference has a considerable value, as probabilistic inference in principle offers diverse and powerful mathematical tools to infer the stochastic dynamics whilst suggesting a probabilistic interpretation of the reward design and policy convergence. In this study, we propose a novel Adaptive Wasserstein Variational Optimization (AWaVO) to tackle these challenges in sequential decision-making. Our approach utilizes formal methods to provide interpretations of reward design, transparency of training convergence, and probabilistic interpretation of sequential decisions. To demonstrate practicality, we show convergent training with guaranteed global convergence rates not only in simulation but also in real robot tasks, and empirically verify a reasonable tradeoff between high performance and conservative interpretability.Comment: 24 pages, 8 figures, containing Appendi

On Solving Close Enough Orienteering Problem with Overlapped Neighborhoods

The Close Enough Traveling Salesman Problem (CETSP) is a well-known variant of the classic Traveling Salesman Problem whereby the agent may complete its mission at any point within a target neighborhood. Heuristics based on overlapped neighborhoods, known as Steiner Zones (SZ), have gained attention in addressing CETSPs. While SZs offer effective approximations to the original graph, their inherent overlap imposes constraints on the search space, potentially conflicting with global optimization objectives. Here we present the Close Enough Orienteering Problem with Non-uniform Neighborhoods (CEOP-N), which extends CETSP by introducing variable prize attributes and non-uniform cost considerations for prize collection. To tackle CEOP-N, we develop a new approach featuring a Randomized Steiner Zone Discretization (RSZD) scheme coupled with a hybrid algorithm based on Particle Swarm Optimization (PSO) and Ant Colony System (ACS) - CRaSZe-AntS. The RSZD scheme identifies sub-regions for PSO exploration, and ACS determines the discrete visiting sequence. We evaluate the RSZD's discretization performance on CEOP instances derived from established CETSP instances, and compare CRaSZe-AntS against the most relevant state-of-the-art heuristic focused on single-neighborhood optimization for CEOP. We also compare the performance of the interior search within SZs and the boundary search on individual neighborhoods in the context of CEOP-N. Our results show CRaSZe-AntS can yield comparable solution quality with significantly reduced computation time compared to the single-neighborhood strategy, where we observe an averaged 140.44% increase in prize collection and 55.18% reduction of execution time. CRaSZe-AntS is thus highly effective in solving emerging CEOP-N, examples of which include truck-and-drone delivery scenarios.Comment: 26 pages, 10 figure

Formation control of robots in nonlinear two-dimensional potential

The formation control of multi-agent systems has garnered significant research attention in both theoretical and practical aspects over the past two decades. Despite this, the examination of how external environments impact swarm formation dynamics and the design of formation control algorithms for multi-agent systems in nonlinear external potentials have not been thoroughly explored. In this paper, we apply our theoretical formulation of the formation control algorithm to mobile robots operating in nonlinear external potentials. To validate the algorithm's effectiveness, we conducted experiments using real mobile robots. Furthermore, the results demonstrate the effectiveness of Dynamic Mode Decomposition in predicting the velocity of robots in unknown environments

TURNOVER RATE OF THE NEURONAL CONNEXIN CX36 IN HELA CELLS

Electrical synapses formed of the gap junction protein Cx36 show a great deal of functional plasticity, much dependent on changes in phosphorylation state of the connexin. However, gap junction turnover may also be important for regulating cell-cell communication, and turnover rates of Cx36 have not been studied. Connexins have relatively fast turnover rates, with short half-lives measured to be 1.5 to 3.5 hours in pulse-chase analyses of connexins (Cx26 and Cx43) in tissue culture cells and whole organs. We utilized HaloTag technology to study the turnover rate of Cx36 in transiently transfected HeLa cells. The HaloTag protein forms irreversible covalent bonds with chloroalkane ligands, allowing pulse-chase experiments to be performed very specifically. The HaloTag open reading frame was inserted into an internal site in the C-terminus of Cx36 designed not to disrupt the regulatory phosphorylation sites and not to block the C-terminal PDZ interaction motif. Functional properties of Cx36-Halo were assessed by Neurobiotin tracer coupling, live cell imaging, and immunostaining. For the pulse-chase study, transiently transfected HeLa cells were pulse labeled with Oregon Green (OG) HaloTag ligand and chase labeled at various times with tetramethylrhodamine (TMR) HaloTag ligand. Cx36-Halo formed large junctional plaques at sites of contact between transfected HeLa cells and was also contained in a large number of intracellular vesicles. The Cx36-Halo transfected HeLa cells supported Neurobiotin tracer coupling that was regulated by activation and inhibition of PKA in the same manner as wild-type Cx36 transfected cells. In the pulse-chase study, junctional protein labeled with the pulse ligand (OG) was gradually replaced by newly synthesized Cx36 labeled with the chase ligand (TMR). The half-life for turnover of protein in junctional plaques was 2.8 hours. Treatment of the pulse-labeled cells with Brefeldin A (BFA) prevented the addition of new connexins to junctional plaques, suggesting that the assembly of Cx36 into gap junctions involves the traditional ER-Golgi-TGN-plasma membrane pathway. In conclusion, Cx36-Halo is functional and has a turnover rate in HeLa cells similar to that of other connexins that have been studied. This turnover rate is likely too slow to contribute substantially to short-term changes in coupling of neurons driven by transmitters such as dopamine, which take minutes to achieve. However, turnover may contribute to longer-term changes in coupling

TWO-COLOR FLUORESCENT ANALYSIS OF CONNEXIN 36 TURNOVER AND TRAFFICKING – RELATIONSHIP TO FUNCTIONAL PLASTICITY

Gap junctions (GJ) formed of Cx36 show tremendous functional plasticity on several time scales. Changes in connexin phosphorylation modify coupling in minutes through an order of magnitude, but recent studies also imply involvement of connexin turnover in regulating cell-cell communication. We utilized Cx36 with an internal HaloTag to study Cx36 turnover and trafficking in cultured cells to discriminate newly formed and pre-existing Cx36. New Cx36 in cargo vesicles was added directly to existing gap junctions and newly made Cx36 was not confined to points of addition, but diffused throughout existing gap junctions. Existing connexins also diffused into photobleached areas with a half-time of less than 2 seconds. Recovery of connexin was impaired when laser power was focused and phototoxicity may be responsible. To better understand mechanisms of turnover we studied the role of cytoskeletal elements, actin filaments in particular, in Cx36 vesicle trafficking and GJ mobility. Phalloidin labeling showed that thick actin bundles connected all edges of GJ plaques, but actin filaments were rare within. Actin filaments were found associated with small, chase-labeled delivery vesicles. Many GJs showed substantial numbers of finger-like filadendrites extending from both the edges and the center of the plaques, and the morphology of these filadendrites changed at a fast pace. Double labeling of HaloTag ligand and phalloidin showed that these filadendrites colocalized with thin actin filaments. Disruption of actin filaments with Cytochalasin D caused loss of GJ at cell-cell contacts. Treatment with Latrunculin A, which prevents new actin elongation, did not disrupt GJ vi plaques and only partially suppressed Cx36 turnover, but eliminated the filadendrite extensions. In conclusion, studies of Cx36-HaloTag revealed novel features of connexin trafficking and demonstrated that phosphorylation-based changes in coupling occur on a different time scale than turnover. The role of rapid mobility of elements of GJ plaques in functional plasticity is unknown, but we hypothesize that it may relate to the mechanisms that control turnover of connexin protein

QuaDUE-CCM: Interpretable Distributional Reinforcement Learning using Uncertain Contraction Metrics for Precise Quadrotor Trajectory Tracking

Accuracy and stability are common requirements for Quadrotor trajectory tracking systems. Designing an accurate and stable tracking controller remains challenging, particularly in unknown and dynamic environments with complex aerodynamic disturbances. We propose a Quantile-approximation-based Distributional-reinforced Uncertainty Estimator (QuaDUE) to accurately identify the effects of aerodynamic disturbances, i.e., the uncertainties between the true and estimated Control Contraction Metrics (CCMs). Taking inspiration from contraction theory and integrating the QuaDUE for uncertainties, our novel CCM-based trajectory tracking framework tracks any feasible reference trajectory precisely whilst guaranteeing exponential convergence. More importantly, the convergence and training acceleration of the distributional RL are guaranteed and analyzed, respectively, from theoretical perspectives. We also demonstrate our system under unknown and diverse aerodynamic forces. Under large aerodynamic forces (>2m/s^2), compared with the classic data-driven approach, our QuaDUE-CCM achieves at least a 56.6% improvement in tracking error. Compared with QuaDRED-MPC, a distributional RL-based approach, QuaDUE-CCM achieves at least a 3 times improvement in contraction rate.Comment: 18 pages, 9 figures, Quadrotor trajectory tracking, Learning-based contro