Biped dynamic walking using reinforcement learning

This thesis presents a study of biped dynamic walking using reinforcement learning. A hardware biped robot was built. It uses low gear ratio DC motors in order to provide free leg movements. The Self Scaling Reinforcement learning algorithm was developed in order to deal with the problem of reinforcement learning in continuous action domains. A new learning architecture was designed to solve complex control problems. It uses different modules that consist of simple controllers and small neural networks. The architecture allows for easy incorporation of modules that represent new knowledge, or new requirements for the desired task. Control experiments were carried out using a simulator and the physical biped. The biped learned dynamic walking on flat surfaces without any previous knowledge about its dynamic model

DESIGN OF IIR DIGITAL FILTERS WITH CRITICAL MONOTONIC PASSBAND AMPLITUDE CHARACTERISTIC - A CASE STUDY

A case study is reported related to the design of IIR digital filters exhibiting critical monotonic amplitude characteristic (CMAC) in the pass band. This kind of amplitude characteristic offers several advantages as compared to its non-monotonic counterparts while it is not studied thoroughly, if at all. After giving a short overview of the way of generation CMACs, arguments will be listed in favor of the IIR version of the digital filter function realization. Next, the IIR implementation of the digital filters will be shortly considered. The main part of the paper will be devoted to the design sequence of this kind of filters which will be illustrated on the example of a band-pass filter obtained by a set of transformations from an all-pole low-pass analogue prototype. This will be for the first time a CMAC band pass IIR digital filters to be reported

An Unsupervised Neural Network for Real-Time Low-Level Control of a Mobile Robot: Noise Resistance, Stability, and Hardware Implementation

We have recently introduced a neural network mobile robot controller (NETMORC). The controller is based on earlier neural network models of biological sensory-motor control. We have shown that NETMORC is able to guide a differential drive mobile robot to an arbitrary stationary or moving target while compensating for noise and other forms of disturbance, such as wheel slippage or changes in the robot's plant. Furthermore, NETMORC is able to adapt in response to long-term changes in the robot's plant, such as a change in the radius of the wheels. In this article we first review the NETMORC architecture, and then we prove that NETMORC is asymptotically stable. After presenting a series of simulations results showing robustness to disturbances, we compare NETMORC performance on a trajectory-following task with the performance of an alternative controller. Finally, we describe preliminary results on the hardware implementation of NETMORC with the mobile robot ROBUTER.Sloan Fellowship (BR-3122), Air Force Office of Scientific Research (F49620-92-J-0499

Onboard multichannel demultiplexer/demodulator

An investigation performed for NASA LeRC by COMSAT Labs, of a digitally implemented on-board demultiplexer/demodulator able to process a mix of uplink carriers of differing bandwidths and center frequencies and programmable in orbit to accommodate variations in traffic flow is reported. The processor accepts high speed samples of the signal carried in a wideband satellite transponder channel, processes these as a composite to determine the signal spectrum, filters the result into individual channels that carry modulated carriers and demodulate these to recover their digital baseband content. The processor is implemented by using forward and inverse pipeline Fast Fourier Transformation techniques. The recovered carriers are then demodulated using a single digitally implemented demodulator that processes all of the modulated carriers. The effort has determined the feasibility of the concept with multiple TDMA carriers, identified critical path technologies, and assessed the potential of developing these technologies to a level capable of supporting a practical, cost effective on-board implementation. The result is a flexible, high speed, digitally implemented Fast Fourier Transform (FFT) bulk demultiplexer/demodulator

Intelligent failure-tolerant control

An overview of failure-tolerant control is presented, beginning with robust control, progressing through parallel and analytical redundancy, and ending with rule-based systems and artificial neural networks. By design or implementation, failure-tolerant control systems are 'intelligent' systems. All failure-tolerant systems require some degrees of robustness to protect against catastrophic failure; failure tolerance often can be improved by adaptivity in decision-making and control, as well as by redundancy in measurement and actuation. Reliability, maintainability, and survivability can be enhanced by failure tolerance, although each objective poses different goals for control system design. Artificial intelligence concepts are helpful for integrating and codifying failure-tolerant control systems, not as alternatives but as adjuncts to conventional design methods

Future supply chains enabled by continuous processing - opportunities and challenges : May 20–21, 2014 continuous manufacturing symposium

This paper examines the opportunities and challenges facing the pharmaceutical industry in moving to a primarily “continuous processing”-based supply chain. The current predominantly “large batch” and centralized manufacturing system designed for the “blockbuster” drug has driven a slow-paced, inventory heavy operating model that is increasingly regarded as inflexible and unsustainable. Indeed, new markets and the rapidly evolving technology landscape will drive more product variety, shorter product life-cycles, and smaller drug volumes, which will exacerbate an already unsustainable economic model. Future supply chains will be required to enhance affordability and availability for patients and healthcare providers alike despite the increased product complexity. In this more challenging supply scenario, we examine the potential for a more pull driven, near real-time demand-based supply chain, utilizing continuous processing where appropriate as a key element of a more “flow-through” operating model. In this discussion paper on future supply chain models underpinned by developments in the continuous manufacture of pharmaceuticals, we have set out; •The significant opportunities to moving to a supply chain flow-through operating model, with substantial opportunities in inventory reduction, lead-time to patient, and radically different product assurance/stability regimes. •Scenarios for decentralized production models producing a greater variety of products with enhanced volume flexibility. •Production, supply, and value chain footprints that are radically different from today's monolithic and centralized batch manufacturing operations. •Clinical trial and drug product development cost savings that support more rapid scale-up and market entry models with early involvement of SC designers within New Product Development. •The major supply chain and industrial transformational challenges that need to be addressed. The paper recognizes that although current batch operational performance in pharma is far from optimal and not necessarily an appropriate end-state benchmark for batch technology, the adoption of continuous supply chain operating models underpinned by continuous production processing, as full or hybrid solutions in selected product supply chains, can support industry transformations to deliver right-first-time quality at substantially lower inventory profiles

Research routing and MAC based on LEACH and S-MAC for energy efficiency and QoS in wireless sensor network

The wireless sensor is a micro-embedded device with weak data processing capability and small storage space. These nodes need to complete complex jobs, including data monitoring, acquisition and conversion, and data processing. Energy efficiency should be considered as one of the important aspects of the Wireless Sensor Network (WSN) throughout architecture and protocol design. At the same time, supporting Quality of Service (QoS) in WSNs is a research field, because the time-sensitive and important information is expected for the transmitting to to the sink node immediately. The thesis is supported by the projects entitled “The information and control system for preventing forest fires”, and “The Erhai information management system”, funded by the Chinese Government. Energy consumption and QoS are two main objectives of the projects. The thesis discusses the two aspects in route and Media Access Control (MAC). For energy efficiency, the research is based on Low Energy Adaptive Clustering Hierarchy (LEACH) protocol. LEACH is a benchmark clustering routing protocol which imposes upon cluster heads to complete a lot of aggregation and relay of messages to the base-station. However, there are limitations in LEACH. LEACH does not suit a wide area in clustering strategy and multi-hop routing. Moreover, routing protocols only focus on one factor, combining the clustering strategy and multi-hop routing mechanism were not considered in routing protocol for performance of network. QoS is supported by the MAC and routing protocol. Sensor MAC(S-MAC) makes the use of the periodically monitoring / sleeping mechanism, as well as collision and crosstalk avoidance mechanism. The mechanism reduces energy costs. Meanwhile, it supports good scalability and avoids the collision. However, the protocols do not take the differentiated services. For supporting QoS,A new route protocol needs to be designed and realized on embed platforms, which has WIFI mode and a Linux operation system to apply on the actual system. This research project was conducted as following the steps: A new protocol called RBLEACH is proposed to solve cluster on a widely scale based on LEACH. The area is divided into a few areas, where LEACH is improved to alter the selecting function in each area. RBLEACH creates routes selected by using a new algorithm to optimize the performance of the network. A new clustering method that has been developed to use several factors is PS-ACO-LEACH. The factors include the residual energy of the cluster head and Euclidean distances between cluster members and a cluster head. It can optimally solve fitness function and maintain a load balance in between the cluster head nodes, a cluster head and the base station. Based on the “Ant Colony” algorithm and transition of probability, a new routing protocol was created by “Pheromone” to find the optimal path of cluster heads to the base station. This protocol can reduce energy consumption of cluster heads and unbalanced energy consumption. Simulations prove that the improved protocol can enhance the performance of the network, including lifetime and energy conservation. Additionally, Multi Index Adaptive Routing Algorithm (MIA-QR) was designed based on network delay, packet loss rate and signal strength for QoS. The protocol is achieved by VC on an embedded Linux system. The MIA-QR is tested and verified by experiment and the protocol is to support QoS. Finally, an improved protocol (SMAC -SD) for wireless sensor networks is proposed, in order to solve the problem of S-MAC protocol that consider either service differentiation or ensure quality of service. According to service differentiation, SMAC-SD adopts an access mechanism based on different priorities including the adjustment of priority mechanisms of channel access probability, channel multi-request mechanisms and the configuring of waiting queues with different priorities and RTS backoff for different service, which makes the important service receive high channel access probability, ensuring the transmission quality of the important service. The simulation results show that the improved protocol is able to gain amount of important service and shortens the delay at the same time. Meanwhile, it improves the performance of the network effectivel