A discrete/rhythmic pattern generating RNN

Biological research supports the concept that advanced motion emerges from modular building blocks, which generate both rhythmical and discrete patterns. Inspired by these ideas, roboticists try to implement such building blocks using different techniques. In this paper, we show how to build such module by using a recurrent neural network (RNN) to encapsulate both discrete and rhythmical motion patterns into a single network. We evaluate the proposed system on a planar robotic manipulator. For training, we record several handwriting motions by back driving the robot manipulator. Finally, we demonstrate the ability to learn multiple motions (even discrete and rhythmic) and evaluate the pattern generation robustness in the presence of perturbations

An analysis of chaining in multi-label classification

The idea of classifier chains has recently been introduced as a promising technique for multi-label classification. However, despite being intuitively appealing and showing strong performance in empirical studies, still very little is known about the main principles underlying this type of method. In this paper, we provide a detailed probabilistic analysis of classifier chains from a risk minimization perspective, thereby helping to gain a better understanding of this approach. As a main result, we clarify that the original chaining method seeks to approximate the joint mode of the conditional distribution of label vectors in a greedy manner. As a result of a theoretical regret analysis, we conclude that this approach can perform quite poorly in terms of subset 0/1 loss. Therefore, we present an enhanced inference procedure for which the worst-case regret can be upper-bounded far more tightly. In addition, we show that a probabilistic variant of chaining, which can be utilized for any loss function, becomes tractable by using Monte Carlo sampling. Finally, we present experimental results confirming the validity of our theoretical findings

Feedback control by online learning an inverse model

A model, predictor, or error estimator is often used by a feedback controller to control a plant. Creating such a model is difficult when the plant exhibits nonlinear behavior. In this paper, a novel online learning control framework is proposed that does not require explicit knowledge about the plant. This framework uses two learning modules, one for creating an inverse model, and the other for actually controlling the plant. Except for their inputs, they are identical. The inverse model learns by the exploration performed by the not yet fully trained controller, while the actual controller is based on the currently learned model. The proposed framework allows fast online learning of an accurate controller. The controller can be applied on a broad range of tasks with different dynamic characteristics. We validate this claim by applying our control framework on several control tasks: 1) the heating tank problem (slow nonlinear dynamics); 2) flight pitch control (slow linear dynamics); and 3) the balancing problem of a double inverted pendulum (fast linear and nonlinear dynamics). The results of these experiments show that fast learning and accurate control can be achieved. Furthermore, a comparison is made with some classical control approaches, and observations concerning convergence and stability are made

Increasing recombinant protein production in Escherichia coli K12 by increasing the biomass yield of the host cell

For more than three decades micro-organisms have been employed as hosts for recombinant protein production, with the most popular organisms being Escherichia coli and Saccharomyces cerevisiae (1). One of the crucial factors to obtain high product yields in recombinant protein bioprocesses is the biomass yield of the host cell. High biomass yields not only result in less carbon loss and higher conversion to recombinant protein due to a potential higher drain of precursors, but are also accompanied by lower conversion to growth inhibiting byproducts, such as acetate (2). Furthermore, acidic byproducts hinder the expression of heterologous proteins (3) and consequently decrease protein yield in a direct and indirect manner. Many strategies have been tested to decrease the amount of acetate produced, including optimal feeding, choice of other carbon sources and metabolic engineering (4). Fed-batch and continuous feeding strategies result in low residual glucose concentrations and minimize overflow metabolism (’Crabtree effect’) (5; 6). Aristidou and coworkers improved biomass yield and protein production by using fructose as a primary carbon source without greatly affecting the fermentation cost (7). A third strategy is to alter the genetic machinery. Knocking out genes that code for acetate producing pathways, i.e. acetate kinase-phosphate acetyltransferase (ackA-pta) and pyruvate oxidase (poxB ) decrease acetate yield dramatically, but at the expense of lactate and pyruvate (8). The objective of this study was to focus on the combined effect of a global and a local regulator to increase biomass yield and hence recombinant protein production using GFP as a biomarker. Deletion of arcA reduces the repression on expression of TCA cycle genes (9) while deletion of iclR removes the repression on the aceBAK operon and opens the glyoxylate pathway (10; 11) in aerobic batch cultivations. This metabolic engineering approach simultaneously decreased the acetate yield with 70% and increased the biomass yield of the host cell with 50%. Due to a lower carbon loss and a lower inhibition of protein production by acetate, the GFP production of the ∆arcA∆iclR double knockout strain increased with 100% as opposed to the wild type E. coli K12. Further deletion of genes lon and ompT encoding for non-specific proteases even further increases GFP-production (3 times the wild type value). The effect of a deletion of arcA and iclR was also evaluated in a E. coli BL21 genetic background. However in this industrial strain the deletion had no effect on protein production. References [1] Ferrer-Miralles N, Domingo-Esp ́ J, Corchero JL, V ́zquez E, Villaverde A: Microbial factories for recombinant pharmaceuticals. Microb Cell Fact 2009, 8:17 [2] El-Mansi EM, Holms WH: Control of carbon flux to acetate excretion during growth of Escherichia coli in batch and continuous cultures. J Gen Microbiol 1989, 135(11):2875–2883. [3] Jensen EB, Carlsen S: Production of recombinant human growth hormone in Escherichia coli: expression of different precursors and physiological effects of glucose, acetate, and salts. Biotechnol Bioeng 1990, 36:1–11 [4] De Mey M, Maeseneire SD, Soetaert W, Vandamme E: Minimizing acetate formation in E. coli fermentations. J. Ind. Microbiol. Biotechnol. 2007, 34:689–700. [5] Babaeipour V, Shojaosadati SA, Khalilzadeh R, Maghsoudi N, Tabandeh F: A proposed feeding strategy for the overproduction of recombinant proteins in Escherichia coli. Biotechnol Appl Biochem 2008, 49(Pt 2):141–147. [6] San KY, Bennett GN, Aristidou AA, Chou CH: Strategies in high-level expression of recombinant protein in Escherichia coli. Ann N Y Acad Sci 1994, 721:257–267. [7] Aristidou AA, San KY, Bennett GN: Improvement of biomass yield and recombinant gene expression in Escherichia coli by using fructose as the primary carbon source. Biotechnol Prog 1999, 15:140–145. [8] De Mey M, Lequeux GJ, Beauprez JJ, Maertens J, Horen EV, Soetaert WK, Vanrolleghem PA, Vandamme EJ: Comparison of different strategies to reduce acetate formation in Escherichia coli. Biotechnol Prog 2007. [9] Perrenoud A, Sauer U: Impact of global transcriptional regulation by ArcA, ArcB, Cra,Crp, Cya, Fnr, and Mlc on glucose catabolism in Escherichia coli . J. Bacteriol. 2005, 187:3171–3179. [10] van de Walle M, Shiloach J: Proposed mechanism of acetate accumulation in two recombinant Escherichia coli strains during high density fermentation. Biotechnol Bioeng 1998, 57:71–78. [11] Maharjan RP, Yu PL, Seeto S, Ferenci T: The role of isocitrate lyase and the glyoxylate cycle in Escherichia coli growing under glucose limitation. Res Microbiol 2005, 156(2):178–183

Adaptive control of compliant robots with Reservoir Computing

In modern society, robots are increasingly used to handle dangerous, repetitive and/or heavy tasks with high precision. Because of the nature of the tasks, either being dangerous, high precision or simply repetitive, robots are usually constructed with high torque motors and sturdy materials, that makes them dangerous for humans to handle. In a car-manufacturing company, for example, a large cage is placed around the robot’s workspace that prevents humans from entering its vicinity. In the last few decades, efforts have been made to improve human-robot interaction. Often the movement of robots is characterized as not being smooth and clearly dividable into sub-movements. This makes their movement rather unpredictable for humans. So, there exists an opportunity to improve the motion generation of robots to enhance human-robot interaction. One interesting research direction is that of imitation learning. Here, human motions are recorded and demonstrated to the robot. Although the robot is able to reproduce such movements, it cannot be generalized to other situations. Therefore, a dynamical system approach is proposed where the recorded motions are embedded into the dynamics of the system. Shaping these nonlinear dynamics, according to recorded motions, allows for dynamical system to generalize beyond demonstration. As a result, the robot can generate motions of other situations not included in the recorded human demonstrations. In this dissertation, a Reservoir Computing approach is used to create a dynamical system in which such demonstrations are embedded. Reservoir Computing systems are Recurrent Neural Network-based approaches that are efficiently trained by considering only the training of the readout connections and retaining all other connections of such a network unchanged given their initial randomly chosen values. Although they have been used to embed periodic motions before, they were extended to embed discrete motions, or both. This work describes how such a motion pattern-generating system is built, investigates the nature of the underlying dynamics and evaluates their robustness in the face of perturbations. Additionally, a dynamical system approach to obstacle avoidance is proposed that is based on vector fields in the presence of repellers. This technique can be used to extend the motion abilities of the robot without need for changing the trained Motion Pattern Generator (MPG). Therefore, this approach can be applied in real-time on any system that generates a certain movement trajectory. Assume that the MPG system is implemented on an industrial robotic arm, similar to the ones used in a car factory. Even though the obstacle avoidance strategy presented is able to modify the generated motion of the robot’s gripper in such a way that it avoids obstacles, it does not guarantee that other parts of the robot cannot collide with a human. To prevent this, engineers have started to use advanced control algorithms that measure the amount of torque that is applied on the robot. This allows the robot to be aware of external perturbations. However, it turns out that, even with fast control loops, the adaptation to compensate for a sudden perturbation, is too slow to prevent high interaction forces. To reduce such forces, researchers started to use mechanical elements that are passively compliant (e.g., springs) and light-weight flexible materials to construct robots. Although such compliant robots are much safer and inherently energy efficient to use, their control becomes much harder. Most control approaches use model information about the robot (e.g., weight distribution and shape). However, when constructing a compliant robot it is hard to determine the dynamics of these materials. Therefore, a model-free adaptive control framework is proposed that assumes no prior knowledge about the robot. By interacting with the robot it learns an inverse robot model that is used as controller. The more it interacts, the better the control be- comes. Appropriately, this framework is called Inverse Modeling Adaptive (IMA) control framework. I have evaluated the IMA controller’s tracking ability on sev- eral tasks, investigating its model independence and stability. Furthermore, I have shown its fast learning ability and comparable performance to taskspecific designed controllers. Given both the MPG and IMA controllers, it is possible to improve the inter- actability of a compliant robot in a human-friendly environment. When the robot is to perform human-like motions for a large set of tasks, we need to demonstrate motion examples of all these tasks. However, biological research concerning the motion generation of animals and humans revealed that a limited set of motion patterns, called motion primitives, are modulated and combined to generate advanced motor/motion skills that humans and animals exhibit. Inspired by these interesting findings, I investigate if a single motion primitive indeed can be modulated to achieve a desired motion behavior. By some elementary experiments, where an MPG is controlled by an IMA controller, a proof of concept is presented. Furthermore, a general hierarchy is introduced that describes how a robot can be controlled in a biology-inspired manner. I also investigated how motion primitives can be combined to produce a desired motion. However, I was unable to get more advanced implementations to work. The results of some simple experiments are presented in the appendix. Another approach I investigated assumes that the primitives themselves are undefined. Instead, only a high-level description is given, which describes that every primitive on average should contribute equally, while still allowing for a single primitive to specialize in a part of the motion generation. Without defining the behavior of a primitive, only a set of untrained IMA controllers is used of which each will represent a single primitive. As a result of the high-level heuristic description, the task space is tiled into sub-regions in an unsupervised manner. Resulting in controllers that indeed represent a part of the motion generation. I have applied this Modular Architecture with Control Primitives (MACOP) on an inverse kinematic learning task and investigated the emerged primitives. Thanks to the tiling of the task space, it becomes possible to control redundant systems, because redundant solutions can be spread over several control primitives. Within each sub region of the task space, a specific control primitive is more accurate than in other regions allowing for the task complexity to be distributed over several less complex tasks. Finally, I extend the use of an IMA-controller, which is tracking controller, to the control of under-actuated systems. By using a sample-based planning algorithm it becomes possible to explore the system dynamics in which a path to a desired state can be planned. Afterwards, MACOP is used to incorporate feedback and to learn the necessary control commands corresponding to the planned state space trajectory, even if it contains errors. As a result, the under-actuated control of a cart pole system was achieved. Furthermore, I presented the concept of a simulation based control framework that allows the learning of the system dynamics, planning and feedback control iteratively and simultaneously

Exact and efficient top-K inference for multi-target prediction by querying separable linear relational models

Many complex multi-target prediction problems that concern large target spaces are characterised by a need for efficient prediction strategies that avoid the computation of predictions for all targets explicitly. Examples of such problems emerge in several subfields of machine learning, such as collaborative filtering, multi-label classification, dyadic prediction and biological network inference. In this article we analyse efficient and exact algorithms for computing the top-$K$ predictions in the above problem settings, using a general class of models that we refer to as separable linear relational models. We show how to use those inference algorithms, which are modifications of well-known information retrieval methods, in a variety of machine learning settings. Furthermore, we study the possibility of scoring items incompletely, while still retaining an exact top-K retrieval. Experimental results in several application domains reveal that the so-called threshold algorithm is very scalable, performing often many orders of magnitude more efficiently than the naive approach