Designing mixture of deep experts

Mixture of Experts (MoE) is a classical architecture for ensembles where each member is specialised in a given part of the input space or its expertise area. Working in this manner, we aim to specialise the experts on smaller problems, solving the original problem through some type of divide and conquer approach. The goal of our research is to initially reproduce the work done by Collobert et al[1] , 2002 followed by extending this work by using neural networks as experts on different datasets. Specialised representations will be learned over different aspects of the problem, and the results of the different members will be merged according to their specific expertise. This expertise can then be learned itself by a given network acting as a gating function. MOE architecture composed on N expert networks. These experts are combined via a gating network, which partition the input space accordingly. It is based on divide and conquer strategy supervised by a gating network. Using a specialised cost function the experts specialise in their sub-space. Using the discriminative power of experts is much better than simply clustering. The gating network needs to needs to learn how to assign examples to different specialists. Such models show promise for building larger networks that are still cheap to compute at test time, and more parallelizable at training time. We were able to reproduce the work by the author and implemented a multi-class gater to classify images. We know that Neural Networks perform the best with lots of data. However, some of our experiments require us to divide the dataset and train multiple Neural Networks. We observe that in data deprived condition our MoE are almost on par and compete with ensembles trained on complete data. Keywords : Machine Learning, Multi Layer Perceptrons, Mixture of Experts, Support Vector Machines, Divide and Conquer, Stochastic Gradient Descent, Optimization

Determination of stiffness reduction factor for U-shaped reinforced concrete shear walls under bi-axial loading

Reinforced Concrete (RC) shear wall, is an effective primary earthquake resisting system due to strong stiffness and large shear-force resisting capacity. For a complex asymmetric wall, severe damage on a portion of the wall may directly affect the stiffness in other directions. Such a secondary damage mechanism is hard to capture. Hence, this study was devoted to determining a stiffness reduction index that can monitor current damage state of the wall system as a whole, and apply the unified damage index to decrease stiffness and strength on other directions. This study proposes an analytical framework at microscopic length scale that is based on a unit cell which consists of nonlinear steel spring, compression only gap, and concrete compression spring. For validation and applications, three U-shaped wall specimens available in literature (designed according to EC8) were modeled and simulated under cyclic lateral loading. These walls have the same dimensions and reinforcement except for the different loading directions. The present study concludes that the proposed unit cell model appears to be successful for predicting the stiffness reductions resulting from localized damages in different loading directions. The proposed unit cell-based framework seems to be a good starting point to consider secondary stiffness reductions for other complex non-rectangle walls such as L-, H- and T-shaped walls. This method may facilitate the fast determination of remaining stiffness of complex RC walls by using quick post-disaster observations

Optimal Routing of Unmanned Vehicles in Persistent Monitoring Missions

Missions such as forest fire monitoring, military surveillance and infrastructure monitoring are referred to as persistent monitoring missions. These missions rely heavily on continual data collection from various locations, referred to as targets. In this dissertation, we consider a framework in which data is collected from the targets with the aid of unmanned aerial vehicles (UAVs). A UAV makes a physical visit to the targets for data collection, and immediately transmits the collected data to a base station for further analysis. Typically, the duration of these monitoring missions is long, and the monitoring vehicles are required to stay in flight for extended periods of time. Therefore, the batteries powering the UAVs must be recharged regularly at a recharging station/depot. From utilitarian and economic points of view, an efficient execution of these missions calls for two requisites: 1) minimizing the time delay between successive data collections at targets; 2) maximizing the total charge/energy drawn from batteries. The maximum time delay between successive data collections at any target is characterized by a function referred to as the walk revisit time, or simply the revisit time. Given a set of targets and a UAV tasked with monitoring the targets, the charge capacity of the battery powering the UAV can be surrogated by the number of visits the UAV can make to the targets without requiring a recharge. To minimize the wastage of energy resources, a charge penalty is imposed on the visits that are unutilized before each recharge. The aim of this work is to find optimal routes for the UAV(s) to visit the targets such that the sum of the revisit time and the charge penalty is minimized. The optimal route planning problem is determined by a number of factors such as the number of UAVs used for monitoring, the aerial platform on which the monitoring UAVs are built, the location of their depots, relative importance of the targets being monitored, etc. In this dissertation, we focus on equally weighted targets and address four different variants of the problem, all of which are computationally extremely challenging. The variants considered are the following: 1) single UAV with no motion constraints and the depot located at one of the targets; 2) single UAV with curvature constraints on its path and the depot located at one of the targets; 3) single UAV with no motion constraints and its depot stationed at a location different from that of the targets; 4) multiple UAVs with no motion constraints with their depots located at the targets. This dissertation builds on the results of Variant 1; specifically, the characterization of the optimal solutions proved in this dissertation is the main contribution of this dissertation; it lends itself to a new formulation of the same problem that results in significant computational savings. The structural characterization also holds for Variant 2. Inspired by this result, conjectures are provided for the structure of optimal solution for variant 3 and is backed up by extensive numerical simulations. Variant 3 can also be perceived as a special case of targets with different weights/priorities, and therefore, the results developed in this dissertation can potentially be extended to solve a few special cases of the general problem involving arbitrarily weighted targets

Combinatorial Path Planning for a System of Multiple Unmanned Vehicles

In this dissertation, the problem of planning the motion of m Unmanned Vehicles (UVs) (or simply vehicles) through n points in a plane is considered. A motion plan for a vehicle is given by the sequence of points and the corresponding angles at which each point must be visited by the vehicle. We require that each vehicle return to the same initial location(depot) at the same heading after visiting the points. The objective of the motion planning problem is to choose at most q(≤ m) UVs and find their motion plans so that all the points are visited and the total cost of the tours of the chosen vehicles is a minimum amongst all the possible choices of vehicles and their tours. This problem is a generalization of the wellknown Traveling Salesman Problem (TSP) in many ways: (1) each UV takes the role of salesman (2) motion constraints of the UVs play an important role in determining the cost of travel between any two locations; in fact, the cost of the travel between any two locations depends on direction of travel along with the heading at the origin and destination, and (3) there is an additional combinatorial complexity stemming from the need to partition the points to be visited by each UV and the set of UVs that must be employed by the mission. In this dissertation, a sub-optimal, two-step approach to motion planning is presented to solve this problem:(1) the combinatorial problem of choosing the vehicles and their associated tours is based on Euclidean distances between points and (2) once the sequence of points to be visited is specified, the heading at each point is determined based on a Dynamic Programming scheme. The solution to the first step is based on a generalization of Held-Karp’s method. We modify the Lagrangian heuristics for finding a close sub-optimal solution. In the later chapters of the dissertation, we relax the assumption that all vehicles are homogenous. The motivation of heterogenous variant of Multi-depot, Multiple Traveling Salesmen Problem (MDMTSP) derives form applications involving Unmanned Aerial Vehicles (UAVs) or ground robots requiring multiple vehicles with different capabilities to visit a set of locations

Development of Modeling and Simulation Platform for Path-Planning and Control of Autonomous Underwater Vehicles in Three-Dimensional Spaces

Autonomous underwater vehicles (AUVs) operating in deep sea and littoral environments have diverse applications including marine biology exploration, ocean environment monitoring, search for plane crash sites, inspection of ship-hulls and pipelines, underwater oil rig maintenance, border patrol, etc. Achieving autonomy in underwater vehicles relies on a tight integration between modules of sensing, navigation, decision-making, path-planning, trajectory tracking, and low-level control. This system integration task benefits from testing the related algorithms and techniques in a simulated environment before implementation in a physical test bed. This thesis reports on the development of a modeling and simulation platform that supports the design and testing of path planning and control algorithms in a synthetic AUV, representing a simulated version of a physical AUV. The approach allows integration between path-planners and closed-loop controllers that enable the synthetic AUV to track dynamically feasible trajectories in three-dimensional spaces. The dynamical behavior of the AUV is modeled using the equations of motion that incorporate the effects of external forces (e.g., buoyancy, gravity, hydrodynamic drag, centripetal force, Coriolis force, etc.), thrust forces, and inertial forces acting on the AUV. The equations of motion are translated into a state space formulation and the S-function feature of the Simulink and MATLAB scripts are used to evolve the state trajectories from initial conditions. A three-dimensional visualization of the resulting AUV motion is achieved by feeding the corresponding position and orientation states into an animation code. Experimental validation is carried out by performing integrated waypoint planner (e.g., using the popular A* algorithm) and PD controller implementations that allow the traversal of the synthetic AUV in two-dimensional (XY, XZ, YZ) and three-dimensional spaces. An underwater pipe-line inspection task carried out by the AUV is demonstrated in a simulated environment. The simulation testbed holds a potential to support planner and controller design for implementation in physical AUVs, thereby allowing exploration of various research topics in the field