A Multi-objective Exploratory Procedure for Regression Model Selection

Variable selection is recognized as one of the most critical steps in statistical modeling. The problems encountered in engineering and social sciences are commonly characterized by over-abundance of explanatory variables, non-linearities and unknown interdependencies between the regressors. An added difficulty is that the analysts may have little or no prior knowledge on the relative importance of the variables. To provide a robust method for model selection, this paper introduces the Multi-objective Genetic Algorithm for Variable Selection (MOGA-VS) that provides the user with an optimal set of regression models for a given data-set. The algorithm considers the regression problem as a two objective task, and explores the Pareto-optimal (best subset) models by preferring those models over the other which have less number of regression coefficients and better goodness of fit. The model exploration can be performed based on in-sample or generalization error minimization. The model selection is proposed to be performed in two steps. First, we generate the frontier of Pareto-optimal regression models by eliminating the dominated models without any user intervention. Second, a decision making process is executed which allows the user to choose the most preferred model using visualisations and simple metrics. The method has been evaluated on a recently published real dataset on Communities and Crime within United States.Comment: in Journal of Computational and Graphical Statistics, Vol. 24, Iss. 1, 201

Incorporating Human Preferences in Decision Making for Dynamic Multi-Objective Optimization in Model Predictive Control

We present a new two-step approach for automatized a posteriori decision making in multi-objective optimization problems, i.e., selecting a solution from the Pareto front. In the first step, a knee region is determined based on the normalized Euclidean distance from a hyperplane defined by the furthest Pareto solution and the negative unit vector. The size of the knee region depends on the Pareto front’s shape and a design parameter. In the second step, preferences for all objectives formulated by the decision maker, e.g., 50–20–30 for a 3D problem, are translated into a hyperplane which is then used to choose a final solution from the knee region. This way, the decision maker’s preference can be incorporated, while its influence depends on the Pareto front’s shape and a design parameter, at the same time favorizing knee points if they exist. The proposed approach is applied in simulation for the multi-objective model predictive control (MPC) of the two-dimensional rocket car example and the energy management system of a building

Negotiating Large Obstacles with a Humanoid Robot via Multi-Contact Motion Planning

Incremental progress in humanoid robot locomotion over the years has achieved essential capabilities such as navigation over at or uneven terrain, stepping over small obstacles and imbing stairls. However, the locomotion research has mostly been limited to using only bipedal gait and only foot contacts with the environment, using the upper body for balancing without considering additional external contacts. As a result, challenging locomotion tasks like climbing over large obstacles relative to the size of the robot have remained unsolved. In this paper, we address this class of open problems with an approach based on multi-contact motion planning, guided by physical human demonstrations. Our goal is to make humanoid locomotion problem more tractable by taking advantage of objects in the surrounding environment instead of avoiding them. We propose a multi-contact motion planning algorithm for humanoid robot locomotion which exploits the multi-contacts at the upper and lower body limbs. We propose a contact stability measure, which simplies the contact search from demonstration and contact transition motion generation for the multi-contact motion planning algorithm. The algorithm uses the whole-body motions generated via Quadratic Programming (QP) based solver methods. The multi-contact motion planning algorithm is applied for a challenging task of climbing over a relatively larger obstacle compared to the robot. We validate our planning approach with simulations and experiments for climbing over a large wooden obstacle with COMAN, which is a complaint humanoid robot with 23 degrees of freedom (DOF). We also propose a generalization method, the \Policy-Contraction Learning Method" to extend the algorithm for generating new multi-contact plans for our multi-contact motion planner, that can adapt to changes in the environment. The method learns a general policy and the multi-contact behavior from the human demonstrations, for generating new multi-contact plans for the obstacle-negotiation

Decision making for multi‐objective problems: Mean and median metrics

When dealing with problems with more than two objectives, sophisticated multi‐objective optimization algorithms might be needed. Pareto optimization, which is based on the concept of dominated and non‐dominated solutions, is the most widely utilized method when comparing solutions within a multi‐objective setting. However, in the context of optimization, where three or more objectives are involved, the effectiveness of Pareto dominance approaches to drive the solutions to convergence is significantly compromised as more and more solutions tend to be non‐dominated by each other. This in turn reduces the selection pressure, especially for algorithms that rely on evolving a population of solutions such as evolutionary algorithms, particle swarm optimization, differential evolution, etc. The size of the non‐dominated set of trade‐off solutions can be quite large, rendering the decision‐making process difficult if not impossible. The size of the non‐dominated solution set increases exponentially with an increase in the number of objectives. This paper aims to expand a framework for coping with many/multi‐objective and multidisciplinary optimization problems through the introduction of a min‐max metric that behaves like a median measure that can locate the center of a data set. We compare this metric to the Chebyshev norm L_∞ metric that behaves like a mean measure in locating the center of a data set. The median metric is introduced in this paper for the first time, and unlike the mean metric is independent of the data normalization method. These metrics advocate balanced, natural, and minimum compromise solutions about all objectives. We also demonstrate and compare the behavior of the two metrics for a Tradespace case study involving more than 1200 CubeSat design alternatives identifying a manageable set of potential solutions for decision‐makers

Understanding knee points in bicriteria problems and their implications as preferred solution principles

A knee point is almost always a preferred trade-off solution, if it exists in a bicriteria optimization problem. In this article, an attempt is made to improve understanding of a knee point and investigate the properties of a bicriteria problem that may exhibit a knee on its Pareto-optimal front. Past studies are reviewed and a couple of new definitions are suggested. Additionally, a knee region is defined for problems in which, instead of one, a set of knee-like solutions exists. Edge-knee solutions, which behave like knee solutions but lie near one of the extremes on the Pareto-optimal front, are also introduced. It is interesting that in many problem-solving tasks, despite the existence of a number of solution methodologies, only one or a few of them are commonly used. Here, it is argued that often such common solution principles are knee solutions to a bicriteria problem formed with two conflicting goals of the underlying problem-solving task. The argument is illustrated on a number of tasks, such as regression, sorting, clustering and a number of engineering designs

Sensor-Based Adaptive Control and Optimization of Lower-Limb Prosthesis.

Recent developments in prosthetics have enabled the development of powered prosthetic ankles (PPA). The advent of such technologies drastically improved impaired gait by increasing balance and reducing metabolic energy consumption by providing net positive power. However, control challenges limit performance and feasibility of today’s devices. With addition of sensors and motors, PPA systems should continuously make control decisions and adapt the system by manipulating control parameters of the prostheses. There are multiple challenges in optimization and control of PPAs. A prominent challenge is the objective setup of the system and calibration parameters to fit each subject. Another is whether it is possible to detect changes in intention and terrain before prosthetic use and how the system should react and adapt to it. In the first part of this study, a model for energy expenditure was proposed using electromyogram (EMG) signals from the residual lower-limbs PPA users. The proposed model was optimized to minimize energy expenditure. Optimization was performed using a modified Nelder-Mead approach with a Latin Hypercube sampling. Results of the proposed method were compared to expert values and it was shown to be a feasible alternative for tuning in a shorter time. In the second part of the study, the control challenges regarding lack of adaptivity for PPAs was investigated. The current PPA system used is enhanced with impedance-controlled parameters that allow the system to provide different assistance. However, current systems are set to a fixed value and fail to acknowledge various terrain and intentions throughout the day. In this study, a pseudo-real-time adaptive control system was proposed to predict the changes in the gait and provide a smoother gait. The proposed control system used physiological, kinetic, and kinematic data and fused them to predict the change. The prediction was done using machine learning-based methods. Results of the study showed an accuracy of up to 89.7 percent for prediction of change for four different cases

Multi-objective design optimization of a mobile-bearing total disc arthroplasty considering spinal kinematics, facet joint loads, and metal-on-polyethylene contact mechanics

Total disc arthroplasty (TDA) is a motion-preserving surgical technique used to treat spinal disorders, when more conservative medical therapies fail. Unfortunately, a high incidence of revision surgery exists due to postoperative complications including abnormal kinematics, facet joint arthritis, and implant failures. However, TDA is still an attractive option, since an optimally designed artificial disc is expected to reproduce native segmental biomechanics. Correspondingly, it would mitigate the development of adjacent segment diseases (a major concern of spinal fusion) caused by altered segmental biomechanics. Design optimization is a process of finding the best design parameters for a component/system to satisfy one/multiple design requirements using optimization algorithms. The shape of a candidate design is parametrized using computer-aided design, such that design parameters are manipulated to minimize one/multiple objective functions subject to performance constraints and design space bounds. Optimization algorithms typically require the gradients of the objective/constraint functions with respect to each design variable. In the traditional design optimization, due to the high computational cost to calculate the gradients by performing finite element analysis in each optimization iteration, it often results in a slow process to seek the optimal solution. To address the problem, an artificial neural network (ANN) was implemented to derive the analytical expressions of the objective/constraint function and their gradients. By incorporating analytical gradients, we successfully developed a multiobjective optimization (MOO) framework considering three performance metrics simultaneously. Furthermore, a new mobile-bearing TDA design concept featuring a biconcave polyethylene (PE) core was proposed, to strengthen the PE rim, where a high risk of fracture exists. It was hypothesized that there is a trade-off relationship among postoperative performance metrics in terms of spinal kinematics, facet joint loading, and metal-on-polyethylene contact mechanics. We tested this hypothesis by refining the new TDA to match normal segmental biomechanics and alleviate PE core stress. After performing MOO, the best-trade-off TDA design was determined by the solved three-dimensional Pareto frontier. The novel MOO framework can be also used to improve existing TDA designs, as well as to push the cutting edge of surgical techniques for the treatment of spinal disorders