TreeGrad: Transferring Tree Ensembles to Neural Networks

Gradient Boosting Decision Tree (GBDT) are popular machine learning algorithms with implementations such as LightGBM and in popular machine learning toolkits like Scikit-Learn. Many implementations can only produce trees in an offline manner and in a greedy manner. We explore ways to convert existing GBDT implementations to known neural network architectures with minimal performance loss in order to allow decision splits to be updated in an online manner and provide extensions to allow splits points to be altered as a neural architecture search problem. We provide learning bounds for our neural network.Comment: Technical Report on Implementation of Deep Neural Decision Forests Algorithm. To accompany implementation here: https://github.com/chappers/TreeGrad. Update: Please cite as: Siu, C. (2019). "Transferring Tree Ensembles to Neural Networks". International Conference on Neural Information Processing. Springer, 2019. arXiv admin note: text overlap with arXiv:1909.1179

Integration of visual and joint information to enable linear reaching motions

A new dynamics-driven control law was developed for a robot arm, based on the feedback control law which uses the linear transformation directly from work space to joint space. This was validated using a simulation of a two-joint planar robot arm and an optimisation algorithm was used to find the optimum matrix to generate straight trajectories of the end-effector in the work space. We found that this linear matrix can be decomposed into the rotation matrix representing the orientation of the goal direction and the joint relation matrix (MJRM) representing the joint response to errors in the Cartesian work space. The decomposition of the linear matrix indicates the separation of path planning in terms of the direction of the reaching motion and the synergies of joint coordination. Once the MJRM is numerically obtained, the feedfoward planning of reaching direction allows us to provide asymptotically stable, linear trajectories in the entire work space through rotational transformation, completely avoiding the use of inverse kinematics. Our dynamics-driven control law suggests an interesting framework for interpreting human reaching motion control alternative to the dominant inverse method based explanations, avoiding expensive computation of the inverse kinematics and the point-to-point control along the desired trajectories

Comparative run-time performance of evolutionary algorithms on multi-objective interpolated continuous optimisation problems.

We propose a new class of multi-objective benchmark problems on which we analyse the performance of four well established multi-objective evolutionary algorithms (MOEAs) – each implementing a different search paradigm – by comparing run-time convergence behaviour over a set of 1200 problem instances. The new benchmarks are created by fusing previously proposed single-objective interpolated continuous optimisation problems (ICOPs) via a common set of Pareto non-dominated seeds. They thus inherit the ICOP property of having tunable fitness landscape features. The benchmarks are of intrinsic interest as they derive from interpolation methods and so can approximate general problem instances. This property is revealed to be of particular importance as our extensive set of numerical experiments indicates that choices pertaining to (i) the weighting of the inverse distance interpolation function and (ii) the problem dimension can be used to construct problems that are challenging to all tested multi-objective search paradigms. This in turn means that the new multi-objective ICOPs problems (MO-ICOPs) can be used to construct well-balanced benchmark sets that discriminate well between the run-time convergence behaviour of different solvers

Locomotor adaptation to a powered ankle-foot orthosis depends on control method

<p>Abstract</p> <p>Background</p> <p>We studied human locomotor adaptation to powered ankle-foot orthoses with the intent of identifying differences between two different orthosis control methods. The first orthosis control method used a footswitch to provide bang-bang control (a kinematic control) and the second orthosis control method used a proportional myoelectric signal from the soleus (a physiological control). Both controllers activated an artificial pneumatic muscle providing plantar flexion torque.</p> <p>Methods</p> <p>Subjects walked on a treadmill for two thirty-minute sessions spaced three days apart under either footswitch control (n = 6) or myoelectric control (n = 6). We recorded lower limb electromyography (EMG), joint kinematics, and orthosis kinetics. We compared stance phase EMG amplitudes, correlation of joint angle patterns, and mechanical work performed by the powered orthosis between the two controllers over time.</p> <p>Results</p> <p>During steady state at the end of the second session, subjects using proportional myoelectric control had much lower soleus and gastrocnemius activation than the subjects using footswitch control. The substantial decrease in triceps surae recruitment allowed the proportional myoelectric control subjects to walk with ankle kinematics close to normal and reduce negative work performed by the orthosis. The footswitch control subjects walked with substantially perturbed ankle kinematics and performed more negative work with the orthosis.</p> <p>Conclusion</p> <p>These results provide evidence that the choice of orthosis control method can greatly alter how humans adapt to powered orthosis assistance during walking. Specifically, proportional myoelectric control results in larger reductions in muscle activation and gait kinematics more similar to normal compared to footswitch control.</p

Granger Causality Mapping during Joint Actions Reveals Evidence for Forward Models That Could Overcome Sensory-Motor Delays

Studies investigating joint actions have suggested a central role for the putative mirror neuron system (pMNS) because of the close link between perception and action provided by these brain regions [1], [2], [3]. In contrast, our previous functional magnetic resonance imaging (fMRI) experiment demonstrated that the BOLD response of the pMNS does not suggest that it directly integrates observed and executed actions during joint actions [4]. To test whether the pMNS might contribute indirectly to the integration process by sending information to brain areas responsible for this integration (integration network), here we used Granger causality mapping (GCM) [5]. We explored the directional information flow between the anterior sites of the pMNS and previously identified integrative brain regions. We found that the left BA44 sent more information than it received to both the integration network (left thalamus, right middle occipital gyrus and cerebellum) and more posterior nodes of the pMNS (BA2). Thus, during joint actions, two anatomically separate networks therefore seem effectively connected and the information flow is predominantly from anterior to posterior areas of the brain. These findings suggest that the pMNS is involved indirectly in joint actions by transforming observed and executed actions into a common code and is part of a generative model that could predict the future somatosensory and visual consequences of observed and executed actions in order to overcome otherwise inevitable neural delays

Leadership in Orchestra Emerges from the Causal Relationships of Movement Kinematics

Non-verbal communication enables efficient transfer of information among people. In this context, classic orchestras are a remarkable instance of interaction and communication aimed at a common aesthetic goal: musicians train for years in order to acquire and share a non-linguistic framework for sensorimotor communication. To this end, we recorded violinists' and conductors' movement kinematics during execution of Mozart pieces, searching for causal relationships among musicians by using the Granger Causality method (GC). We show that the increase of conductor-to-musicians influence, together with the reduction of musician-to-musician coordination (an index of successful leadership) goes in parallel with quality of execution, as assessed by musical experts' judgments. Rigorous quantification of sensorimotor communication efficacy has always been complicated and affected by rather vague qualitative methodologies. Here we propose that the analysis of motor behavior provides a potentially interesting tool to approach the rather intangible concept of aesthetic quality of music and visual communication efficacy

Predictive feedback control and Fitts' law

Fitts’ law is a well established empirical formula, known for encapsulating the “speed-accuracy trade-off”. For discrete, manual movements from a starting location to a target, Fitts’ law relates movement duration to the distance moved and target size. The widespread empirical success of the formula is suggestive of underlying principles of human movement control. There have been previous attempts to relate Fitts’ law to engineering-type control hypotheses and it has been shown that the law is exactly consistent with the closed-loop step-response of a time-delayed, first-order system. Assuming only the operation of closed-loop feedback, either continuous or intermittent, this paper asks whether such feedback should be predictive or not predictive to be consistent with Fitts law. Since Fitts’ law is equivalent to a time delay separated from a first-order system, known control theory implies that the controller must be predictive. A predictive controller moves the time-delay outside the feedback loop such that the closed-loop response can be separated into a time delay and rational function whereas a non- predictive controller retains a state delay within feedback loop which is not consistent with Fitts’ law. Using sufficient parameters, a high-order non-predictive controller could approximately reproduce Fitts’ law. However, such high-order, “non-parametric” controllers are essentially empirical in nature, without physical meaning, and therefore are conceptually inferior to the predictive controller. It is a new insight that using closed-loop feedback, prediction is required to physically explain Fitts’ law. The implication is that prediction is an inherent part of the “speed-accuracy trade-off”

Multiple Imputation Ensembles (MIE) for dealing with missing data

Missing data is a significant issue in many real-world datasets, yet there are no robust methods for dealing with it appropriately. In this paper, we propose a robust approach to dealing with missing data in classification problems: Multiple Imputation Ensembles (MIE). Our method integrates two approaches: multiple imputation and ensemble methods and compares two types of ensembles: bagging and stacking. We also propose a robust experimental set-up using 20 benchmark datasets from the UCI machine learning repository. For each dataset, we introduce increasing amounts of data Missing Completely at Random. Firstly, we use a number of single/multiple imputation methods to recover the missing values and then ensemble a number of different classifiers built on the imputed data. We assess the quality of the imputation by using dissimilarity measures. We also evaluate the MIE performance by comparing classification accuracy on the complete and imputed data. Furthermore, we use the accuracy of simple imputation as a benchmark for comparison. We find that our proposed approach combining multiple imputation with ensemble techniques outperform others, particularly as missing data increases

How Plastic Can Phenotypic Plasticity Be? The Branching Coral Stylophora pistillata as a Model System

Phenotypic plasticity enables multicellular organisms to adjust morphologies and various life history traits to variable environmental challenges. Here, we elucidate fixed and plastic architectural rules for colony astogeny in multiple types of colonial ramets, propagated by cutting from genets of the branching coral Stylophora pistillata from Eilat, the Red Sea. We examined 16 morphometric parameters on 136 one-year old S. pistillata colonies (of seven genotypes), originating from small fragments belonging, each, to one of three single-branch types (single tips, start-up, and advanced bifurcating tips) or to structural preparative manipulations (representing a single or two growth axes). Experiments were guided by the rationale that in colonial forms, complexity of evolving phenotypic plasticity can be associated with a degree of structural modularity, where shapes are approached by erecting iterative growth patterns at different levels of coral-colony organization. Analyses revealed plastic morphometric characters at branch level, and predetermined morphometric traits at colony level (only single trait exhibited plasticity under extreme manipulation state). Therefore, under the experimental manipulations of this study, phenotypic plasticity in S. pistillata appears to be related to branch level of organization, whereas colony traits are controlled by predetermined genetic architectural rules. Each level of organization undergoes its own mode of astogeny. However, depending on the original ramet structure, the spherical 3-D colonial architecture in this species is orchestrated and assembled by both developmental trajectories at the branch level, and traits at the colony level of organization. In nature, branching colonial forms are often subjected to harsh environmental conditions that cause fragmentation of colony into ramets of different sizes and structures. Developmental traits that are plastic, responding to fragment structure and are not predetermine in controlling astogeny, allow formation of species-specific architecture product through integrated but variable developmental routes. This adaptive plasticity or regeneration is an efficient mechanism by which isolated fragments of branching coral species cope with external environmental forces