Forward-Backward Greedy Algorithms for General Convex Smooth Functions over A Cardinality Constraint

We consider forward-backward greedy algorithms for solving sparse feature selection problems with general convex smooth functions. A state-of-the-art greedy method, the Forward-Backward greedy algorithm (FoBa-obj) requires to solve a large number of optimization problems, thus it is not scalable for large-size problems. The FoBa-gdt algorithm, which uses the gradient information for feature selection at each forward iteration, significantly improves the efficiency of FoBa-obj. In this paper, we systematically analyze the theoretical properties of both forward-backward greedy algorithms. Our main contributions are: 1) We derive better theoretical bounds than existing analyses regarding FoBa-obj for general smooth convex functions; 2) We show that FoBa-gdt achieves the same theoretical performance as FoBa-obj under the same condition: restricted strong convexity condition. Our new bounds are consistent with the bounds of a special case (least squares) and fills a previously existing theoretical gap for general convex smooth functions; 3) We show that the restricted strong convexity condition is satisfied if the number of independent samples is more than $\bar{k}\log d$ where $\bar{k}$ is the sparsity number and $d$ is the dimension of the variable; 4) We apply FoBa-gdt (with the conditional random field objective) to the sensor selection problem for human indoor activity recognition and our results show that FoBa-gdt outperforms other methods (including the ones based on forward greedy selection and L1-regularization)

An approach to spacecraft anomaly detection problem using kernel feature space

Development of advanced anomaly detection and failure diagnosis technologies for spacecraft is a quite significant issue in the space industry, because the space environment is harsh, distant and uncertain. While several modern approaches based on qualitative reasoning, expert systems, and probabilistic reasoning have been developed recently for this purpose, any of them has a common difficulty in obtaining accurate and complete a priori knowledge on the space systems from human experts. A reasonable alternative to this conventional anomaly detection method is to reuse a vast amount of telemetry data which is multi-dimensional time-series continuously produced from a number of system components in the spacecraft. This paper proposes a novel ”knowledge-free ” anomaly detection method for spacecraft based on Kernel Feature Space and directional distribution, which constructs a system behavior model from the past normal telemetry data from a set of telemetry data in normal operation and monitors the current system status by checking incoming data with the model. In this method, we regard anomaly phenomena as unexpected changes of causal associations in the spacecraft system, and hypothesize that the significant causal associations inside the system will appear in the form of principal component directions in a high-dimensional non-linear feature space which is constructed by a kernel function and a set of data. We have confirmed the effectiveness of the proposed anomaly detection method by applying it to the telemetry data obtained from a simulator of an orbital transfer vehicle designed to make a rendezvous maneuver with the Internationa

Large-Scale Price Optimization via Network Flow

Abstract This paper deals with price optimization, which is to find the best pricing strategy that maximizes revenue or profit, on the basis of demand forecasting models. Though recent advances in regression technologies have made it possible to reveal price-demand relationship of a large number of products, most existing price optimization methods, such as mixed integer programming formulation, cannot handle tens or hundreds of products because of their high computational costs. To cope with this problem, this paper proposes a novel approach based on network flow algorithms. We reveal a connection between supermodularity of the revenue and cross elasticity of demand. On the basis of this connection, we propose an efficient algorithm that employs network flow algorithms. The proposed algorithm can handle hundreds or thousands of products, and returns an exact optimal solution under an assumption regarding cross elasticity of demand. Even if the assumption does not hold, the proposed algorithm can efficiently find approximate solutions as good as other state-of-the-art methods, as empirical results show