We consider forward-backward greedy algorithms for solving sparse feature
selection problems with general convex smooth functions. A state-of-the-art
greedy method, the Forward-Backward greedy algorithm (FoBa-obj) requires to
solve a large number of optimization problems, thus it is not scalable for
large-size problems. The FoBa-gdt algorithm, which uses the gradient
information for feature selection at each forward iteration, significantly
improves the efficiency of FoBa-obj. In this paper, we systematically analyze
the theoretical properties of both forward-backward greedy algorithms. Our main
contributions are: 1) We derive better theoretical bounds than existing
analyses regarding FoBa-obj for general smooth convex functions; 2) We show
that FoBa-gdt achieves the same theoretical performance as FoBa-obj under the
same condition: restricted strong convexity condition. Our new bounds are
consistent with the bounds of a special case (least squares) and fills a
previously existing theoretical gap for general convex smooth functions; 3) We
show that the restricted strong convexity condition is satisfied if the number
of independent samples is more than kˉlogd where kˉ is the
sparsity number and d is the dimension of the variable; 4) We apply FoBa-gdt
(with the conditional random field objective) to the sensor selection problem
for human indoor activity recognition and our results show that FoBa-gdt
outperforms other methods (including the ones based on forward greedy selection
and L1-regularization)