131 research outputs found
Totally corrective boosting algorithm and application to face recognition
Boosting is one of the most well-known learning methods for building highly accurate classifiers or regressors from a set of weak classifiers. Much effort has been devoted to the understanding of boosting algorithms. However, questions remain unclear about the success of boosting.
In this thesis, we study boosting algorithms from a new perspective. We started our research by empirically comparing the LPBoost and AdaBoost algorithms. The result and the corresponding analysis show that, besides the minimum margin, which is directly and globally optimized in LPBoost, the margin distribution plays a more important role. Inspired by this observation, we theoretically prove that the Lagrange dual problems of AdaBoost, LogitBoost and soft-margin LPBoost with generalized hinge loss are all entropy maximization problems. By looking at the dual problems of these boosting algorithms, we show that the success of boosting algorithms can be understood in terms of maintaining a better margin distribution by maximizing margins and at the same time controlling the margin variance. We further point out that AdaBoost approximately maximizes the average margin, instead of the minimum margin. The duality formulation also enables us to develop column-generation based optimization algorithms, which are totally corrective. The new algorithm, which is termed AdaBoost-CG, exhibits almost identical classification results to those of standard stage-wise additive boosting algorithms, but with much faster convergence rates. Therefore, fewer weak classifiers are needed to build the ensemble using our proposed optimization technique.
The significance of margin distribution motivates us to design a new column-generation based algorithm that directly maximizes the average margin while minimizes the margin variance at the same time. We term this novel method MDBoost and show its superiority over other boosting-like algorithms. Moreover, consideration of the primal and dual problems together leads to important new insights into the characteristics of boosting algorithms. We then propose a general framework that can be used to design new boosting algorithms. A wide variety of machine learning problems essentially minimize a regularized risk functional. We show that the proposed boosting framework, termed AnyBoostTc, can accommodate various loss functions and different regularizers in a totally corrective optimization way. A large body of totally corrective boosting algorithms can actually be solved very efficiently, and no sophisticated convex optimization solvers are needed, by solving the primal rather than the dual. We also demonstrate that some boosting algorithms like AdaBoost can be interpreted in our framework, even their optimization is not totally corrective, .
We conclude our study by applying the totally corrective boosting algorithm to a long-standing computer vision problem-face recognition. Linear regression face recognizers, constrained by two categories of locality, are selected and combined within both the traditional and totally corrective boosting framework. To our knowledge, it is the first time that linear-representation classifiers are boosted for face recognition. The instance-based weak classifiers bring some advantages, which are theoretically or empirically proved in our work. Benefiting from the robust weak learner and the advanced learning framework, our algorithms achieve the best reported recognition rates on face recognition benchmark datasets
Robust and Real-time Deep Tracking Via Multi-Scale Domain Adaptation
Visual tracking is a fundamental problem in computer vision. Recently, some
deep-learning-based tracking algorithms have been achieving record-breaking
performances. However, due to the high complexity of deep learning, most deep
trackers suffer from low tracking speed, and thus are impractical in many
real-world applications. Some new deep trackers with smaller network structure
achieve high efficiency while at the cost of significant decrease on precision.
In this paper, we propose to transfer the feature for image classification to
the visual tracking domain via convolutional channel reductions. The channel
reduction could be simply viewed as an additional convolutional layer with the
specific task. It not only extracts useful information for object tracking but
also significantly increases the tracking speed. To better accommodate the
useful feature of the target in different scales, the adaptation filters are
designed with different sizes. The yielded visual tracker is real-time and also
illustrates the state-of-the-art accuracies in the experiment involving two
well-adopted benchmarks with more than 100 test videos.Comment: 6 page
On the Dual Formulation of Boosting Algorithms
We study boosting algorithms from a new perspective. We show that the
Lagrange dual problems of AdaBoost, LogitBoost and soft-margin LPBoost with
generalized hinge loss are all entropy maximization problems. By looking at the
dual problems of these boosting algorithms, we show that the success of
boosting algorithms can be understood in terms of maintaining a better margin
distribution by maximizing margins and at the same time controlling the margin
variance.We also theoretically prove that, approximately, AdaBoost maximizes
the average margin, instead of the minimum margin. The duality formulation also
enables us to develop column generation based optimization algorithms, which
are totally corrective. We show that they exhibit almost identical
classification results to that of standard stage-wise additive boosting
algorithms but with much faster convergence rates. Therefore fewer weak
classifiers are needed to build the ensemble using our proposed optimization
technique.Comment: 16 pages. To publish/Published in IEEE Transactions on Pattern
Analysis and Machine Intelligence, 201
Boosting through Optimization of Margin Distributions
Boosting has attracted much research attention in the past decade. The
success of boosting algorithms may be interpreted in terms of the margin
theory. Recently it has been shown that generalization error of classifiers can
be obtained by explicitly taking the margin distribution of the training data
into account. Most of the current boosting algorithms in practice usually
optimizes a convex loss function and do not make use of the margin
distribution. In this work we design a new boosting algorithm, termed
margin-distribution boosting (MDBoost), which directly maximizes the average
margin and minimizes the margin variance simultaneously. This way the margin
distribution is optimized. A totally-corrective optimization algorithm based on
column generation is proposed to implement MDBoost. Experiments on UCI datasets
show that MDBoost outperforms AdaBoost and LPBoost in most cases.Comment: 9 pages. To publish/Published in IEEE Transactions on Neural
Networks, 21(7), July 201
An Analysis of Urban Vacant Land on the Macau Peninsula
With the development and construction of the city, the urban development of the Macau Peninsula has entered an era of stock development against the background of the limited scale of urban land. With the shortage of land resources, the problem of unused land on the Macau Peninsula is coming to the fore. This paper mainly studies the problem of idle land in the Macau Peninsula, based on the urban development and particular historical background of the region, investigates and elaborates on its complex formation causes through the literature research method, and analyzes the spatial distribution characteristics of idle land in the current situation of the Macau Peninsula by using GIS technology. Based on the above research,suggestions are put forward to prevent and manage the urban vacant land problem in the future urban management and development of Macau
Target before Shooting: Accurate Anomaly Detection and Localization under One Millisecond via Cascade Patch Retrieval
In this work, by re-examining the "matching" nature of Anomaly Detection
(AD), we propose a new AD framework that simultaneously enjoys new records of
AD accuracy and dramatically high running speed. In this framework, the anomaly
detection problem is solved via a cascade patch retrieval procedure that
retrieves the nearest neighbors for each test image patch in a coarse-to-fine
fashion. Given a test sample, the top-K most similar training images are first
selected based on a robust histogram matching process. Secondly, the nearest
neighbor of each test patch is retrieved over the similar geometrical locations
on those "global nearest neighbors", by using a carefully trained local metric.
Finally, the anomaly score of each test image patch is calculated based on the
distance to its "local nearest neighbor" and the "non-background" probability.
The proposed method is termed "Cascade Patch Retrieval" (CPR) in this work.
Different from the conventional patch-matching-based AD algorithms, CPR selects
proper "targets" (reference images and locations) before "shooting"
(patch-matching). On the well-acknowledged MVTec AD, BTAD and MVTec-3D AD
datasets, the proposed algorithm consistently outperforms all the comparing
SOTA methods by remarkable margins, measured by various AD metrics.
Furthermore, CPR is extremely efficient. It runs at the speed of 113 FPS with
the standard setting while its simplified version only requires less than 1 ms
to process an image at the cost of a trivial accuracy drop. The code of CPR is
available at https://github.com/flyinghu123/CPR.Comment: 13 pages,8 figure
Efficient Anomaly Detection with Budget Annotation Using Semi-Supervised Residual Transformer
Anomaly Detection is challenging as usually only the normal samples are seen
during training and the detector needs to discover anomalies on-the-fly. The
recently proposed deep-learning-based approaches could somehow alleviate the
problem but there is still a long way to go in obtaining an industrial-class
anomaly detector for real-world applications. On the other hand, in some
particular AD tasks, a few anomalous samples are labeled manually for achieving
higher accuracy. However, this performance gain is at the cost of considerable
annotation efforts, which can be intractable in many practical scenarios.
In this work, the above two problems are addressed in a unified framework.
Firstly, inspired by the success of the patch-matching-based AD algorithms, we
train a sliding vision transformer over the residuals generated by a novel
position-constrained patch-matching. Secondly, the conventional pixel-wise
segmentation problem is cast into a block-wise classification problem. Thus the
sliding transformer can attain even higher accuracy with much less annotation
labor. Thirdly, to further reduce the labeling cost, we propose to label the
anomalous regions using only bounding boxes. The unlabeled regions caused by
the weak labels are effectively exploited using a highly-customized
semi-supervised learning scheme equipped with two novel data augmentation
methods. The proposed method outperforms all the state-of-the-art approaches
using all the evaluation metrics in both the unsupervised and supervised
scenarios. On the popular MVTec-AD dataset, our SemiREST algorithm obtains the
Average Precision (AP) of 81.2% in the unsupervised condition and 84.4% AP for
supervised anomaly detection. Surprisingly, with the bounding-box-based
semi-supervisions, SemiREST still outperforms the SOTA methods with full
supervision (83.8% AP) on MVTec-AD.Comment: 20 pages,6 figure
- …