43 research outputs found
Screening for a Reweighted Penalized Conditional Gradient Method
The conditional gradient method (CGM) is widely used in large-scale sparse
convex optimization, having a low per iteration computational cost for
structured sparse regularizers and a greedy approach to collecting nonzeros. We
explore the sparsity acquiring properties of a general penalized CGM (P-CGM)
for convex regularizers and a reweighted penalized CGM (RP-CGM) for nonconvex
regularizers, replacing the usual convex constraints with gauge-inspired
penalties. This generalization does not increase the per-iteration complexity
noticeably. Without assuming bounded iterates or using line search, we show
convergence of the gap of each subproblem, which measures distance to
a stationary point. We couple this with a screening rule which is safe in the
convex case, converging to the true support at a rate where
measures how close the problem is to degeneracy. In the
nonconvex case the screening rule converges to the true support in a finite
number of iterations, but is not necessarily safe in the intermediate iterates.
In our experiments, we verify the consistency of the method and adjust the
aggressiveness of the screening rule by tuning the concavity of the
regularizer
Recommended from our members
Endoluminal surface registration for CT colonography using Haustral Fold Matching
Computed Tomographic (CT) colonography is a technique used for the detection of bowel cancer or potentially precancerous polyps. The procedure is performed routinely with the patient both prone and supine to differentiate fixed colonic pathology from mobile faecal residue. Matching corresponding locations is difficult and time consuming for radiologists due to colonic deformations that occur during patient repositioning.
We propose a novel method to establish correspondence between the two acquisitions automatically. The problem is first simplified by detecting haustral folds using a graph cut method applied to a curvature-based metric applied to a surface mesh generated from segmentation of the colonic lumen. A virtual camera is used to create a set of images that provide a metric for matching pairs of folds between the prone and supine acquisitions. Image patches are generated at the fold positions using depth map renderings of the endoluminal surface and optimised by performing a virtual camera registration over a restricted set of degrees of freedom. The intensity difference between image pairs, along with additional neighbourhood information to enforce geometric constraints over a 2D parameterisation of the 3D space, are used as unary and pair-wise costs respectively, and included in a Markov Random Field (MRF) model to estimate the maximum a-posteriori fold labelling assignment.
The method achieved fold matching accuracy of 96.0% and 96.1% in patient cases with and without local colonic collapse. Moreover, it improved upon an existing surface-based registration algorithm by providing an initialisation. The set of landmark correspondences is used to non-rigidly transform a 2D source image derived from a conformal mapping process on the 3D endoluminal surface mesh. This achieves full surface correspondence between prone and supine views and can be further refined with an intensity based registration showing a statistically significant improvement (p<0.001p<0.001), and decreasing mean error from 11.9mm11.9mm to 6.0mm6.0mm measured at 1743 reference points from 17 CTC datasets
SCALABLE ALGORITHMS FOR HIGH DIMENSIONAL STRUCTURED DATA
Emerging technologies and digital devices provide us with increasingly large volume
of data with respect to both the sample size and the number of features. To explore the benefits of massive data sets, scalable statistical models and machine learning algorithms are more and more important in different research disciplines. For robust and accurate prediction, prior knowledge regarding dependency structures within data needs to be formulated appropriately in these models. On the other hand, scalability and computation complexity of existing algorithms may not meet the needs to analyze massive high-dimensional data. This dissertation presents several novel methods to scale up sparse learning models to analyze massive data sets. We first present our novel safe active incremental feature (SAIF) selection algorithm for LASSO (least absolute shrinkage and selection operator), with the time complexity analysis to show the advantages over state of the art existing methods. As SAIF is targeting general convex loss functions, it potentially can be extended to many learning models and big-data applications, and we show how support vector machines (SVM) can be scaled up based on the idea of SAIF. Secondly, we propose screening methods to generalized LASSO (GL), which specifically considers the dependency structure among features. We also propose a scalable feature selection method for non-parametric, non-linear models based on sparse structures and kernel methods. Theoretical analysis and
experimental results in this dissertation show that model complexity can be significantly
reduced with the sparsity and structure assumptions
Graph Connectivity and Single Element Recovery via Linear and OR Queries
We study the problem of finding a spanning forest in an undirected,
-vertex multi-graph under two basic query models. One is the Linear query
model which are linear measurements on the incidence vector induced by the
edges; the other is the weaker OR query model which only reveals whether a
given subset of plausible edges is empty or not. At the heart of our study lies
a fundamental problem which we call the {\em single element recovery} problem:
given a non-negative real vector in dimension, return a single element
from the support. Queries can be made in rounds, and our goals is to
understand the trade-offs between the query complexity and the rounds of
adaptivity needed to solve these problems, for both deterministic and
randomized algorithms. These questions have connections and ramifications to
multiple areas such as sketching, streaming, graph reconstruction, and
compressed sensing. Our main results are:
* For the single element recovery problem, it is easy to obtain a
deterministic, -round algorithm which makes -queries per-round.
We prove that this is tight: any -round deterministic algorithm must make
linear queries in some round. In contrast, a -round
-query randomized algorithm which succeeds 99% of the time is
known to exist.
* We design a deterministic -round, -OR query
algorithm for graph connectivity. We complement this with an
-lower bound for any -round deterministic
algorithm in the OR-model.
* We design a randomized, -round algorithm for the graph connectivity
problem which makes -OR queries. In contrast, we prove that any
-round algorithm (possibly randomized) requires -OR
queries
The Geometry of Uniqueness, Sparsity and Clustering in Penalized Estimation
We provide a necessary and sufficient condition for the uniqueness of
penalized least-squares estimators whose penalty term is given by a norm with a
polytope unit ball, covering a wide range of methods including SLOPE and LASSO,
as well as the related method of basis pursuit. We consider a strong type of
uniqueness that is relevant for statistical problems. The uniqueness condition
is geometric and involves how the row span of the design matrix intersects the
faces of the dual norm unit ball, which for SLOPE is given by the sign
permutahedron. Further considerations based this condition also allow to derive
results on sparsity and clustering features. In particular, we define the
notion of a SLOPE model to describe both sparsity and clustering properties of
this method and also provide a geometric characterization of accessible SLOPE
models.Comment: new title, minor change
Novel image descriptors and learning methods for image classification applications
Image classification is an active and rapidly expanding research area in computer vision and machine learning due to its broad applications. With the advent of big data, the need for robust image descriptors and learning methods to process a large number of images for different kinds of visual applications has greatly increased. Towards that end, this dissertation focuses on exploring new image descriptors and learning methods by incorporating important visual aspects and enhancing the feature representation in the discriminative space for advancing image classification.
First, an innovative sparse representation model using the complete marginal Fisher analysis (CMFA-SR) framework is proposed for improving the image classification performance. In particular, the complete marginal Fisher analysis method extracts the discriminatory features in both the column space of the local samples based within class scatter matrix and the null space of its transformed matrix. To further improve the classification capability, a discriminative sparse representation model is proposed by integrating a representation criterion such as the sparse representation and a discriminative criterion. Second, the discriminative dictionary distribution based sparse coding (DDSC) method is presented that utilizes both the discriminative and generative information to enhance the feature representation. Specifically, the dictionary distribution criterion reveals the class conditional probability of each dictionary item by using the dictionary distribution coefficients, and the discriminative criterion applies new within-class and between-class scatter matrices for discriminant analysis. Third, a fused color Fisher vector (FCFV) feature is developed by integrating the most expressive features of the DAISY Fisher vector (D-FV) feature, the WLD-SIFT Fisher vector (WS-FV) feature, and the SIFT-FV feature in different color spaces to capture the local, color, spatial, relative intensity, as well as the gradient orientation information. Furthermore, a sparse kernel manifold learner (SKML) method is applied to the FCFV features for learning a discriminative sparse representation by considering the local manifold structure and the label information based on the marginal Fisher criterion. Finally, a novel multiple anthropological Fisher kernel framework (M-AFK) is presented to extract and enhance the facial genetic features for kinship verification. The proposed method is derived by applying a novel similarity enhancement approach based on SIFT flow and learning an inheritable transformation on the multiple Fisher vector features that uses the criterion of minimizing the distance among the kinship samples and maximizing the distance among the non-kinship samples.
The effectiveness of the proposed methods is assessed on numerous image classification tasks, such as face recognition, kinship verification, scene classification, object classification, and computational fine art painting categorization. The experimental results on popular image datasets show the feasibility of the proposed methods
LIPIcs, Volume 248, ISAAC 2022, Complete Volume
LIPIcs, Volume 248, ISAAC 2022, Complete Volum