Towards Practicality of Sketch-Based Visual Understanding

Sketches have been used to conceptualise and depict visual objects from pre-historic times. Sketch research has flourished in the past decade, particularly with the proliferation of touchscreen devices. Much of the utilisation of sketch has been anchored around the fact that it can be used to delineate visual concepts universally irrespective of age, race, language, or demography. The fine-grained interactive nature of sketches facilitates the application of sketches to various visual understanding tasks, like image retrieval, image-generation or editing, segmentation, 3D-shape modelling etc. However, sketches are highly abstract and subjective based on the perception of individuals. Although most agree that sketches provide fine-grained control to the user to depict a visual object, many consider sketching a tedious process due to their limited sketching skills compared to other query/support modalities like text/tags. Furthermore, collecting fine-grained sketch-photo association is a significant bottleneck to commercialising sketch applications. Therefore, this thesis aims to progress sketch-based visual understanding towards more practicality.Comment: PhD thesis successfully defended by Ayan Kumar Bhunia, Supervisor: Prof. Yi-Zhe Song, Thesis Examiners: Prof Stella Yu and Prof Adrian Hilto

Learning from Very Few Samples: A Survey

Few sample learning (FSL) is significant and challenging in the field of machine learning. The capability of learning and generalizing from very few samples successfully is a noticeable demarcation separating artificial intelligence and human intelligence since humans can readily establish their cognition to novelty from just a single or a handful of examples whereas machine learning algorithms typically entail hundreds or thousands of supervised samples to guarantee generalization ability. Despite the long history dated back to the early 2000s and the widespread attention in recent years with booming deep learning technologies, little surveys or reviews for FSL are available until now. In this context, we extensively review 300+ papers of FSL spanning from the 2000s to 2019 and provide a timely and comprehensive survey for FSL. In this survey, we review the evolution history as well as the current progress on FSL, categorize FSL approaches into the generative model based and discriminative model based kinds in principle, and emphasize particularly on the meta learning based FSL approaches. We also summarize several recently emerging extensional topics of FSL and review the latest advances on these topics. Furthermore, we highlight the important FSL applications covering many research hotspots in computer vision, natural language processing, audio and speech, reinforcement learning and robotic, data analysis, etc. Finally, we conclude the survey with a discussion on promising trends in the hope of providing guidance and insights to follow-up researches.Comment: 30 page

Rethinking Pen Input Interaction: Enabling Freehand Sketching Through Improved Primitive Recognition

Online sketch recognition uses machine learning and artificial intelligence techniques to interpret markings made by users via an electronic stylus or pen. The goal of sketch recognition is to understand the intention and meaning of a particular user's drawing. Diagramming applications have been the primary beneficiaries of sketch recognition technology, as it is commonplace for the users of these tools to rst create a rough sketch of a diagram on paper before translating it into a machine understandable model, using computer-aided design tools, which can then be used to perform simulations or other meaningful tasks. Traditional methods for performing sketch recognition can be broken down into three distinct categories: appearance-based, gesture-based, and geometric-based. Although each approach has its advantages and disadvantages, geometric-based methods have proven to be the most generalizable for multi-domain recognition. Tools, such as the LADDER symbol description language, have shown to be capable of recognizing sketches from over 30 different domains using generalizable, geometric techniques. The LADDER system is limited, however, in the fact that it uses a low-level recognizer that supports only a few primitive shapes, the building blocks for describing higher-level symbols. Systems which support a larger number of primitive shapes have been shown to have questionable accuracies as the number of primitives increase, or they place constraints on how users must input shapes (e.g. circles can only be drawn in a clockwise motion; rectangles must be drawn starting at the top-left corner). This dissertation allows for a significant growth in the possibility of free-sketch recognition systems, those which place little to no drawing constraints on users. In this dissertation, we describe multiple techniques to recognize upwards of 18 primitive shapes while maintaining high accuracy. We also provide methods for producing confidence values and generating multiple interpretations, and explore the difficulties of recognizing multi-stroke primitives. In addition, we show the need for a standardized data repository for sketch recognition algorithm testing and propose SOUSA (sketch-based online user study application), our online system for performing and sharing user study sketch data. Finally, we will show how the principles we have learned through our work extend to other domains, including activity recognition using trained hand posture cues

Verificaciónn de firma y gráficos manuscritos: Características discriminantes y nuevos escenarios de aplicación biométrica

Tesis doctoral inédita leída en la Escuela Politécnica Superior, Departamento de Tecnología Electrónica y de las Comunicaciones. Fecha de lectura: Febrero 2015The proliferation of handheld devices such as smartphones and tablets brings a new scenario for biometric authentication, and in particular to automatic signature verification. Research on signature verification has been traditionally carried out using signatures acquired on digitizing tablets or Tablet-PCs. This PhD Thesis addresses the problem of user authentication on handled devices using handwritten signatures and graphical passwords based on free-form doodles, as well as the effects of biometric aging on signatures. The Thesis pretends to analyze: (i) which are the effects of mobile conditions on signature and doodle verification, (ii) which are the most distinctive features in mobile conditions, extracted from the pen or fingertip trajectory, (iii) how do different similarity computation (i.e. matching) algorithms behave with signatures and graphical passwords captured on mobile conditions, and (iv) what is the impact of aging on signature features and verification performance. Two novel datasets have been presented in this Thesis. A database containing free-form graphical passwords drawn with the fingertip on a smartphone is described. It is the first publicly available graphical password database to the extent of our knowledge. A dataset containing signatures from users captured over a period 15 months is also presented, aimed towards the study of biometric aging. State-of-the-art local and global matching algorithms are used, namely Hidden Markov Models, Gaussian Mixture Models, Dynamic Time Warping and distance-based classifiers. A large proportion of features presented in the research literature is considered in this Thesis. The experimental contribution of this Thesis is divided in three main topics: signature verification on handheld devices, the effects of aging on signature verification, and free-form graphical password-based authentication. First, regarding signature verification in mobile conditions, we use a database captured both on a handheld device and digitizing tablet in an office-like scenario. We analyze the discriminative power of both global and local features using discriminant analysis and feature selection techniques. The effects of the lack of pen-up trajectories on handheld devices (when the stylus tip is not in contact with the screen) are also studied. We then analyze the effects of biometric aging on the signature trait. Using three different matching algorithms, Hidden Markov Models (HMM), Dynamic Time Warping (DTW), and distance-based classifiers, the impact in verification performance is studied. We also study the effects of aging on individual users and individual signature features. Template update techniques are analyzed as a way of mitigating the negative impact of aging. Regarding graphical passwords, the DooDB graphical password database is first presented. A statistical analysis is performed comparing the database samples (free-form doodles and simplified signatures) with handwritten signatures. The sample variability (inter-user, intra-user and inter-session) is also analyzed, as well as the learning curve for each kind of trait. Benchmark results are also reported using state of the art classifiers. Graphical password verification is afterwards studied using features and matching algorithms from the signature verification state of the art. Feature selection is also performed and the resulting feature sets are analyzed. The main contributions of this work can be summarized as follows. A thorough analysis of individual feature performance has been carried out, both for global and local features and on signatures acquired using pen tablets and handheld devices. We have found which individual features are the most robust and which have very low discriminative potential (pen inclination and pressure among others). It has been found that feature selection increases verification performance dramatically, from example from ERRs (Equal Error Rates) over 30% using all available local features, in the case of handheld devices and skilled forgeries, to rates below 20% after feature selection. We study the impact of the lack of trajectory information when the pen tip is not in contact with the acquisition device surface (which happens when touchscreens are used for signature acquisitions), and we have found that the lack of pen-up trajectories negatively affects verification performance. As an example, the EER for the local system increases from 9.3% to 12.1% against skilled forgeries when pen-up trajectories are not available. We study the effects of biometric aging on signature verification and study a number of ways to compensate the observed performance degradation. It is found that aging does not affect equally all the users in the database and that features related to signature dynamics are more degraded than static features. Comparing the performance using test signatures from the first months with the last months, a variable effect of aging on the EER against random forgeries is observed in the three systems that are evaluated, from 0.0% to 0.5% in the DTW system, from 1.0% to 5.0% in the distance-based system using global features, and from 3.2% to 27.8% in the HMM system. A new graphical password database has been acquired and made publicly available. Verification algorithms for finger-drawn graphical passwords and simplified signatures are compared and feature analysis is performed. We have found that inter-session variability has a highly negative impact on verification performance, but this can be mitigated performing feature selection and applying fusion of different matchers. It has also been found that some feature types are prevalent in the optimal feature vectors and that classifiers have a very different behavior against skilled and random forgeries. An EER of 3.4% and 22.1% against random and skilled forgeries is obtained for free-form doodles, which is a promising performance

Using contour information and segmentation for object registration, modeling and retrieval

This thesis considers different aspects of the utilization of contour information and syntactic and semantic image segmentation for object registration, modeling and retrieval in the context of content-based indexing and retrieval in large collections of images. Target applications include retrieval in collections of closed silhouettes, holistic w ord recognition in handwritten historical manuscripts and shape registration. Also, the thesis explores the feasibility of contour-based syntactic features for improving the correspondence of the output of bottom-up segmentation to semantic objects present in the scene and discusses the feasibility of different strategies for image analysis utilizing contour information, e.g. segmentation driven by visual features versus segmentation driven by shape models or semi-automatic in selected application scenarios. There are three contributions in this thesis. The first contribution considers structure analysis based on the shape and spatial configuration of image regions (socalled syntactic visual features) and their utilization for automatic image segmentation. The second contribution is the study of novel shape features, matching algorithms and similarity measures. Various applications of the proposed solutions are presented throughout the thesis providing the basis for the third contribution which is a discussion of the feasibility of different recognition strategies utilizing contour information. In each case, the performance and generality of the proposed approach has been analyzed based on extensive rigorous experimentation using as large as possible test collections

Václav Philomathes’ Musicorum Libri Quattuor (1512): Translation, Commentary, and Contextualization

The Czech-born music theorist, Václav Philomathes, wrote the Musicorum libri quattuor in 1512 while attending the University of Vienna. This didactic treatise became one of the most widely published theory treatise of its time with 26 copies of five editions remaining today and covers the topics of Gregorian chant practice, Solmization, Mensural Notation, Choir Practice and Conducting, and Four-voice Counterpoint. Of particular note, is the section on choir practice and conducting, of which there is no equivalent prior example extant today. This dissertation provides a Latin-English translation of Philomathes’s work, as well as produces a critical commentary and comparison of the five editions while positioning the editions within the context of the musico-theoretical background of early-to-mid-16th century scholarship in Central Europe

The Rise of iWar: Identity, Information, and the Individualization of Modern Warfare

During a decade of global counterterrorism operations and two extended counterinsurgency campaigns, the United States was confronted with a new kind of adversary. Without uniforms, flags, and formations, the task of identifying and targeting these combatants represented an unprecedented operational challenge for which Cold War era doctrinal methods were largely unsuited. This monograph examines the doctrinal, technical, and bureaucratic innovations that evolved in response to these new operational challenges. It discusses the transition from a conventionally focused, Cold War-era targeting process to one optimized for combating networks and conducting identity-based targeting. It analyzes the policy decisions and strategic choices that were the catalysts of this change and concludes with an in depth examination of emerging technologies that are likely to shape how this mode of warfare will be waged in the future.https://press.armywarcollege.edu/monographs/1436/thumbnail.jp