Clothing Co-Parsing by Joint Image Segmentation and Labeling

This paper aims at developing an integrated system of clothing co-parsing, in order to jointly parse a set of clothing images (unsegmented but annotated with tags) into semantic configurations. We propose a data-driven framework consisting of two phases of inference. The first phase, referred as "image co-segmentation", iterates to extract consistent regions on images and jointly refines the regions over all images by employing the exemplar-SVM (E-SVM) technique [23]. In the second phase (i.e. "region co-labeling"), we construct a multi-image graphical model by taking the segmented regions as vertices, and incorporate several contexts of clothing configuration (e.g., item location and mutual interactions). The joint label assignment can be solved using the efficient Graph Cuts algorithm. In addition to evaluate our framework on the Fashionista dataset [30], we construct a dataset called CCP consisting of 2098 high-resolution street fashion photos to demonstrate the performance of our system. We achieve 90.29% / 88.23% segmentation accuracy and 65.52% / 63.89% recognition rate on the Fashionista and the CCP datasets, respectively, which are superior compared with state-of-the-art methods.Comment: 8 pages, 5 figures, CVPR 201

Field sketching and the interpretation of landscape : exploring the benefits of fieldwork and drawing in contemporary landscape practice

This thesis explores potential roles for field sketching in, landscape observation and assessment, landscape planning and design, landscape representation, and in addressing the experiential dimension of the landscape.The research seeks to define and legitimise the old technique of field sketching, and the use and development of field sketches by students and practitioners of landscape architecture, and other landscape disciplines. The wider values of, fieldwork, hand -generated field notations, drawing as an interactive dialogue with others, and the sketch as a type of landscape representation, are also recognised.Whilst accurate representation and precise geometrical definition of the landscape can now be achieved quickly with photographs and by semi - automated digital means, interpretation requires careful observation. Sketching involves an observer stopping and looking and interpreting slowly and carefully. Field sketching and the uses of the field sketch are proposed as bringing an effectiveness to landscape work, valuable because of the interpretation it involves, and the time it does take: timeless because of its simplicity.A personal way of working is investigated, based on a Grounded Theory approach. Systematic analysis of case studies is made through reflection-on-practice. Practice observations (data) are collated and interpreted by practical sorting tasks, to propose a series of how to do and why important principles regarding field sketching. External support for the research findings is sought from literature, considering the broad themes of: fieldwork and the experience of landscapes, field sketching and drawing as craft and expression, and developing and using field sketches.Applications for field sketching to meet contemporary needs in landscape architecture are proposed: the sketch as a designer's tool, sketch-based visualisations as interpretive images, and field sketching as a participative technique that can be used to engage the inquirer, collaborators, and the public with landscape experience -grounded decisions

Scones: Towards Conversational Authoring of Sketches

Iteratively refining and critiquing sketches are crucial steps to developing effective designs. We introduce Scones, a mixed-initiative, machine-learning-driven system that enables users to iteratively author sketches from text instructions. Scones is a novel deep-learning-based system that iteratively generates scenes of sketched objects composed with semantic specifications from natural language. Scones exceeds state-of-the-art performance on a text-based scene modification task, and introduces a mask-conditioned sketching model that can generate sketches with poses specified by high-level scene information. In an exploratory user evaluation of Scones, participants reported enjoying an iterative drawing task with Scones, and suggested additional features for further applications. We believe Scones is an early step towards automated, intelligent systems that support human-in-the-loop applications for communicating ideas through sketching in art and design.Comment: Long Paper, IUI '20: Proceedings of the 25th International Conference on Intelligent User Interface

Towards Practicality of Sketch-Based Visual Understanding

Sketches have been used to conceptualise and depict visual objects from pre-historic times. Sketch research has flourished in the past decade, particularly with the proliferation of touchscreen devices. Much of the utilisation of sketch has been anchored around the fact that it can be used to delineate visual concepts universally irrespective of age, race, language, or demography. The fine-grained interactive nature of sketches facilitates the application of sketches to various visual understanding tasks, like image retrieval, image-generation or editing, segmentation, 3D-shape modelling etc. However, sketches are highly abstract and subjective based on the perception of individuals. Although most agree that sketches provide fine-grained control to the user to depict a visual object, many consider sketching a tedious process due to their limited sketching skills compared to other query/support modalities like text/tags. Furthermore, collecting fine-grained sketch-photo association is a significant bottleneck to commercialising sketch applications. Therefore, this thesis aims to progress sketch-based visual understanding towards more practicality.Comment: PhD thesis successfully defended by Ayan Kumar Bhunia, Supervisor: Prof. Yi-Zhe Song, Thesis Examiners: Prof Stella Yu and Prof Adrian Hilto

Free-hand sketch synthesis with deformable stroke models

We present a generative model which can automatically summarize the stroke composition of free-hand sketches of a given category. When our model is fit to a collection of sketches with similar poses, it discovers and learns the structure and appearance of a set of coherent parts, with each part represented by a group of strokes. It represents both consistent (topology) as well as diverse aspects (structure and appearance variations) of each sketch category. Key to the success of our model are important insights learned from a comprehensive study performed on human stroke data. By fitting this model to images, we are able to synthesize visually similar and pleasant free-hand sketches

Cognitive Effectiveness of Visual Instructional Design Languages

The introduction of learning technologies into education is making the design of courses and instructional materials an increasingly complex task. Instructional design languages are identified as conceptual tools for achieving more standardized and, at the same time, more creative design solutions, as well as enhancing communication and transparency in the design process. In this article we discuss differences in cognitive aspects of three visual instructional design languages (E²ML, PoEML, coUML), based on user evaluation. Cognitive aspects are of relevance for learning a design language, creating models with it, and understanding models created using it. The findings should enable language constructors to improve the usability of visual instructional design languages in the future. The paper concludes with directions with regard to how future research on visual instructional design languages could strengthen their value and enhance their actual use by educators and designers by synthesizing existing efforts into a unified modeling approach for VIDLs