Benchmarking non-photorealistic rendering of portraits

We present a set of images for helping NPR practitioners evaluate their image-based portrait stylisation algorithms. Using a standard set both facilitates comparisons with other methods and helps ensure that presented results are representative. We give two levels of difficulty, each consisting of 20 images selected systematically so as to provide good coverage of several possible portrait characteristics. We applied three existing portrait-specific stylisation algorithms, two general-purpose stylisation algorithms, and one general learning based stylisation algorithm to the first level of the benchmark, corresponding to the type of constrained images that have often been used in portrait-specific work. We found that the existing methods are generally effective on this new image set, demonstrating that level one of the benchmark is tractable; challenges remain at level two. Results revealed several advantages conferred by portrait-specific algorithms over general-purpose algorithms: portrait-specific algorithms can use domain-specific information to preserve key details such as eyes and to eliminate extraneous details, and they have more scope for semantically meaningful abstraction due to the underlying face model. Finally, we provide some thoughts on systematically extending the benchmark to higher levels of difficulty

Looking Beyond Appearances: Synthetic Training Data for Deep CNNs in Re-identification

Re-identification is generally carried out by encoding the appearance of a subject in terms of outfit, suggesting scenarios where people do not change their attire. In this paper we overcome this restriction, by proposing a framework based on a deep convolutional neural network, SOMAnet, that additionally models other discriminative aspects, namely, structural attributes of the human figure (e.g. height, obesity, gender). Our method is unique in many respects. First, SOMAnet is based on the Inception architecture, departing from the usual siamese framework. This spares expensive data preparation (pairing images across cameras) and allows the understanding of what the network learned. Second, and most notably, the training data consists of a synthetic 100K instance dataset, SOMAset, created by photorealistic human body generation software. Synthetic data represents a good compromise between realistic imagery, usually not required in re-identification since surveillance cameras capture low-resolution silhouettes, and complete control of the samples, which is useful in order to customize the data w.r.t. the surveillance scenario at-hand, e.g. ethnicity. SOMAnet, trained on SOMAset and fine-tuned on recent re-identification benchmarks, outperforms all competitors, matching subjects even with different apparel. The combination of synthetic data with Inception architectures opens up new research avenues in re-identification.Comment: 14 page

AlteredAvatar: Stylizing Dynamic 3D Avatars with Fast Style Adaptation

This paper presents a method that can quickly adapt dynamic 3D avatars to arbitrary text descriptions of novel styles. Among existing approaches for avatar stylization, direct optimization methods can produce excellent results for arbitrary styles but they are unpleasantly slow. Furthermore, they require redoing the optimization process from scratch for every new input. Fast approximation methods using feed-forward networks trained on a large dataset of style images can generate results for new inputs quickly, but tend not to generalize well to novel styles and fall short in quality. We therefore investigate a new approach, AlteredAvatar, that combines those two approaches using the meta-learning framework. In the inner loop, the model learns to optimize to match a single target style well; while in the outer loop, the model learns to stylize efficiently across many styles. After training, AlteredAvatar learns an initialization that can quickly adapt within a small number of update steps to a novel style, which can be given using texts, a reference image, or a combination of both. We show that AlteredAvatar can achieve a good balance between speed, flexibility and quality, while maintaining consistency across a wide range of novel views and facial expressions.Comment: 10 main pages, 14 figures. Project page: https://alteredavatar.github.i

Beyond factual to formulated silhouettes

When sketching terrain, a view-dependent framework of silhouette-related cues is required. This framework is prominent in manual sketches and is especially important in small-scale depictions viewed obliquely from above. Occluding contours, namely the lines delineating depth discontinuities in the projected surface, are insufficient for forming this framework. The role which the occluding contour, or Factual Silhouette, plays in structuring the sketch becomes increasingly minimal as more of the terrain becomes visible, as the viewpoint is raised.The aim of this research is to extend the set of occluding contours to encompass situations that are perceived as causing an occlusion and would therefore be sketched in a similar manner. These locations, termed Formulated Silhouettes supplement the set of occluding contours and provide a successful structuring framework. The proposed method processes visible areas of terrain, which are turning away from view, to extract a classified, vector-based description for a given view of a Digital Elevation Model. Background approaches to silhouette rendering are reviewed and the specific contributions of this thesis are discussed.The method is tested using case studies composed of terrain of varying scale and character and two application studies demonstrate how silhouettes can be used to enhance existing terrain visualization techniques, both abstract and realistic. In addition, consultation with cartographic designers provides external verification of the research. The thesis concludes by noting how silhouette contours relate to perceived entities rather than actual occlusions

FREEHAND DIGITAL DRAWING: A BOOST TO CREATIVE DESIGN. THE OBSERVER’S EYE AND THE DRAFTSMAN’S BRAIN

Representation of an architectural project aims at several intents; one of the most relevant being the visualization of a design. The subject of this dissertation is therefore the relationship between the draftsman, the creativity of his brain, the represented design, and the mind of the observer, while raising the question whether the current habit of an ever increasing digitalisation may be detrimental or advantageous to such relationship. After an analysis of architectural representation in history since Renaissance times, as reported in Part 1, the current techniques are reviewed and discussed. The first question, addressed in Part 2, has been whether the simulation of reality with renders of photographic quality, a product of automated computerisation, would relate to the observer better or worse than a traditionally hand drawn image. A questionnaire has been constructed to probe the communication and representation qualities of the images. The results suggested that these two qualities were best represented in the case of freehand drawing compared to photorealistic renders. The Part 3 was focused on the designer and used the instruments provided by neuroscience, i.e. the EEG and evoked responses, to quantify the brain activity in connection with freehand and CAD drawing. Because the involved technology had never been applied before to a drawing subject, the investigation had to be divided into three separate experiments. The first was dedicated to find a reliable way to perform the recordings in a subject who freely moved arm and hand while drawing for a few minutes; the second and third experiment were aimed at the analysis of cerebral activity occurring before and after the drawing movement. All results demonstrated that a definitely larger cerebral activity preceded and followed each movement in freehand drawing compared to CAD design. This finding may be considered a robust step towards the notion that also creativity is consequently improved. The final conclusion is that the freehand drawn images make a better link between author and observer, and at the same time the very movement and haptic perception of the hand elicit creativity. Nevertheless, the most recent advances in digitalising analogic dimensions through drawing tablets, have provided a new medium for freehand drawing, which can exploit the capacity of data handling of computers, time and effort saving, with the natural movement of using pencil and paper, ending up in an identical product. A wise usage of modern technology can therefore merge together the human factor of hand and brain reciprocal thrust with the fictitious world of the bits

Freehand Digital Drawing: a Boost to Creative Design. The Observer's Eye and the Draftsman Brain

[ES] La representación de un proyecto arquitectónico tiene diferentes finalidades, una de las más relevantes es su visualización. El tema de esta tesis es la relación entre el diseñador, su actividad cerebral durante la fase creativa, el proyecto representado y la mente del observador. La pregunta, planteada por arquitectos, diseñadores y académicos en los años ochenta y aún abierta a soluciones estimulantes, es una pregunta bien conocida sobre si la creciente digitalización puede ser perjudicial o beneficiosa para esta relación. Después de un análisis de la imagen en la arquitectura a partir del Renacimiento, informado en la Parte 1, se revisan y discuten las técnicas actuales de representación. La primera pregunta, abordada en la Parte 2, fue si la simulación de la realidad con renderizado de calidad fotográfica garantiza una visualización mejor o peor que una imagen tradicionalmente dibujada a mano. Se elaboró un cuestionario para verificar las características de comunicación y representación de las imágenes producidas con las dos técnicas. Los resultados mostraron que estas dos cualidades estaban más presentes en el caso del dibujo a mano alzada que en los renders fotorrealistas. La Parte 3 se centró en el diseñador y utilizó las herramientas proporcionadas por la neurociencia, a saber, el EEG y las 'respuestas evocadas', para cuantificar la actividad cerebral de alguien que hace un dibujo a mano alzada con respecto a CAD. Dado que la tecnología utilizada nunca antes se había aplicado a un sujeto al realizar un diseño, la investigación se dividió en tres experimentos separados. El primero se dedicó a encontrar una forma confiable de realizar registros durante el movimiento del miembro superior en la ejecución del dibujo; los experimentos segundo y tercero tenían como objetivo el análisis específico de la actividad cerebral que precede y sigue a los movimientos individuales de los signos trazados. Los resultados mostraron que la actividad motora relacionada con el dibujo a mano alzada estuvo acompañada por un aumento en la amplitud de las respuestas electroencefalográficas en comparación con el dibujo con CAD. Este dato es probablemente un importante paso adelante a favor de la hipótesis de que la creatividad también puede mejorarse mediante el movimiento en el dibujo a mano alzada. La conclusión es que las imágenes así dibujadas crean una mejor correspondencia entre el autor y el observador, y que el movimiento y la percepción sensorial de la mano pueden fomentar la creatividad. Recientemente, los avances en la técnica de las tabletas de dibujo han proporcionado una nueva técnica para el dibujo a mano alzada, donde se combina la capacidad de gestión de datos del ordenador con los gestos naturales tradicionalmente ligados al uso del papel y el lápiz. de la tecnología moderna puede fusionar el factor humano junto con las nuevas tecnologías digitales.[CA] La representació d'un projecte arquitectònic té diferents finalitats, una de les més rellevants és la seva visualització. El tema d'aquesta tesi és la relació entre el dissenyador, la seva activitat cerebral durant la fase creativa, el projecte representat i la ment de l'observador. La pregunta, plantejada per arquitectes, dissenyadors i estudiosos als anys vuitanta i encara oberta a solucions estimulants, és una qüestió ben coneguda si augmentar la digitalització pot ser perjudicial o beneficiós per a aquesta relació. Després d'una anàlisi de la imatge en arquitectura a partir del període renaixentista, informada a la part 1, es revisen i discuteixen les tècniques de representació actuals. La primera pregunta, abordada a la part 2, va ser si la simulació de la realitat amb renderització de qualitat fotogràfica garanteix una visualització millor o pitjor que una imatge tradicionalment dibuixada a mà. Es va elaborar un qüestionari per comprovar les característiques de comunicació i representació de les imatges produïdes amb les dues tècniques. Els resultats van mostrar que aquestes dues qualitats estaven més presents en el cas del dibuix a mà alçada que en els renders fotorealistes. La part 3 es va centrar en el dissenyador i va utilitzar les eines proporcionades per la neurociència, és a dir, l'EEG i les 'respostes evocades', per quantificar l'activitat cerebral d'algú que realitza un dibuix a mà alçada respecte al CAD. Com que la tecnologia utilitzada mai abans s'havia aplicat a un tema mentre es feia un disseny, la investigació es va dividir en tres experiments separats. El primer es va dedicar a trobar una manera fiable de fer enregistraments durant el moviment de l'extremitat superior en l'execució del dibuix; el segon i el tercer experiment estaven dirigits a l'anàlisi específica de l'activitat cerebral que precedeix i segueix els moviments individuals dels signes traçats. Els resultats van mostrar que l'activitat motora relacionada amb el dibuix a mà alçada va anar acompanyada d'un augment de l'amplitud de les respostes electroencefalogràfiques en comparació amb el dibuix amb CAD. Aquesta dada és probablement un pas endavant significatiu a favor de la hipòtesi que la creativitat també es pot millorar amb el moviment en el dibuix a mà alçada. La conclusió és que les imatges dibuixades d'aquesta manera creen una millor correspondència entre autor i observador, i que el moviment i la percepció sensorial de la mà poden fomentar la creativitat. Recentment, els avenços en la tècnica de dibuix a tauletes han aportat una nova tècnica de dibuix a mà alçada, on la capacitat de gestió de dades de l'ordinador es combina amb els gestos naturals tradicionalment vinculats a l'ús del paper i el llapis. de la tecnologia moderna pot fusionar el factor humà amb les noves tecnologies digitals.[EN] The representation of an architectural project aims at several intents, one of the most relevant being the visualisation of a design. The subject of this dissertation is therefore the relationship between the draftsman, the creativity of his brain, the represented design, and the mind of the observer. Since the Eighties, architects, designers and scholars wondered whether the current habit of an ever increasing digitalisation could be detrimental or advantageous to such relationship. After an analysis of architectural imaging since the Renaissance, as reported in Part 1, the current techniques are reviewed and discussed. The first question, addressed in Part 2, has been whether the simulation of reality with renders of photographic quality, would relate to the observer better or worse than a traditionally hand drawn image. A questionnaire has been constructed to probe the communication and representation qualities of the images. The results suggested that these two qualities were best represented in the case of freehand drawing compared to photorealistic renders. Part 3 was focused on the designer and used the instruments provided by neuroscience, i.e. the EEG and 'evoked responses', to quantify the brain activity in connection with freehand and CAD drawing. Because the involved technology had never been applied before to a drawing subject, the investigation had to be divided into three separate experiments. The first one was dedicated to finding a reliable way to perform the recordings in subjects who freely moved their arm and hand while drawing. The second and third experiments were aimed at analysing the cerebral activity occurring before and after each drawing movement. All results demonstrated that a larger cerebral activity preceded and followed each movement in freehand drawing compared to CAD design. This finding may be considered a robust step towards the notion that also creativity may consequently be improved. The final conclusion is that the freehand drawn images make a better link between author and observer, and at the same time the very movement and haptic perception of the hand elicit creativity. Indeed, the most recent advances in technology of drawing tablets have provided a new medium for freehand drawing, which can exploit the capacity of data handling of computers with the natural movement of using pencil and paper, ending up in a traditional hand made product. A wise usage of modern technology can therefore merge the human factor with the digital world.Leandri, G. (2022). Freehand Digital Drawing: a Boost to Creative Design. The Observer's Eye and the Draftsman Brain [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/188091TESI