Sketchy rendering for information visualization

We present and evaluate a framework for constructing sketchy style information visualizations that mimic data graphics drawn by hand. We provide an alternative renderer for the Processing graphics environment that redefines core drawing primitives including line, polygon and ellipse rendering. These primitives allow higher-level graphical features such as bar charts, line charts, treemaps and node-link diagrams to be drawn in a sketchy style with a specified degree of sketchiness. The framework is designed to be easily integrated into existing visualization implementations with minimal programming modification or design effort. We show examples of use for statistical graphics, conveying spatial imprecision and for enhancing aesthetic and narrative qualities of visual- ization. We evaluate user perception of sketchiness of areal features through a series of stimulus-response tests in order to assess users’ ability to place sketchiness on a ratio scale, and to estimate area. Results suggest relative area judgment is compromised by sketchy rendering and that its influence is dependent on the shape being rendered. They show that degree of sketchiness may be judged on an ordinal scale but that its judgement varies strongly between individuals. We evaluate higher-level impacts of sketchiness through user testing of scenarios that encourage user engagement with data visualization and willingness to critique visualization de- sign. Results suggest that where a visualization is clearly sketchy, engagement may be increased and that attitudes to participating in visualization annotation are more positive. The results of our work have implications for effective information visualization design that go beyond the traditional role of sketching as a tool for prototyping or its use for an indication of general uncertainty

Play and Learn: Using Video Games to Train Computer Vision Models

Video games are a compelling source of annotated data as they can readily provide fine-grained groundtruth for diverse tasks. However, it is not clear whether the synthetically generated data has enough resemblance to the real-world images to improve the performance of computer vision models in practice. We present experiments assessing the effectiveness on real-world data of systems trained on synthetic RGB images that are extracted from a video game. We collected over 60000 synthetic samples from a modern video game with similar conditions to the real-world CamVid and Cityscapes datasets. We provide several experiments to demonstrate that the synthetically generated RGB images can be used to improve the performance of deep neural networks on both image segmentation and depth estimation. These results show that a convolutional network trained on synthetic data achieves a similar test error to a network that is trained on real-world data for dense image classification. Furthermore, the synthetically generated RGB images can provide similar or better results compared to the real-world datasets if a simple domain adaptation technique is applied. Our results suggest that collaboration with game developers for an accessible interface to gather data is potentially a fruitful direction for future work in computer vision.Comment: To appear in the British Machine Vision Conference (BMVC), September 2016. -v2: fixed a typo in the reference

3D Line textures and the visualization of confidence in Architecture

technical reportThis work introduces a technique for interactive walkthroughs of non-photorealistically rendered (NPR) scenes using 3D line primitives to define architectural features of the model, as well as indicate textural qualities. Line primitives are not typically used in this manner in favor of texture mapping techniques which can encapsulate a great deal of information in a single texture map, and take advantage of GPU optimizations for accelerated rendering. However, texture mapped images may not maintain the visual quality or aesthetic appeal that is possible when using 3D lines to simulate NPR scenes such as hand-drawn illustrations or architectural renderings. In addition, line textures can be modi ed interactively, for instance changing the sketchy quality of the lines, and can be exported as vectors to allow the automatic generation of illustrations and further modi cation in vector-based graphics programs. The technique introduced here extracts feature edges from a model, and using these edges, generates a reduced set of line textures which indicate material properties while maintaining interactive frame rates. A clipping algorithm is presented to enable 3D lines to reside only in the interior of the 3D model without exposing the underlying triangulated mesh. The resulting system produces interactive illustrations with high visual quality that are free from animation artifacts

Freehand Digital Drawing: a Boost to Creative Design. The Observer's Eye and the Draftsman Brain

[ES] La representación de un proyecto arquitectónico tiene diferentes finalidades, una de las más relevantes es su visualización. El tema de esta tesis es la relación entre el diseñador, su actividad cerebral durante la fase creativa, el proyecto representado y la mente del observador. La pregunta, planteada por arquitectos, diseñadores y académicos en los años ochenta y aún abierta a soluciones estimulantes, es una pregunta bien conocida sobre si la creciente digitalización puede ser perjudicial o beneficiosa para esta relación. Después de un análisis de la imagen en la arquitectura a partir del Renacimiento, informado en la Parte 1, se revisan y discuten las técnicas actuales de representación. La primera pregunta, abordada en la Parte 2, fue si la simulación de la realidad con renderizado de calidad fotográfica garantiza una visualización mejor o peor que una imagen tradicionalmente dibujada a mano. Se elaboró un cuestionario para verificar las características de comunicación y representación de las imágenes producidas con las dos técnicas. Los resultados mostraron que estas dos cualidades estaban más presentes en el caso del dibujo a mano alzada que en los renders fotorrealistas. La Parte 3 se centró en el diseñador y utilizó las herramientas proporcionadas por la neurociencia, a saber, el EEG y las 'respuestas evocadas', para cuantificar la actividad cerebral de alguien que hace un dibujo a mano alzada con respecto a CAD. Dado que la tecnología utilizada nunca antes se había aplicado a un sujeto al realizar un diseño, la investigación se dividió en tres experimentos separados. El primero se dedicó a encontrar una forma confiable de realizar registros durante el movimiento del miembro superior en la ejecución del dibujo; los experimentos segundo y tercero tenían como objetivo el análisis específico de la actividad cerebral que precede y sigue a los movimientos individuales de los signos trazados. Los resultados mostraron que la actividad motora relacionada con el dibujo a mano alzada estuvo acompañada por un aumento en la amplitud de las respuestas electroencefalográficas en comparación con el dibujo con CAD. Este dato es probablemente un importante paso adelante a favor de la hipótesis de que la creatividad también puede mejorarse mediante el movimiento en el dibujo a mano alzada. La conclusión es que las imágenes así dibujadas crean una mejor correspondencia entre el autor y el observador, y que el movimiento y la percepción sensorial de la mano pueden fomentar la creatividad. Recientemente, los avances en la técnica de las tabletas de dibujo han proporcionado una nueva técnica para el dibujo a mano alzada, donde se combina la capacidad de gestión de datos del ordenador con los gestos naturales tradicionalmente ligados al uso del papel y el lápiz. de la tecnología moderna puede fusionar el factor humano junto con las nuevas tecnologías digitales.[CA] La representació d'un projecte arquitectònic té diferents finalitats, una de les més rellevants és la seva visualització. El tema d'aquesta tesi és la relació entre el dissenyador, la seva activitat cerebral durant la fase creativa, el projecte representat i la ment de l'observador. La pregunta, plantejada per arquitectes, dissenyadors i estudiosos als anys vuitanta i encara oberta a solucions estimulants, és una qüestió ben coneguda si augmentar la digitalització pot ser perjudicial o beneficiós per a aquesta relació. Després d'una anàlisi de la imatge en arquitectura a partir del període renaixentista, informada a la part 1, es revisen i discuteixen les tècniques de representació actuals. La primera pregunta, abordada a la part 2, va ser si la simulació de la realitat amb renderització de qualitat fotogràfica garanteix una visualització millor o pitjor que una imatge tradicionalment dibuixada a mà. Es va elaborar un qüestionari per comprovar les característiques de comunicació i representació de les imatges produïdes amb les dues tècniques. Els resultats van mostrar que aquestes dues qualitats estaven més presents en el cas del dibuix a mà alçada que en els renders fotorealistes. La part 3 es va centrar en el dissenyador i va utilitzar les eines proporcionades per la neurociència, és a dir, l'EEG i les 'respostes evocades', per quantificar l'activitat cerebral d'algú que realitza un dibuix a mà alçada respecte al CAD. Com que la tecnologia utilitzada mai abans s'havia aplicat a un tema mentre es feia un disseny, la investigació es va dividir en tres experiments separats. El primer es va dedicar a trobar una manera fiable de fer enregistraments durant el moviment de l'extremitat superior en l'execució del dibuix; el segon i el tercer experiment estaven dirigits a l'anàlisi específica de l'activitat cerebral que precedeix i segueix els moviments individuals dels signes traçats. Els resultats van mostrar que l'activitat motora relacionada amb el dibuix a mà alçada va anar acompanyada d'un augment de l'amplitud de les respostes electroencefalogràfiques en comparació amb el dibuix amb CAD. Aquesta dada és probablement un pas endavant significatiu a favor de la hipòtesi que la creativitat també es pot millorar amb el moviment en el dibuix a mà alçada. La conclusió és que les imatges dibuixades d'aquesta manera creen una millor correspondència entre autor i observador, i que el moviment i la percepció sensorial de la mà poden fomentar la creativitat. Recentment, els avenços en la tècnica de dibuix a tauletes han aportat una nova tècnica de dibuix a mà alçada, on la capacitat de gestió de dades de l'ordinador es combina amb els gestos naturals tradicionalment vinculats a l'ús del paper i el llapis. de la tecnologia moderna pot fusionar el factor humà amb les noves tecnologies digitals.[EN] The representation of an architectural project aims at several intents, one of the most relevant being the visualisation of a design. The subject of this dissertation is therefore the relationship between the draftsman, the creativity of his brain, the represented design, and the mind of the observer. Since the Eighties, architects, designers and scholars wondered whether the current habit of an ever increasing digitalisation could be detrimental or advantageous to such relationship. After an analysis of architectural imaging since the Renaissance, as reported in Part 1, the current techniques are reviewed and discussed. The first question, addressed in Part 2, has been whether the simulation of reality with renders of photographic quality, would relate to the observer better or worse than a traditionally hand drawn image. A questionnaire has been constructed to probe the communication and representation qualities of the images. The results suggested that these two qualities were best represented in the case of freehand drawing compared to photorealistic renders. Part 3 was focused on the designer and used the instruments provided by neuroscience, i.e. the EEG and 'evoked responses', to quantify the brain activity in connection with freehand and CAD drawing. Because the involved technology had never been applied before to a drawing subject, the investigation had to be divided into three separate experiments. The first one was dedicated to finding a reliable way to perform the recordings in subjects who freely moved their arm and hand while drawing. The second and third experiments were aimed at analysing the cerebral activity occurring before and after each drawing movement. All results demonstrated that a larger cerebral activity preceded and followed each movement in freehand drawing compared to CAD design. This finding may be considered a robust step towards the notion that also creativity may consequently be improved. The final conclusion is that the freehand drawn images make a better link between author and observer, and at the same time the very movement and haptic perception of the hand elicit creativity. Indeed, the most recent advances in technology of drawing tablets have provided a new medium for freehand drawing, which can exploit the capacity of data handling of computers with the natural movement of using pencil and paper, ending up in a traditional hand made product. A wise usage of modern technology can therefore merge the human factor with the digital world.Leandri, G. (2022). Freehand Digital Drawing: a Boost to Creative Design. The Observer's Eye and the Draftsman Brain [Tesis doctoral]. Universitat Politècnica de València. https://doi.org/10.4995/Thesis/10251/188091TESI

FREEHAND DIGITAL DRAWING: A BOOST TO CREATIVE DESIGN. THE OBSERVER’S EYE AND THE DRAFTSMAN’S BRAIN

Representation of an architectural project aims at several intents; one of the most relevant being the visualization of a design. The subject of this dissertation is therefore the relationship between the draftsman, the creativity of his brain, the represented design, and the mind of the observer, while raising the question whether the current habit of an ever increasing digitalisation may be detrimental or advantageous to such relationship. After an analysis of architectural representation in history since Renaissance times, as reported in Part 1, the current techniques are reviewed and discussed. The first question, addressed in Part 2, has been whether the simulation of reality with renders of photographic quality, a product of automated computerisation, would relate to the observer better or worse than a traditionally hand drawn image. A questionnaire has been constructed to probe the communication and representation qualities of the images. The results suggested that these two qualities were best represented in the case of freehand drawing compared to photorealistic renders. The Part 3 was focused on the designer and used the instruments provided by neuroscience, i.e. the EEG and evoked responses, to quantify the brain activity in connection with freehand and CAD drawing. Because the involved technology had never been applied before to a drawing subject, the investigation had to be divided into three separate experiments. The first was dedicated to find a reliable way to perform the recordings in a subject who freely moved arm and hand while drawing for a few minutes; the second and third experiment were aimed at the analysis of cerebral activity occurring before and after the drawing movement. All results demonstrated that a definitely larger cerebral activity preceded and followed each movement in freehand drawing compared to CAD design. This finding may be considered a robust step towards the notion that also creativity is consequently improved. The final conclusion is that the freehand drawn images make a better link between author and observer, and at the same time the very movement and haptic perception of the hand elicit creativity. Nevertheless, the most recent advances in digitalising analogic dimensions through drawing tablets, have provided a new medium for freehand drawing, which can exploit the capacity of data handling of computers, time and effort saving, with the natural movement of using pencil and paper, ending up in an identical product. A wise usage of modern technology can therefore merge together the human factor of hand and brain reciprocal thrust with the fictitious world of the bits

Non-photorealistic rendering: a critical examination and proposed system.

In the first part of the program the emergent field of Non-Photorealistic Rendering is explored from a cultural perspective. This is to establish a clear understanding of what Non-Photorealistic Rendering (NPR) ought to be in its mature form in order to provide goals and an overall infrastructure for future development. This thesis claims that unless we understand and clarify NPR's relationship with other media (photography, photorealistic computer graphics and traditional media) we will continue to manufacture "new solutions" to computer based imaging which are confused and naive in their goals. Such solutions will be rejected by the art and design community, generally condemned as novelties of little cultural worth ( i.e. they will not sell). This is achieved by critically reviewing published systems that are naively described as Non-photorealistic or "painterly" systems. Current practices and techniques are criticised in terms of their low ability to articulate meaning in images; solutions to this problem are given. A further argument claims that NPR, while being similar to traditional "natural media" techniques in certain aspects, is fundamentally different in other ways. This similarity has lead NPR to be sometimes proposed as "painting simulation" — something it can never be. Methods for avoiding this position are proposed. The similarities and differences to painting and drawing are presented and NPR's relationship to its other counterpart, Photorealistic Rendering (PR), is then delineated. It is shown that NPR is paradigmatically different to other forms of representation — i.e. it is not an "effect", but rather something basically different. The benefits of NPR in its mature form are discussed in the context of Architectural Representation and Design in general. This is done in conjunction with consultations with designers and architects. From this consultation a "wish-list" of capabilities is compiled by way of a requirements capture for a proposed system. A series of computer-based experiments resulting in the systems "Expressive Marks" and 'Magic Painter" are carried out; these practical experiments add further understanding to the problems of NPR. The exploration concludes with a prototype system "Piranesi" which is submitted as a good overall solution to the problem of NPR. In support of this written thesis are : - • The Expressive Marks system • Magic Painter system • The Piranesi system (which includes the EPixel and Sketcher systems) • A large portfolio of images generated throughout the exploration