Artistic vision: painterly rendering using computer vision techniques

Journal ArticleWe present a method that takes a raster image as input and produces a painting-like image composed of strokes rather than pixels. Unlike previous automatic painting methods, we attempt to keep the number of brush-stroke small. This is accomplished by first segmenting the image into features, finding the medial axes points of these features, converting the medial axes points into ordered lists of image tokens, and finally rendering these lists as brush strokes. Our process creates images reminiscent of modern realist painters who often want an abstract or sketchy quality in their work

Accurate and discernible photocollages

There currently exist several techniques for selecting and combining images from a digital image library into a single image so that the result meets certain prespecified visual criteria. Image mosaic methods, first explored by Connors and Trivedi[18], arrange library images according to some tiling arrangement, often a regular grid, so that the combination of images, when viewed as a whole, resembles some input target image. Other techniques, such as Autocollage of Rother et al.[78], seek only to combine images in an interesting and visually pleasing manner, according to certain composition principles, without attempting to approximate any target image. Each of these techniques provide a myriad of creative options for artists who wish to combine several levels of meaning into a single image or who wish to exploit the meaning and symbolism contained in each of a large set of images through an efficient and easy process. We first examine the most notable and successful of these methods, and summarize the advantages and limitations of each. We then formulate a set of goals for an image collage system that combines the advantages of these methods while addressing and mitigating the drawbacks. Particularly, we propose a system for creating photocollages that approximate a target image as an aggregation of smaller images, chosen from a large library, so that interesting visual correspondences between images are exploited. In this way, we allow users to create collages in which multiple layers of meaning are encoded, with meaningful visual links between each layer. In service of this goal, we ensure that the images used are as large as possible and are combined in such a way that boundaries between images are not immediately apparent, as in Autocollage. This has required us to apply a multiscale approach to searching and comparing images from a large database, which achieves both speed and accuracy. We also propose a new framework for color post-processing, and propose novel techniques for decomposing images according to object and texture information

Bractwo Prerafaelickie - malarstwo i poezja

Pre-Raphaelitism in art and literature is in many respects regarded as a foreshadowing of Modernism. Rossetti, in his creation of mood and inclusion of psychological analysis in poetry, heralds the techniques and concerns of Imagism, and his experiments with consciousness and phenomenology are combined with a devotion to an idea of the image as a compact and revelatory expression of a state or feeling (an idea comparable to the Modernist “epiphany”). Pre-Raphaelitism in poetry had a major influence upon the writers of the Decadence, and on Gerard Manley Hopkins and William Butler Yeats, who were also influenced by Ruskin and visual Pre-Raphaelitism. Additionally, Yeats’s private symbolism and the visual aspects of his poetry bear much resemblance to Pre-Raphaelite ideas. The examples gathered in the present study show that the work of the Brotherhood did not pass away as a short-lived phenomenon, but instead paved the way for future generations of poets and painters. The Pre-Raphaelite legacy is significant, and the achievements of the movement encompass both poetry and painting, which they freed from their constraints simultaneously, bringing the two areas of artistic expression closer together. The analytical approach taken towards the interrelationship of poetry and painting in this study has consisted of a treatment of both forms of expression as “secondary modelling systems,” composed of distinguishable meaningful units (refered to as “images” throughout the argument). The application of this approach to both arts has demonstrated that those units are indeed comparable even though they are created in different artistic domains and employ different tools, methods and material in order to express meanings. As has been shown, these arts can be analysed in a parallel fashion if they are treated as texts that can be read regardless of the kind of code they are composed of. Therefore, it is possible to demonstrate that the intellectual operations involved in both the poetic and the painterly text creation and reception processes are similar. To show that it is possible to compare these arts has been the broadest aim of this study. Such a comparison leads to the discovery of specific similarities between the two artistic domains. In this study, a few fields of intellectual operations have een treated as those that particularly facilitate investigation of the resemblances between poetry and painting. First of all, notions of spatiality and temporality have been the most informative and constructive in terms of cross-artistic references. As has been proved, neither space nor time can be unequivocally associated with only one form of expression; even though the refutation of Lessing’s traditional approach is not an innovative contribution of the present study to the research field, my argument provides evidence that spatiality and temporality are not realms differentiating poetry from painting but are, in fact, spheres that facilitate the substantiation of the close connection between the arts. The evidence was gathered throughout the demonstration of the interconnectedness, but also ambivalence, of time and space as concepts as well as due to the revelation of the presence of temporal structures in painting and spatial relations in poetry, which was a traditionally repudiated argument. The next broad notion that this study has been concerned with is spectatorship. The concept has appeared with the same frequency in interpretations of poetry and in analyses of painting; this fact can serve as an argument supporting the proximity of these arts. More specifically, the idea of looking, when it appears in discussions of poetry, immediately steers these discussions towards the topic of human visual perception, which conventionally relates to pictorial arts. At this point, what I believe to be two significant achievements of the current study should be mentioned: first, not only are the themes of human perception, visuality and even physical optics discussed in relation to painterly creation, but, and this is worth emphasising, they are also explored in connection with poetic accounts. Secondly, the argument successfully, I believe, merges the concepts of the “reader” and the “viewer” into one locution – the “reader/viewer” – in discussions of these two, thereby conceptually fused arts. The most prominent (but not the only) illustration of this achievement is the phenomenon of the double work of art, which was discussed as being, in varying degrees, a unified form of painterly/poetic expression. Of course, it would be an overstatement to claim that such an approach to the recipient of the artistic experience is an innovative contribution of the present study, but contemporary criticism is still too bashful in using the “reader/viewer” as a legitimate, conceptual entity in discussions of painting/poetry as fused forms of expression. Inter-systemic correspondence was most evidently exemplified in the parts of this dissertation concerned with the verbal features of painterly expression and the visual qualities of poetry. As was pointed out, paintings do indeed show “exclusively” poetic attributes, like the ability to generate narratives, equivalents of rhetorical figures, devices that indicate mood, and self-reflexive remarks. Poetry, on the other hand, has painterly characteristics such as indications of space, attention to detail and references to the act of looking itself. These sets of attributes do not only reveal that painting and poetry share each other’s features, but also that each of those forms of art can imitate the other. The intermedial “exchange” of properties leads to a situation in which verbal images can actually “simulate” pictorial occurrences, and, reversibly, visual images “pretend” to belong to the realm of verbalism. The presentation of such examples in this study is, I believe, a further noteworthy accomplishment. As was stated at the outset of this study, similarities inevitably entail differences. These are most vividly observable in transformations of poetry and painting into the other medium, in which cases the mode of inter-systemic translation is exposed. The most frequent alterations involve the emphases and contexts that are changed during the process of conversion. For example, the painter or the poet may choose to focus on a different area than the one highlighted in the source work, or the converted work can be situated in altered historical or cultural circumstances. Frequently, the time perspectives are also changed; poetic extensions of momentary actions from painting obviously indicate a discrepancy between the verbal and visual renderings. This fact, however, does not mean that prolonged time perspectives are not achievable in painting: this study examines examples of visual narratives, or socalled “pregnant moments” with extended chronology. Thus, the difference in question is one between particular versions of the same motif or a deliberate modification of the source meanings by the “appropriator,” not a difference dictated by the insufficiency of any of these forms of art. Another dissimilarity, and, this time, a certain deficiency of the visual medium, surfaces in the painters’ attempts to render ambiguity, moral dilemmas or complex spiritual experiences. Such transformations often involve a narrowing of meaning, and the resulting painterly representations prove limited, simplified or too obvious (as in the case of Hunt’s depictions of Shakespeare’s characters). Nevertheless, instances of successful renderings of even the most intricate spiritual states are also numerous, as is proved by the example of Rossetti’s Dantesque paintings. Therefore, it is not legitimate to claim that painting is incapable of conveying abstract complexities. It has not been the aim of this study to desperately search for differences, and, as it turns out, when dissimilarities between poetry and painting occur, they are not essentially conditioned by the nature of these art forms. One could even say that there are no discrepancies between the arts that could not be reconciled on the condition that both arts are approached with the same attitude towards their creation, structure and mechanisms. The stance assumed in the present study is, in fact, exactly of this kind, with its emphasis on the image-istic, textual and readerly character of both forms of expression

Non-photorealistic rendering: a critical examination and proposed system.

In the first part of the program the emergent field of Non-Photorealistic Rendering is explored from a cultural perspective. This is to establish a clear understanding of what Non-Photorealistic Rendering (NPR) ought to be in its mature form in order to provide goals and an overall infrastructure for future development. This thesis claims that unless we understand and clarify NPR's relationship with other media (photography, photorealistic computer graphics and traditional media) we will continue to manufacture "new solutions" to computer based imaging which are confused and naive in their goals. Such solutions will be rejected by the art and design community, generally condemned as novelties of little cultural worth ( i.e. they will not sell). This is achieved by critically reviewing published systems that are naively described as Non-photorealistic or "painterly" systems. Current practices and techniques are criticised in terms of their low ability to articulate meaning in images; solutions to this problem are given. A further argument claims that NPR, while being similar to traditional "natural media" techniques in certain aspects, is fundamentally different in other ways. This similarity has lead NPR to be sometimes proposed as "painting simulation" — something it can never be. Methods for avoiding this position are proposed. The similarities and differences to painting and drawing are presented and NPR's relationship to its other counterpart, Photorealistic Rendering (PR), is then delineated. It is shown that NPR is paradigmatically different to other forms of representation — i.e. it is not an "effect", but rather something basically different. The benefits of NPR in its mature form are discussed in the context of Architectural Representation and Design in general. This is done in conjunction with consultations with designers and architects. From this consultation a "wish-list" of capabilities is compiled by way of a requirements capture for a proposed system. A series of computer-based experiments resulting in the systems "Expressive Marks" and 'Magic Painter" are carried out; these practical experiments add further understanding to the problems of NPR. The exploration concludes with a prototype system "Piranesi" which is submitted as a good overall solution to the problem of NPR. In support of this written thesis are : - • The Expressive Marks system • Magic Painter system • The Piranesi system (which includes the EPixel and Sketcher systems) • A large portfolio of images generated throughout the exploration

Integrated multi-scale architecture of the cortex with application to computer vision

Tese de dout., Engenharia Electrónica e de Computadores, Faculdade de Ciência e Tecnologia, Universidade do Algarve, 2007The main goal of this thesis is to try to understand the functioning of the visual cortex through the development of computational models. In the input layer V1 of the visual cortex there are simple, complex and endstopped cells. These provide a multi-scale representation of objects and scene in terms of lines, edges and keypoints. In this thesis we combine recent progress concerning the development of computational models of these and other cells with processes in higher cortical areas V2 and V4 etc. Three pertinent challenges are discussed: (i) object recognition embedded in a cortical architecture; (ii) brightness perception, and (iii) painterly rendering based on human vision. Specific aspects are Focusof- Attention by means of keypoint-based saliency maps, the dynamic routing of features from V1 through higher cortical areas in order to obtain translation, rotation and size invariance, and the construction of normalized object templates with canonical views in visual memory. Our simulations show that the multi-scale representations can be integrated into a cortical architecture in order to model subsequent processing steps: from segregation, via different categorization levels, until final object recognition is obtained. As for real cortical processing, the system starts with coarse-scale information, refines categorization by using mediumscale information, and employs all scales in recognition. We also show that a 2D brightness model can be based on the multi-scale symbolic representation of lines and edges, with an additional low-pass channel and nonlinear amplitude transfer functions, such that object recognition and brightness perception are combined processes based on the same information. The brightness model can predict many different effects such as Mach bands, grating induction, the Craik-O’Brien-Cornsweet illusion and brightness induction, i.e. the opposite effects of assimilation (White effect) and simultaneous brightness contrast. Finally, a novel application is introduced: painterly rendering has been linked to computer vision, but we propose to link it to human vision because perception and painting are two processes which are strongly interwoven

Implementing non-photorealistic rendreing enhancements with real-time performance

We describe quality and performance enhancements, which work in real-time, to all well-known Non-photorealistic (NPR) rendering styles for use in an interactive context. These include Comic rendering, Sketch rendering, Hatching and Painterly rendering, but we also attempt and justify a widening of the established definition of what is considered NPR. In the individual Chapters, we identify typical stylistic elements of the different NPR styles. We list problems that need to be solved in order to implement the various renderers. Standard solutions available in the literature are introduced and in all cases extended and optimised. In particular, we extend the lighting model of the comic renderer to include a specular component and introduce multiple inter-related but independent geometric approximations which greatly improve rendering performance. We implement two completely different solutions to random perturbation sketching, solve temporal coherence issues for coal sketching and find an unexpected use for 3D textures to implement hatch-shading. Textured brushes of painterly rendering are extended by properties such as stroke-direction and texture, motion, paint capacity, opacity and emission, making them more flexible and versatile. Brushes are also provided with a minimal amount of intelligence, so that they can help in maximising screen coverage of brushes. We furthermore devise a completely new NPR style, which we call super-realistic and show how sample images can be tweened in real-time to produce an image-based six degree-of-freedom renderer performing at roughly 450 frames per second. Performance values for our other renderers all lie between 10 and over 400 frames per second on homePC hardware, justifying our real-time claim. A large number of sample screen-shots, illustrations and animations demonstrate the visual fidelity of our rendered images. In essence, we successfully achieve our attempted goals of increasing the creative, expressive and communicative potential of individual NPR styles, increasing performance of most of them, adding original and interesting visual qualities, and exploring new techniques or existing ones in novel ways.KMBT_363Adobe Acrobat 9.54 Paper Capture Plug-i

Toward a Perceptually-relevant Theory of Appearance

Two approaches are commonly employed in Computer Graphics to design and adjust the appearance of objects in a scene. A full 3D environment may be created, through geometrical, material and lighting modeling, then rendered using a simulation of light transport; appearance is then controlled in ways similar to photography. A radically different approach consists in providing 2D digital drawing tools to an artist, whom with enough talent and time will be able to create images of objects having the desired appearance; this is obviously strongly similar to what traditional artists do, with the computer being a mere modern drawing tool.In this document, I present research projects that have investigated a third approach, whereby pictorial elements of appearance are explicitly manipulated by an artist. On the one side, such an alternative approach offers a direct control over appearance, with novel applications in vector drawing, scientific illustration, special effects and video games. On the other side, it provides an modern method for putting our current knowledge of the perception of appearance to the test, as well as to suggest new models for human vision along the way

Colour videos with depth : acquisition, processing and evaluation

The human visual system lets us perceive the world around us in three dimensions by integrating evidence from depth cues into a coherent visual model of the world. The equivalent in computer vision and computer graphics are geometric models, which provide a wealth of information about represented objects, such as depth and surface normals. Videos do not contain this information, but only provide per-pixel colour information. In this dissertation, I hence investigate a combination of videos and geometric models: videos with per-pixel depth (also known as RGBZ videos). I consider the full life cycle of these videos: from their acquisition, via filtering and processing, to stereoscopic display. I propose two approaches to capture videos with depth. The first is a spatiotemporal stereo matching approach based on the dual-cross-bilateral grid – a novel real-time technique derived by accelerating a reformulation of an existing stereo matching approach. This is the basis for an extension which incorporates temporal evidence in real time, resulting in increased temporal coherence of disparity maps – particularly in the presence of image noise. The second acquisition approach is a sensor fusion system which combines data from a noisy, low-resolution time-of-flight camera and a high-resolution colour video camera into a coherent, noise-free video with depth. The system consists of a three-step pipeline that aligns the video streams, efficiently removes and fills invalid and noisy geometry, and finally uses a spatiotemporal filter to increase the spatial resolution of the depth data and strongly reduce depth measurement noise. I show that these videos with depth empower a range of video processing effects that are not achievable using colour video alone. These effects critically rely on the geometric information, like a proposed video relighting technique which requires high-quality surface normals to produce plausible results. In addition, I demonstrate enhanced non-photorealistic rendering techniques and the ability to synthesise stereoscopic videos, which allows these effects to be applied stereoscopically. These stereoscopic renderings inspired me to study stereoscopic viewing discomfort. The result of this is a surprisingly simple computational model that predicts the visual comfort of stereoscopic images. I validated this model using a perceptual study, which showed that it correlates strongly with human comfort ratings. This makes it ideal for automatic comfort assessment, without the need for costly and lengthy perceptual studies

The sublime network; painterly passage and materiality in the post internet era

This practice-led research project investigates painting in the Post Internet era. In the vast database of the Internet, paintings both historical and contemporary are distanced from their makers and contexts. Their diachronic position in a once considered "linear" historical model has been disrupted. The Internet fractures historical narratives, identities and oeuvres and makes them miscellaneous. Through painting I aim to communicate the paradoxical disparity and conflation that occurs between artworks, oeuvres and artistic identities online. Within the rhizomatic space of the Internet, I source and montage various digital examples of Romantic landscape painting into new mashed Romantic images that become source material for my paintings. Here I find synergy between the Post Enlightenment definition of the sublime and a contemporary fascination in the digital sublime - of which the Internet is champion. The project interrogates the Internet in a new and promiscuous way. I use the Internet to identify and disseminate what was once only trusted to the canonical archives of printed literature - the narrative of history