Painterly rendering techniques: A state-of-the-art review of current approaches

In this publication we will look at the different methods presented over the past few decades which attempt to recreate digital paintings. While previous surveys concentrate on the broader subject of non-photorealistic rendering, the focus of this paper is firmly placed on painterly rendering techniques. We compare different methods used to produce different output painting styles such as abstract, colour pencil, watercolour, oriental, oil and pastel. Whereas some methods demand a high level of interaction using a skilled artist, others require simple parameters provided by a user with little or no artistic experience. Many methods attempt to provide more automation with the use of varying forms of reference data. This reference data can range from still photographs, video, 3D polygonal meshes or even 3D point clouds. The techniques presented here endeavour to provide tools and styles that are not traditionally available to an artist. Copyright © 2012 John Wiley & Sons, Ltd

Art Directed Shader for Real Time Rendering - Interactive 3D Painting

In this work, I develop an approach to include Global Illumination (GI) effects in non-photorealistic real-time rendering; real-time rendering is one of the main areas of focus in the gaming industry and the booming virtual reality(VR) and augmented reality(AR) industries. My approach is based on adapting the Barycentric shader to create a wide variety of painting effects. This shader helps achieve the look of a 2D painting in an interactively rendered 3D scene. The shader accommodates robust computation to obtain artistic reflection and refraction. My contributions can be summarized as follows: Development of a generalized Barycentric shader that can provide artistic control, integration of this generalized Barycentric shader into an interactive ray tracer, and interactive rendering of a 3D scene that closely represent the reference painting

Chinese Ink-and-Brush Painting with Film Lighting Aesthetics in 3D Computer Graphics

This thesis explores the topic of recreating Chinese ink-and-brush painting in 3D computer graphics and introducing film lighting aesthetics into the result. The method is primarily based on non-photorealistic shader development and digital compositing. The goal of this research is to study how to bing the visual aesthetics of Chinese ink-and-brush painting into 3D computer graphics as well as explore the artistic possibility of using film lighting principles in Chinese painting for visual story telling by using 3D computer graphics. In this research, we use the Jiangnan water country paintings by renowned contemporary Chinese artist Yang Ming-Yi as our primary visual reference. An analysis of the paintings is performed to study the visual characteristics of Yang's paintings. These include how the artist expresses shading, forms, shadow, reflection and compositing principles, which will be used as the guidelines for recreating the painting in computer graphics. 3D meshes are used to represent the subjects in the painting like houses, boats and water. Then procedural non-photorealistic shaders are developed and applied on 3D meshes to give the models an ink-look. Additionally, different types of 3D data are organized and rendered into different layers, which include shading, depth, and geometric information. Those layers are then composed together by using 2D image processing algorithms with custom artistic controls to achieve a more natural-looking ink-painting result. As a result, a short animation of Chinese ink-and-brush painting in 3D computer graphics will be created in which the same environment is rendered with different lighting designs to demonstrate the artistic intention

Natural Calligraphy

The term ‘natural calligraphy’ is introduced to describe a class of dynamic lines found in nature. These lines are well-defined although sometimes short-lived. They occur in different natural processes and on different scales. A selection is presented in this paper so as to better understand the class. They possess, for instance, cusps in density, involved topology and sweeping curves. Their occurrence is sometimes surprising because diffusive processes lead to smoothening of densities, the weakening of strong gradients, and the erasure of edges and sharp boundaries. As they change over time, forms of the line - ‘characters’ - emerge that seem particularly striking but these can disappear as quickly as they appear. As these characters evolve from and to less remarkable forms, we have a better chance of understanding the characteristics that make them appear compelling or beautiful. The lines exist in higher-dimensional spaces and we present two-dimensional projections of these spaces (photographs and images from simulations) as illustrations in this paper. Features that we describe as striking are often a result of this projection so that the natural form and the viewer both play a part in creating a ‘work of art’. We distinguish two types of experiment that we can conduct simultaneously in natural calligraphy. First, there is the scientific study of the natural processes creating the line. These processes possess both predictable and random (chaotic) elements. They are therefore not strictly repeatable although some features of the process are robust. Second, there is our placement in, and subjective selection from, the process as artists. We discuss these two types of experiment in more detail in relation to our examples. The exciting possibility suggested by these natural processes is a dynamic evolving calligraphy with a continuum of forms and symbols. Modern developments in the concept of trajectory in science have enriched the simple idea of a line and the extremal principles at the heart of these developments underlie the aesthetic of natural calligraphy. This paper aims to give an introduction to these ideas that might be of interest to both to scientists and practitioners in the arts.Peer reviewedFinal Published versio

Ink-and-Ray: Bas-Relief Meshes for Adding Global Illumination Effects to Hand-Drawn Characters

We present a new approach for generating global illumination renderings of hand-drawn characters using only a small set of simple annotations. Our system exploits the concept of bas-relief sculptures, making it possible to generate 3D proxies suitable for rendering without requiring side-views or extensive user input. We formulate an optimization process that automatically constructs approximate geometry sufficient to evoke the impression of a consistent 3D shape. The resulting renders provide the richer stylization capabilities of 3D global illumination while still retaining the 2D handdrawn look-and-feel. We demonstrate our approach on a varied set of handdrawn images and animations, showing that even in comparison to ground truth renderings of full 3D objects, our bas-relief approximation is able to produce convincing global illumination effects, including self-shadowing, glossy reflections, and diffuse color bleeding

Physical pixels

Thesis (S.M.)--Massachusetts Institute of Technology, School of Architecture and Planning, Program in Media Arts and Sciences, 2000.Includes bibliographical references (leaves 48-51).The picture element, or pixel, is a conceptual unit of representation for digital information. Like all data structures of the computer, pixels are invisible and therefore require an output device to be seen. The physical unit of display, or physical pixel, can be any form that makes the pixel visible. Pixels are often represented as the electronically addressable phosphors of a video monitor, but the potential for different visualizations inspires the development of novel phenotypes. Four new systems of physical pixels are presented: Nami, Peano, the Digital Palette and 20/20 Refurbished. In each case, the combination of material, hardware and software design results in a unique visualization of computation. The chief contribution of this research is the articulation of a mode of artistic practice in which custom units of representation integrate physical and digital media to engender a new art.by Kelly Bowman Heaton.S.M

Designing parametric matter:Exploring adaptive material scale self-assembly through tuneable environments

3D designs can be created using generative processes, which can be transformed and adapted almost infinitely if they remain within their digital design software. For example, it is easy to alter a 3D object's colour, size, transparency, topology and geometry by adjusting values associated with those attributes. Significantly, these design processes can be seen as morphogenetic, where form is grown out of bottom-up logic’s and processes. However, when the designs created using these processes are fabricated using traditional manufacturing processes and materials they lose all of these abilities. For example, even the basic ability to change a shapes' size or colour is lost. This is partly because the relationships that govern the changes of a digital design are no longer present once fabricated. The motivating aim is: how can structures be grown and adapted throughout the fabrication processes using programmable self-assembly? In comparison the highly desirable attribute of physical adaptation and change is universally present within animals and biological processes. Various biological organisms and their systems (muscular or skeletal) can continually adapt to the world around them to meet changing demands across different ranges of time and to varying degrees. For example, a cuttlefish changes its skin colour and texture almost immediately to hide from predators. Muscles grow in response to exercise, and over longer time periods bones remodel and heal when broken, meaning biological structures can adapt to become more efficient at meeting regularly imposed demands. Emerging research is rethinking how digital designs are fabricated and the materials they are made from, leading to physically responsive and reconfigurable structures. This research establishes an interdisciplinary and novel methodology for building towards an adaptive design and fabrication system when utilising material scale computation process (e.g. self-assembly) within the fabrication process, which are guided by stimuli. In this context, adaption is the ability of a physical design (shape, pattern) to change its local material and or global properties, such as: shape, composition, texture and volume. Any changes to these properties are not predefined or constrained to set limits when subjected to environmental stimulus, (temperature, pH, magnetism, electrical current). Here, the stimulus is the fabrication mechanisms, which are governed and monitored by digital design tools. In doing so digital design tools will guide processes of material scale self-assembly and the resultant physical properties. The fabrication system is created through multiple experiments based on various material processes and platforms, from paint and additives, to ink diffusion and the mineral accretion process. A research through design methodology is used to develop the experiments, although the experiments by nature are explorative and incremental. Collectively they are a mixture of analogue and digital explorations, which establish principles and a method of how to grow physical designs, which can adapt based on digital augmentations by guiding material scale self-assembly. The results demonstrate that it is possible to grow physical 2D and 3D designs (shapes and patterns) that could have their properties tuned and adapted by creating tuneable environments to guide the mineral accretion process. Meaning, the desirable and dynamic traits of digital computational designs can be leveraged and extended the as they are made physical. Tuneable environments are developed and defined thought the series experiments within this thesis. Tuneable environments are not restricted to the mineral accretion process, as it is demonstrated how they can manipulate ink cloud patterns (liquid diffusion), which are less constrained in comparison to the mineral accretion process. This is possible due to the use of support mediums that dissipate energy and also contrast materially (they do not diffuse). Combining contrasting conditions (support mediums, resultant material effects) with the idea of tuneable environments reveals how: 1) material growth and properties can be monitored and 2) the possibilities of growing 3D designs using material scale self-assembly, which is not confined to a scaffold framework. The results and methodology highlight how tuneable environments can be applied to advance other areas of emerging research, such as altering environmental conditions during methods of additive manufacturing, such as, suspended deposition, rapid liquid printing, computed axial lithography or even some strategies of bioprinting. During the process, deposited materials and global properties could adapt because of changing conditions. Going further and combining it with the idea of contrasting mediums, this could lead to new types 3D holographic displays, which are grown and not restricted to scaffold frameworks. The results also point towards a potential future where buildings and infrastructure are part of a material ecosystem, which can share resources to meet fluctuating demands, such as, solar shading, traffic congestion, live loading