Caricature is an illustration of a person or a subject that uses a way of exaggerating the most distinguishable characteristic traits and simplifying the common features in order to magnify the unique features of the subject. Recently, automatic caricature generation has become a research area due to the advantageous features of amusement in the fields such as network, communications, online games, and the animation industry. The aim of this study is to present a perceptual caricaturization approach practicing the concept of exaggeration, which is very common in traditional art and caricature, on 3D mesh models synthesizing the idea of mesh saliency. © 2013 Springer-Verlag London

Bulbul, A.

Capin, T.

Cimen G.

Ozguc, B.

Bilkent University Institutional Repository

Perceptual Caricaturization of 3D ModelsGokcen Cimen, Abdullah Bulbul, Bulent Ozguc and Tolga CapinAbstract Caricature is an illustration of a person or a subject that uses a way ofexaggerating the most distinguishable characteristic traits and simplifying the com-mon features in order to magnify the unique features of the subject. Recently, auto-matic caricature generation has become a research area due to the advantageousfeatures of amusement in the fields such as network, communications, online games,and the animation industry. The aim of this study is to present a perceptual carica-turization approach practicing the concept of exaggeration, which is very commonin traditional art and caricature, on 3D mesh models synthesizing the idea of meshsaliency.1 IntroductionCaricature is a visual art that greatly exaggerates certain features of a subject to createa comic effect. The aim of the caricature is to find individual differences and moreattractive features than other parts of the subject and exaggerate these features tocreate humorous look.To automatically generate a caricature, a system should attempt to imitate the tra-ditional artistic skills. Identification of the unique features is essential for creatingcaricatures since these features will eventually be exaggerated. To accomplish thischallenge, we employ the idea of mesh saliency, proposed by Lee et al. 2005 [7],which is a technique to measure regional importance of 3D meshes. After extract-ing salient parts of the mesh feature points, the caricature is generated by warpingthe original 3D model from original feature points to saliency based exaggeratedfeature points by using a deformation technique. The deformation of 3D meshes ismaintained by Free Form Deformation (FFD) method which is a common methodG. Cimen (B)Bilkent University, 06800 Ankara, Turkeye-mail: gokcen.cimen@cs.bilkent.edu.trE. Gelenbe and R. Lent (eds.), Computer and Information Sciences III, 201DOI: 10.1007/978-1-4471-4594-3_21, © Springer-Verlag London 2013202 G. Cimen et al.used for mesh deformation [11]. With automatically calculated saliency values anddeformation through these values, the method is fully automatic, but the user candetermine the scale of feature exaggeration.Among existing methods, most of them are based on manipulating 2D facial pho-tographs or images. However, one major drawback of 2D facial caricaturization isthat they are limited to make exaggeration with one missing dimension so that everyfeature deformed on a face is on the same plane. This creates pixel-based deforma-tion, so results undesired distortion artifacts by pulling or pushing adjacent pixels. Byperforming the caricaturization in 3D space, we have successfully overcome theseproblems. With respect to this, in this study, we present a perceptual approach apply-ing the exaggeration by using automatically perceptual based feature extraction.The rest of this paper is organized as follows. In the next section, related workis discussed. In Sect. 3, the overview of the proposed framework is given andthe steps of the technique are explained. The results and the conclusions arepresented in Sect. 4.2 Related WorksSusan Brennan’s work is perhaps the milestone in formally defining the first caricaturegeneration system in 1982 [3]. After Brennan’s work, several cartoon caricaturegeneration methods have been proposed with or without a training process [10]. Thebasic idea of these approaches is exaggerating the difference from the mean face,but forming a mean face requires large databases of random faces with hundreds offeatures identified in everyone.Another proposed approach is training the computer to imitate a specific draw-ing style from examples. Based on a set of training images and their relatedhand drawn sketches by an artist, Chen et al. [4]’s method automatically gener-ates an artist’s styli tic caricature from an input image. In another study, Lianget al. [8]’s work attempts to learn the style of a caricature artist from exam-ple caricatures drawn by the artist using partial least squares. Besides, a recentstudy managed to generate digital caricatures from facial photographs that cap-ture artistic deformation styles from hand-drawn caricatures by Clarke et al. [5].Even if this approach observes and learns from the artist’s products, they needartist’s hand drawn style and can only get the copy of the style and results withlimited success.There have been a number of studies to interactively generate facial caricatures.Akleman [1, 2]. However, the limitation of these techniques is that they requireprior knowledge of how a face would be deformed to generate caricature andthey are designed for artists or art students. In addition, recently Fu et al. [6]proposed an interactive system that decomposes various facial components of a3D head model and replace these components with other head and facial com-ponents stored in a database. Since these method does not make use of a ref-Perceptual Caricaturization of 3D Models 203Fig. 1 Overview of the systemFig. 2 Saliency map for three different 3D models. Green regions are more salient than bluesregions, so they are far more deformederence head, the resulting caricature are lack of the likeliness to the originalmodel.3 Automatic Caricature GenerationOur framework, shown in Fig. 1, includes two main components—mesh saliencycalculation and free form deformation. In the saliency calculation step, visuallyimportant regions are determined and in the free-form deformation step these regionsare enhanced to obtain the caricaturized style.3.1 Mesh Saliency CalculationSaliency is the property of objects that attracts attention based on the difference ofthe object from its surroundings. In Perceptual Caricaturization of 3D Models, forthe mesh saliency calculation, a center-surround mechanism proposed by Lee [7],which is a perceptual approach, is applied to determine salient parts of an object.Figure 2 shows results with colorized saliency map of three different models.204 G. Cimen et al.Mesh saliency calculation process as follows:• The first step of saliency calculation involves computing surface curvatures. Tocompute the curvature of a mesh at a vertex v, the approach of Meyer [9] is applied.• The Gaussian-weighted average of the mean curvature of a vertex is calculatedwith the Eq. (1).G(C(v), s) =xεN (v,2s)C(x) exp[−||x−v||2/(2s2)]xεN (v,2s)exp[−||x−v||2/(2s2)](1)C(v) denotes the mean curvature of vertex v and N(v, 2s) is the neighborhoodfunction which gives the set of points within the distance s for a vertex v.• Then the computation of the saliency S(v) of a vertex v is calculated by takingthe absolute difference between the Gaussian-weighted average computed at fine(small) and coarse (large) scales in Eq. (2)Si (v) = |G(C(v), si ) − G(C(v), 2si )| (2)3.2 Free Form DeformationFundamentally, FFD is a kind of deformation of an object that changes its originallymodeled appearance which is known as the object’s rest state to other state which isknown as deformed state.To define the lattice space which can be called bounding box in free form deformationtechnique, bezier volumes are used. Bezier volume defines a volume with the controlpoints within it and can be freely altered to deform the object. Equation (3) showsthe calculation for the Bezier volume.Q(u, v, w) = 3i=03j=03k=0 Pijk Bi,3(u)Bi,3(v)Bi,3(w) (3)3.3 IntegrationAt this stage, the pre-calculated saliency values of the 3D object are mapped to thecontrol points of the bounding box surrounding the object. Then the control pointsare extended from an estimated pivot points according to new calculated saliencyvalues. To accomplish this, the algorithm works in three steps:1. The Eq. (4) calculate the saliency value for one control point by computting theweighted average of the saliency values (sv(i)) of the vertices of the object withinan estimated distance threshold.Perceptual Caricaturization of 3D Models 205Sc( j) =mi=0S′v(i)d(P ′v(i)Pc( j))2∗ (max(d) − min(d))2m(4)In the equation, Pc(j) gives the position of the control points. S′v(i) is the saliencyvalues and P ′v(i) are the positions of the objects vertices under the estimateddistance threshold. If the distance between the control point and vertex is smallerthan the threshold, the saliency value of this vertex is taken into account forcalculation of the saliency value of the control point.P ′v(i) = {Pv(i)|d(Pv(i); Pc(j)) < distThres}S′v(i) = {Sv(i)|d(Pv(i); Pc(j)) < distThres}2. Pivot points are calculated to determine the exaggeration direction of the controlpoints. Equation (5) shows the calculation of the pivot point position for onecontrol point.Vc( j) = ni=0P ′v(i)n(5)3. After saliency values of all control points are calculated with the Eq. (4) and virtualpivot points of them are computed with the Eq. (5), Eq. (6) gives the calculationfor the new positions of the one control point.P ′c( j) =Fd ∗ (Pc( j) − Vc( j)) ∗ Sc( j)max(Sc) − min(Sc) (6)In the equation, Fd is the deformation factor which is the only parameter thatcan be changed by user is used to determine the scale of the exaggeration of thecaricaturized 3D model.4 Results and ConclusionFigures 3 and 4 show the results of our technique on different 3D models. Eachfigure starts with the saliency colorization of the models and through (b) to (d),the deformation factor Fd is incremented respectively. As the deformation factor isincreased, the salient parts of the models are far more deformed and results in moreexaggerated caricaturized models.In the Perceptual Caricaturization of 3D Models, we have implemented the exag-geration concept by using free form deformation technique with respect to the meshsaliency by trying to extract perceptually more attractive features of the 3D objectsto imitate an artist thinking when drawing caricature. This technique can be extendedto generate multiple caricatures which can be employed in caricature animation andnon-photorealistic animation in the future.206 G. Cimen et al.(a) (b) (c) (d)(e) (f) (g) (h)(h) (i) (j) (k)Fig. 3 Image a shows saliency. Images from a to d shows deformation at different deformationfactors(a) (b) (c)Fig. 4 Images from (a) to (c) shows deformation at different deformation factorsAcknowledgments We thank Ufuk Celikcan and Bengu Kevinc for their support during our study.This work is supported by the Scientific and Technical Research Council of Turkey (TUBITAK,Project number: 110E029).Perceptual Caricaturization of 3D Models 207References1. Akleman, E.: Making caricatures with morphing. In: ACM SIGGRAPH 97 Visual Proceedings:The Art and Interdisciplinary Programs of SIGGRAPH’97. SIGGRAPH’97, p. 145 (1997)2. Akleman, E., Reisch, J.: Modeling expressive 3D caricatures. In: ACM SIGGRAPH 2004Sketches. SIGGRAPH ’04, p. 61 (2004)3. Brennan, S.E.: Caricature the generator: faces dynamic exaggeration of by computer. Leonardo18, 170–178 (1985)4. Chen, H., Xu, Y.Q., Shum, H.Y., Zhu, S.C., Zheng, N.N.: Example-based facial sketch gener-ation with non-parametric sampling. In: ICCV ’01, pp. 433–438 (2001)5. Clarke, L., Chen, M., Mora, B.: Automatic generation of 3D caricatures based on artis-tic deformation styles. Vis. Comput. Graph. IEEE Trans. 17(6), 808–821 (2011). doi:10.1109/TVCG.2010.766. Fu, G., Chen, Y., Liu, J., Zhou, J., Li, P.: Interactive expressive 3D caricatures design. In:Proceedings of Conference IEEE Multimedia and Expo, pp. 965–968 (2008)7. Lee, C.H., Varshney, A., Jacobs, D.: Mesh saliency. ACM transactions on graphics. Proc.SIGGRAPH 24(3), 659–666 (2005)8. Liang, L., Chen, H., Xu, Y.Q., Shum, H.Y.: Example-based caricature generation with exag-geration. In: 10th Pacific Conference on Computer Graphics and Applications PG’02, p. 386(2002)9. Meyer, M., Desbrun, M., Schroder, P., Barr, A.H.: Discrete differential-geometry operators fortriangulated 2-manifolds. Vis. Math. III 3(7), 35–57 (2003)10. Mo, Z., Lewis, J.P., Neumann, U.: Improved automatic caricature by feature normalization andexaggeration. In: Proceedings ACM SIGGRAPH 2004 Sketches, p. 57 (2004)11. Sederberg, T.W., Parry, S.R.: Free-form deformation of solid geometric models. Proc. ACMSIGGRAPH 20, 151–160 (1986)

Perceptual caricaturization of 3D models

http://repository.bilkent.edu.tr/bitstream/11693/27941/1/Perceptual%20caricaturization%20of%203D%20models.pdf

Perceptual caricaturization of 3D models

Abstract

Similar works

Full text

Available Versions

Bilkent University Institutional Repository