International audienceIn order to make virtual modeling as easy as real clay manipulation, we describe a realtime virtual clay model, specially designed for direct hand manipulation. We build on a previous layered model for clay, extending it to handle local properties such as colour or fluidity, to deal with an arbitrary number of tools, and to capture twist effects due to rotating tools. The resulting clay model is the first step towards a more long term goal, namely direct interaction through video tracking of the user's hands

Cani, Marie-Paule

Dewaele, Guillaume

Hal - Université Grenoble Alpes

Virtual Clay for Direct Hand Manipulation

INRIA a CCSD electronic archive server

HAL Id: inria-00537460https://hal.inria.fr/inria-00537460Submitted on 18 Nov 2010HAL is a multi-disciplinary open accessarchive for the deposit and dissemination of sci-entific research documents, whether they are pub-lished or not. The documents may come fromteaching and research institutions in France orabroad, or from public or private research centers.L’archive ouverte pluridisciplinaire HAL, estdestinée au dépôt et à la diffusion de documentsscientifiques de niveau recherche, publiés ou non,émanant des établissements d’enseignement et derecherche français ou étrangers, des laboratoirespublics ou privés.Virtual Clay for Direct Hand ManipulationGuillaume Dewaele, Marie-Paule CaniTo cite this version:Guillaume Dewaele, Marie-Paule Cani. Virtual Clay for Direct Hand Manipulation. Eurographics(short papers), Aug 2004, Grenoble, France. ￿inria-00537460￿EUROGRAPHICS 2004 / M. Alexa and E. Galin Short PresentationsVirtual Clay for Direct Hand ManipulationGuillaume Dewaele and Marie-Paule CaniGRAVIR/IMAG-INRIA dewaele|cani@imag.frAbstractIn order to make virtual modeling as easy as real clay manipulation, we describe a realtime virtualclay model, specially designed for direct hand manipulation. We build on a previous layered model forclay, extending it to handle local properties such as colour or fluidity, to deal with an arbitrary numberof tools, and to capture twist effects due to rotating tools. The resulting clay model is the first steptowards a more long term goal, namely direct interaction through video tracking of the user’s hands.1. IntroductionMany artists find clay modelling to be a simpler, moreintuitive way to design complex shapes than numericalshape modelling. We believe that a virtual clay modelcould be even more advantageous. While keeping anintuitive interaction, it would enable copy/paste andundo operations, avoid unwanted drying of the clay,enable changing physical properties, and allow thetools to scale as much as needed for easily modellingboth large scale and very fine features.We previously presented a volumetric model for vir-tual clay [DC03]. This was the first realtime model toenable global and local deformations mimicking thecentral features of real clay, namely plasticity, volumeconservation and surface tension. However, this modelsuffered from important limitations: the clay had to behomogeneous, only two tools could simultaneously in-teract with the clay, and only the tools translationswere taken into account during interaction.This paper presents an extension of the model thatenables local properties to the clay such as colour andfluidity, allows an arbitrary number of tools, and cap-tures twist effects due to rotation of the tools. Thesecontributions make the clay ready for direct hand ma-nipulation, as will be discussed below.1.1. Previous workVolumetric clay models: Both surfaces and volu-metric representations have been used for interactivesculpting (see [DC03] for a detailed state of the art).This section only reviews previous volumetric mod-els, which are closer in spirit to real-clay and easilycapture topological changes.Most volumetric models define the object as aniso-surface of a scalar field, either stored in a gridor controlled through primitives such as B-splinevolumes [IESQK99, PF01, RE00]. These models al-low volumetric operations such as adding materialor carving it. Local deformations were made possi-ble [FCG00], but with no control of volume varia-tions. Cellular automata were also used to computelocal deformation of virtual clay [ATTY99, DAB03] orsand [ON03]. These models allow free-form modellingwith volume conservation and topological changes.However, the user can’t perform large-scale deforma-tions such as bending the limbs of a model.Physically-based simulation is another option tomodel clay, viewed as a material that lies between plas-tic solids and viscous fluids. However, plastic-solids re-quire the addition of specific mechanisms for handlingtopological changes [MQW01]. Non-realtime, particle-based models were used to represent viscous mate-rial [Ton91] somewhat similar to clay. Realtime per-formances were achieved using Eulerian fluid simula-tions [Sta99], but this approach requires the fluid tooccupy the entire space and thus fails to describe clay,which is bounded by a time-varying surface.A layered model for virtual clay: The model pre-sented in this paper is an extension of that introducedin [DC03]. This was the first model to simulate in real-c© The Eurographics Association 2004.G. Dewaele & M.-P. Cani / Virtual Clay for Direct Hand Manipulationtime the main features of real clay, namely plasticity,volume conservation and surface tension. The clay sur-face is defined as the iso-surface at 0.5 of a scalar fieldmodelling the clay density. Field values are stored ina 3D grid and clamped between 0 (an empty cell) and1 (a cell full of clay). In addition to classical carvingor addition of material, the model captures local andglobal deformations which are expressed through claydisplacement from a grid cell to another. To our knowl-edge, there is no realtime physically-based model forvirtual clay . Three layers are used to achieve defor-mations, applied in turn at every time step:- The first layer simulates large-scale, plastic de-formations. It allows the user to bend parts of thesculpted model using two rigid tools, possibly one ofwhich freezes a region of the clay to keep it still. Thetranslation δ to apply to clay material lying in a givencell is computed as a linear combination:δ = (1− f)(k1δ1 + k2δ2) (1)of the two tools translations vectors δ1 and δ2. Thefluidity f scales the displacement field to take intoaccount that only a part of the motion of the toolis transmitted to clay when its fluidity is high. Theweights are defined as:k1 =1− d1−d2d122and k2 =1− d2−d1d122(2)where d1 and d2 are the respective pseudo-distancefrom the current cell to the tools, and d12 the pseudo-distance between them. The pseudo-distance modelsthe propagation of the quantity of movement insidea semi-fluid material. It can be seen as the length ofthe path, inside the object, along which the motionis transmitted. The longer this path is or the smallerthe clay density is along it, the smaller the generatedmotion is. The pseudo-distance is computed through apropagation scheme that starts from a tool and prop-agates in the clay until it reaches its border or cellscovered by other tools. For each non-empty cell ci:di = minneighbours(dj) +1ρi(3)where ρi is the density of clay in ci.- The second layer prevents volume variation by it-eratively poring clay in excess (i.e. clay in cells whichdensity value exceeds 1 or which are occupied by atool) into neighbouring cells. This results into intu-itive folds and local bulges when the user deforms ofpresses the clay.- The third layer mimics surface tension by mov-ing clay in cells where the density value is below 0.5towards the surface of the sculpted model. As a re-sult, clay does not spread into non-visible low-densityregions, and the object remains compact anytime.1.2. Discussion and overviewAlthough this model mimics clay behaviour in real-time, it suffers from three major limitations: first, theclay parameters, such as fluidity, are uniform acrossthe volume. The clay thus always behaves as a ho-mogeneous material, although in the real world, a de-signer often moistens specific regions in order to getlocally smaller scale deformations. Specifying differ-ent parameter values in each cell is not sufficient tosolve the problem: the values also have to be trans-ported with the clay and to be taken into accountwhen computing deformations. Secondly, although adesigner generally needs all his fingers for deform-ing real clay, the large-scale deformation layer is onlyable to handle interaction with two tools, due to thevery specific formula (2) used for computing the in-fluence of the tools. This makes the model non-usablein the context of direct-hand manipulation, which isour long term goal. Lastly, only the tools translationswere considered when computing large scale deforma-tions. Tool rotation is highly desirable, for instance tolocally twist the clay.The remainder of this paper presents an extension ofthe model that overcomes these limitations. Section 2shows how local properties (fluidity, colour...) can bedefined and transported when clay deforms. The influ-ence of a locally-defined fluidity on large-scale defor-mations is addressed. Section 3 generalizes the modelto handle the simultaneous interaction of an arbitrarynumber of tools. Section 4 extends it to take into ac-count tools rotations. Section 5 discusses the use ofthis model in the context of direct hand manipulation,which is the long-term goal we are trying to reach.2. Local properties of clayOur first goal is to be able to define local param-eters for the clay and adequately transport themduring deformations. This enables us to model non-homogeneous clay exhibiting local variations of fluid-ity, but may also serve for attaching any other localproperty to the clay. We illustrate this by introduc-ing a colour parameter. Depending on the needs, onecould also think of attaching temperature or surfaceroughness values to the clay, or of using local parame-ters to tune the effects of the volume preservation andsurface tension layers.Adding a new local property is made straight-forwards by our volumetric representation: we juststore the extra parameter values in each non-emptygrid cell. Classical 3D painting tools are used to in-crease or decrease the values in the clay volume. Themain concern is how to adequately attach the localproperties to the clay when it moves and deforms.c© The Eurographics Association 2004.G. Dewaele & M.-P. Cani / Virtual Clay for Direct Hand Manipulation2.1. Updating local propertiesA local property should be linked to the clay materialin a cell rather than to its specific position in space.We thus have to update cell parameter values eachtime some clay moves due to the action of one of thelayers. Although clay motion is modelled by increasingthe density value in a destination cell while decreasingit in a source cell, the values of local properties onlyhave to be updated in the destination cell, since thematerial’s nature in the source cell remains the same.Let ρi represent the amount of clay being trans-ported to the destination cell and νi be the vectorof associated local properties. Let ρj and νj respec-tively be the quantity of clay and its parameters al-ready stored in the destination cell. Then the naturalchoice for computing the new values of local proper-ties associated to the quantity of clay ρi + ρj in thedestination cell is the weighted average:νdestination =ρjνj + ρiνiρj + ρi(4)For instance, in the case of fluidity, ν represents theproportion of water in the clay. The new proportionis indeed the weighted average of the previous values.Another example is to attach colour parameters to theclay: a vector of three values respectively represent-ing the amount of red, green and blue is then storedin each cell. Deformations result into a quite naturalcolour blending effects, as depicted in Figure 1.Figure 1: Twisting coloured pieces of clay.2.2. Deforming non-homogeneous clayIn the previous model, fluidity f used in equa-tion citeeq:translation was considered uniformthroughout the clay. Clay with locally varying fluidityis modelled by setting an extra parameter in each cellin order to store the proportion of water contained. Inareas where the density of clay is low or where fluidityis high, the movement should propagate less froma cell to another. To achieve this, we still computethe pseudo-distance from a tool using the previousscheme, but we replace equation 3 by:di = minneighbours(dj) +1(1− fi)ρi(5)so that the ”distance” increases more quickly wheredensity ρi is low and fluidity f is high.Figure 2: Bending homogeneous clay (left) vs claywith locally-defined fluidity, depicted using colour in-dexes (right).3. Simultaneously applying multiple toolsInteracting with the clay using an arbitrary numberof tools is not a problem for the second and third lay-ers of the model, since they simply move clay if theamount of it in a cell is above of below given thresh-olds. However, the specific formula used for computingthe displacement field in the first layer (see equation 2)was restricting the previous model to the interactionwith no more than two tools.First, we can come up to an intuitive interpretationof the previously used formula: the parameters ki kindof partition the volume into voronoi regions attachedto the tools. But these regions have a ”soft border”between the tools where their respective actions areblended.Figure 3: Influence region for the red tool.We aim at defining similar regions of influence inthe case of several tools. In order to achieve this, foreach tool involved, we compute the weight coefficientski and displacement δ as:ki =1− di−minj(dj)minj(dij)2and δ =∑ikiδi∑iki(6)where j refers to all the other involved tools. As in thecase of two tools, this results into Voronoi-like regionsof influence, with a continuously varying effect of atool’s motion between them (see figure 3).Figure 4: Simultaneous action of four toolsc© The Eurographics Association 2004.G. Dewaele & M.-P. Cani / Virtual Clay for Direct Hand Manipulation4. Rotating toolsUp to this point, we only considered that tools mo-tion would be translations when they are in contactwith the clay. However, the user may also rotate tools,expecting to produce rotations or twists in the clay.The motion of a solid rotating object cannot be de-scribed by a simple displacement vector. Instead, apoint A rigidly linked to the tool moves according todisplacement field:δA = δO + OA× ω (7)where δO is the translation of point O (the center ofthe tool) and ω, the screw of the tool. To take intoaccount the rotation of the tool, we simply replacethe previous δi in equation (6) by the δA for cell iin equation (7). This way, more general deformationscan be generated. For instance, a bar of clay can betwisted by simply turning a tool at one end of the bar.Figure 5: Up: displacement field induced by rotatingtools. Bottom: bending and twisting a bar of clay5. Towards hand manipulation of clayThis paper presented a realtime clay model enablinglocal user-controlled variations of physical parameters,deformation through the simultaneous action of an ar-bitrary number of tools, and the use of tools rotationsduring interactions.This model is a first step towards a very challeng-ing long-term goal: enabling an artist to use his handsfor modelling virtual clay as he would with a real ma-terial. Of course, an appropriate interface is neededto capture hands and fingers motion. The most obvi-ous solution would be to use an haptic glove, possiblywith an exoskeleton for force feedback. We’re currentlystudying a less invasive approach, using a vision-basedinterface. The motion of the user’s hands, filmed by afew video cameras, will be used as an input to con-trol virtual hands serving as multiple tools. Howeverour current solution to hand tracking [DDH04] is notrealtime yet. We are also considering the user gettingforce-feedback by manipulating a real object (e.g. aball full of sand), the latter serving as an avatar forthe virtual clay.References[ATTY99] Arata H., Takai Y., Takai N. K., Ya-mamoto T.: Free-form shape modeling by 3dcellular automata. In Shape Modeling Interna-tional (1999), pp. 242–247.[DAB03] Druon S., A.Crosnier, Brigandat L.: Effi-cient cellular automata for 2d / 3d free-formmodeling. In WSCG (Feb. 2003).[DC03] Dewaele G., Cani M.-P.: Interactive globaland local deformations for virtual clay. In Pa-cific Graphics 2003 (2003).[DDH04] Dewaele G., Devernay F., Horaud R.:Hand motion from 3d point trajectories anda smooth surface model. In ECCV (2004).[FCG00] Ferley E., Cani M.-P., Gascuel J.-D.: Prac-tical volumetric sculpting. the Visual Com-puter 16, 8 (Dec. 2000), 469–480.[IESQK99] IX F. D., El-Sana J., Qin H., Kaufman A.:Haptic sculpting of dynamic surfaces. 1999ACM Symposium on Interactive 3D Graphics(April 1999), 103–110.[MQW01] McDonnell K. T., Qin H., WlodarczykR. A.: Virtual clay: A real-time sculpting sys-tem with haptic toolkits. 2001 ACM Sympo-sium on Interactive 3D Graphics (2001).[ON03] Onoue K., Nishita T.: Virtual sandbox. InPacific Graphics 2003 (2003), pp. 252–259.[PF01] Perry R. N., Frisken S. F.: Kizamu: A sys-tem for sculpting digital characters. Proceed-ings of SIGGRAPH 2001 (2001), 47–56.[RE00] Raviv A., Elber G.: Three-dimensionalfreeform sculpting via zero sets of scalartrivariate functions. Computer-Aided Design32, 8-9 (2000), 513–526.[Ros03] Rossignac J.: Finger sculpting with digi-tal clay: 3d shape input and output througha computer-controlled real surface. In ShapeModeling International (2003), pp. 229–231.[Sta99] Stam J.: Stable fluids. In Proceedings of SIG-GRAPH 99 (Aug. 1999), Computer GraphicsProceedings, pp. 121–128.[Ton91] Tonnesen D.: Modeling liquids and solids us-ing thermal particles. In Graphics Interface’91(Calgary, AL, June 1991), pp. 255–262.c© The Eurographics Association 2004.

https://hal.inria.fr/inria-00537460/document

Virtual Clay for Direct Hand Manipulation

Abstract

Similar works

Full text

Available Versions

Hal - Université Grenoble Alpes

INRIA a CCSD electronic archive server