This paper presents a novel surface modelling approach by utilising a 3D motion capture system. For designing a large-sized surface, a network of splines is initially set up. Artists or designers wearing motion markers on their hands can then change shapes of the splines with their hands. Literarily they can move their bodies freely to any positions to perform their tasks. They can also move their hands in 3D free space to detail surface characteristics by their gestures. All their design motions are recorded in the motion capturing system and transferred into 3D curves and surfaces correspondingly. This paper reports this novel surface design method and some case studies

Kang, J

Prieto, PA

Qin, SF

Wright, DK

Proceedings of the Institution of Mechanical Engineers Part B Journal of Engineering Manufacture

English

Brunel University Research Archive

 1 PROC. OF IMECHE,  PART B, JOURNAL OF ENGINEERING MANUFACTURE, VOL 220(2), PP. 335-339.  2006.  USE OF 3D BODY MOTION TO FREEFORM SURFACE DESIGN Shengfeng Qin, David K Wright, Jinsheng Kang, P A  Prieto School of Engineering and Design, Brunel University, Uxbridge, Middlesex, UB8 3PH   ABSTRACT  This paper presents a novel surface modelling approach by utilising a 3D motion capture system. For designing a large-sized surface, a network of splines is initially set up. Artists or designers wearing motion markers on their hands can then change shapes of the splines with their hands. Literarily they can move their bodies freely to any positions to perform their tasks. They can also move their hands in 3D free space to detail surface characteristics by their gestures.  All their design motions are recorded in the motion capturing system and transferred into 3D curves and surfaces correspondingly. This paper reports this novel surface design method and some case studies.  Key words: Motion Capture, Body Gestures, Freeform Surface Design, Conceptual design  1. INTRODUCTION  Large-sized free-form surface design presents some challenges in practice.  Traditionally at the conceptual design stage, product form-design and making mock-ups are key activities for designers to present and compare ideas, and sculpting physical models is still essential for surface development. These sculpted physical models can be then scanned and converted into CAD models. However, to physically modify the mock-up models is time consuming, and if the physical models are too big, designers may have problems in finding a suitable position to conduct their operations or simply the models are difficult to be scanned in.  2 Constructing freeform surfaces is essential in product form design (aesthetic design and industrial styling). Especially at the conceptual design stage, various design ideas need to be explored and quickly evaluated by prototyping. Use of computer aided design (CAD) models is a very effective way to demonstrate design ideas, illustrate surface and form design details with various renderings, and communicate the design within design teams geographically distributed at different sites.  On the other hand, CAD models are eventually needed for downstream manufacturing applications. However, creation of CAD models is not a simple job for industrial designers because current CAD systems require accurate 3D inputs for surfaces such as a set of 3D characteristic curves or a range of 3D points.   Traditionally, in the early product form-design stage, idea development and making mock-ups are key activities for designers. Soft prototypes made from easily carved or sculpted materials are useful for the more complicated concepts. However, these soft prototypes need to be transferred to CAD models. Currently, reverse engineering (RE) technique [1] is widely used for obtaining 3D CAD models from real 3D physical prototypes (models). It is designed for one-off applications, but is not good for rapid conceptual design applications because they require creation of CAD models rapidly. An alternative technique has been investigated and developed to support rapid CAD model updating from the corresponding sculpted physical models [2]. But it can only support small-sized models (dimensions like a computer mouse or an engine block). For a large-sized design object such as a car body, if being scaled down too much, its surface curvature will change noticeably. Consequentially, the scaled small prototypes cannot represent true design ideas and intent forms.  On the other hand, research on sketch-based conceptual design has being carried out in 2D sketch recognition [3-6] and 3D virtual sketching [7-10]. Styling or industrial designers start a free-form  3 surface design by drawing characteristic curves which describe the basic product shape characteristics with pencil and paper. The curve mesh may be regular or irregular. This is a very effective way to design a shape intuitively. These idealised 2D sketches are then transferred into 3D surface models based on design feature recognition [3], and design gestures [5,6]. However, the primary difficulty in transferring 2D sketches into 3D surfaces is that an inverse projection should be performed from the sketch plane to the 3D space. This process is mathematically indeterminate. Therefore, research on direct 3D sketching and modelling has been investigated alternatively [7-10].   A 3D digitiser (the sensor or ‘pen’) based on a digitising tablet was used in [7] to simulate the action of free sketching. During the sketching phase, a user operated the devices solely by contacting with the ‘pen’ which contains the enabling switch. Free sketching movements were made in space without the need for a solid reference model. The sketch was viewed on a 2D monitor.  The drawback of the research is that it provides only 2D image as feedback. Early work in 3D shape creation with 3D input devices was explored in 3-Draw [8]. In 3-Draw, the user holds a mirroe-like plate in the left hand to rotate the whole screen display, and holds a 3D stylus in the right hand to sketch curves in 3D space.  The 3D sketches are displayed on a 2D screen. Deering followed up on this work with HoloSketch [9], which allows the drawings of tubes of toothpaste by moving a tracked stylus with a glove through 3D space.  In order to provide 3D visual feedback, Surface Drawing [10] was developed to support drawing with the hand in free space. When the hand is moved through space, its path takes forms and hovers in the air as surface. This concept was realized using the Responsive Workbench [11], a large table that acts as a display surface.  Special interface hardware includes a head-tracked stereoscopic display and sensors which track the body and handheld tools allowing the artist to share the space of the artwork.  Even with the stereoscopic display, Surface Drawing found that it has difficulty in making strokes touch  4 (or intersected) due to both the complexity of 3D stroke boundaries and the noticeable error of the tracker system. Therefore, instead of using boundary curves, it uses surface strips and merges them together to form rough surfaces. It is not suitable for engineering applications. Furthermore, this work is limited by the size of the Responsive Workbench.   2. NEW METHOD  In order to support large-sized freeform surface design at the conceptual stage, we investigated a novel surface modelling approach by utilising a 3D motion capture system. Artists or designers wearing motion marks on their hands can sketch out 3D design splines when moving their body and hand in space. Literarily they can combine their body and hands movements freely to any positions within a predefined motion capturing volume to perform their design tasks. During sketching, the designers can use the working ground as the X and Y directional references and their body parts for Z directional reference. They can also use rough physical wireframes as 3D references. All their design motions are recorded in the motion capturing system and transferred into 3D curves correspondingly. Based on sketched 3D curves, a large-sized freeform surface can be obtained easily. Obviously, various body gestures and biomedical signals have a potential to be incorporated into the design process. For example, motion velocity and acceleration can be used to estimate surface curvature changes and further smooth the surface. On the other hand, some designers or artists may have mobility problems in their hands, but they still can conduct their creative design activity through their arm motion or even body motion.   5 This motion-based 3D surface design process includes four phases: design scene setting up, surface design by motion capturing, design data processing, 3D curve and surface generation. It has been shown in Figure 1.                                Figure  2.  Design scene   2.1 Design Scene Setting Up    The concept of surface design with motion was realized using a motion capturing system by MotionAnalysis Co.[12]. It is a 3D optical motion capturing system that measures the movement of objects. The system combines proprietary hardware, software and electro-optical techniques with standard computer and video hardware. In our system, 7 Eagle digital cameras are used with the EVaRT 4.4 software to collect motion data.  The 7 cameras can be located circularly to let the system to capture body motions within a rough volume about 5m X 8m X 4m along the x, y directions on the ground and the z direction vertically.  This volume can support larger sized surface design activities. Motion is captured from reflective markers. The size of marks varies from 6 mm to 25 mm in diameter. Different sizes of markers will affect the effective size of the volume. After setting up the motion capturing system, a design scene with various 3D references can be set up as well in the centre of the capturing Design Scene Setting Up Surface Design by Motion Capturing Design Data Processing 3D Curve and Surface Generation Start Figure 1. Design process   6 region. For example, some wood spline strips had been set up for designing a car body (Fig. 2). The design scene can provide 3D physical references for designers. These references are visible, touchable and pressure-sensible.  2.2 Surface Design by Motion Capturing  For designing a surface by motion capturing, designers need to wear reflective markers (sensors) on their hands. If wearing markers on both hands, they can use them both for design, making more body gestures available. If they only touch sensors on one hand, they can use the other hand to hold design tools such as rulers to assist the design. Normally, one marker on one hand is enough. It can be typically attached to the tip of a finger. During the design process, the designers can control the markers in function or not by covering the markers with their thumbs. When they move their bodies with stretched hands (markers are in function), the movements of the markers against timing are captured and recorded in a data file. When they change their positions with covered markers, no motion will be captured because the markers are not in function. For example, the designer in Fig 2 used only one sensor on his right hand. The design motion was recorded in a Track Row Column (.TRC) file, which contains X-Y-Z position data for the reflective marker. The file is in the ASCII format that can be easily read into a spreadsheet program such as ExcelTM. The position data for each marker is organized into 3 columns per marker (X, Y and Z position) with each row being a new frame. The position data is relative to the global coordinate system of the capture volume and the position values are in the same units as used for calibration. Missing data is represented as an empty frame of position data.  2.3 Design Data Processing   With obtained design motion, we first need to segment the raw motion data into 3D curves. This task is performed within our MATLAB programs. Each segment in motion data represents a 3D design curve.  7 For each curve segment, it is then filtered for smoothly fitting with a NURBS curve. Collectively, these design curves are characteristic lines on the intent surface. Finally, the 3D design curves are imported into a commercial CAD package -Alias Studio for constructing the surface.  Segmentation: In a .TRC file, each design curve is represented as consecutive position data, which is separated by missing data (empty frames of position data). Therefore, we initially segment design curves by finding empty frames in the motion file and then refine the segmentation by checking if two successive segments are real separated. Based on our observation, when designers want to separate two curves, they simply use their hands or fingers to cove the reflective markers, move them to a new position and uncover them again. This action takes about 2 seconds or more.  If the designers just want to pause and think for a while, they usually hold their hand positions (the markers may be covered or not). Therefore, if the time interval between the two successive segments is larger than 2 seconds and the distance between their closest end points is bigger than a threshold such as 50 mm, the two segments are two real curves.  If not, the two segments should be merged together to form one curve because a design curve might be separated due to body blockage of views. This threshold distance can be calculated as a product of 2 seconds and the average motion speed over the current curve.  Filter:  After the segmentation, a design curve is represented by a great number of 3D points. By nature of motion capture, the number of points is more than enough to create a smooth curve. Therefore, these raw points have to be filtered. While filtering, two end points of a curve are kept and then from one end to the other, points within a distance threshold such as 50-100 mm will be filtered out. As a result, the remaining points represent a smooth curve. This threshold can be varied with different sizes of models and design requirements. If its value is too big, some details of curves may be missed out; if it is too  8 small, the smoothness of the curves may be reduced. The threshold also depends sizes of models. In general, the bigger the sizes are, the bigger the value is.   For conceptual design in large sized models, designers focus much more on 3D forms (overall shapes) rather than small details. Thus, a bigger value such as 100mm can be used.  2.4 3D Curve and Surface Generation Curve and surface construction: When the filtering process is finished, resultant points for each curve are fitted with a NURBS curve and it is then imported into a CAD package, e.g., Alias Studio. In the CAD package, curves can be further treated for constructing a large-sized surface. In Alias, all these imported 3D curves are viewed in three orthogonal view planes: Front, Side and Top. Based on different views, some curves are extended to intersect with some others. After that, relevant curves are reparameterised and rebuilt to make them compatible because original points on each curve may not be compatible. Finally, a new network of curves is formed and used to construct a surface model. 3. RESULTS  The proposed novel design method for modeling larger-sized freeform surfaces has been tested with examples. For instance, we set up the design scene (Fig.2) to demonstrate a large conceptual surface design. The initial motion data was recorded in a .TRC file. After segmentation, each curve was displayed in Fig. 3. Figure 3a shows the size of the intent surface: about 4m long X 2m wide X 1.5m high. It is a large-sized object. The segmented curves contain all raw motion data. Thus, they were further filtered to make them smooth with a distance threshold of 100mm. From this filtering process, the number of points for each curve was reduced by about 20%. The new curve data (points) were then fitted with NURBS curves and the curves were imported into Alias Studio through its .OBJ file. In Alias Studio, the input curves were further modified to form a network for surface construction (Fig. 3b).  9 Figure 3c shows the resulting surface with its rendering model. This model can be used for design evaluation.                      (a)                                                  (b)                                                   (c) Figure 3 4. DISCUSSION  The proposed method is based on 3D body motion capturing. It is difficult for designers to generate a characteristic curve with well-defined connections. For example, end-points of related curves should meet together. Therefore, a 3D tidy-up process (or beautification) to connect curves correctly and identify design intents such as symmetry may be needed before exporting curves into a commercial CAD package. On the other hand, it is necessary to modify imported curves in the CAD package because current CAD systems cannot support surface constructions from arbitrary curve networks. Otherwise, the imported curves can be directly used for surface modeling. After discussing with designers, it was found that the isometric or perspective display of captured 3D motion from the EVaRT motion capturing software can be used for 3D visual feedback if the display is projected to the wall (or a big flat surface) through a LCD projector.   10 5. CONCLUSIONS  Large-sized free-form surface design faces some challenges in practice. We investigated a novel surface modeling approach by utilising a 3D motion capture system. For designing a large-sized surface, a network of splines is initially set up. Artists or designers wearing reflective markers on their hands can then change shapes of the splines with their hands. Literarily they can move their bodies freely to any positions to perform their tasks. They can also move their hands in 3D free space to detail surface characteristics by their gestures.  All their design motions are recorded in the motion capturing system and transferred into 3D curves and surfaces correspondingly. From the experimental tests, it is shown that the method is useful and practical for designing large-sized freeform surfaces.  REFERENCES  [1] Varady T, Martin RR, Cox J, Reverse engineering of geometric models - An introduction. Computer Aided Design 1997, 29 (4): 255-268. [2] Prieto PA, Wright DK, and Qin SF, A novel method for early formal developments using CAD and RP technology. Proceedings of the Institution of Mechanical Engineering Part B, Journal of Engineering Manufacture 2003, 217: 695-698. [3] Qin SF, Wright DK, Jordanov I N. From on-line Sketch to 2D and 3D geometry: A System based on fuzzy knowledge. Computer-Aided Design 2000, 32(14): 851-866. [4] Qin SF, Wright DK, Jordanov IN. A conceptual design tool: a sketch and fuzzy logic based system. Proceedings of the Institution of Mechanical Engineering Part B, Journal of Engineering Manufacture 2001, 215: 111-116.  11 [5] Zeleznik RC, Herndon KP, Hughes JF. SKETCH: an interface for sketching 3D scenes. ACM SIGGRAPH Computer Graphics Proceedings, Annual Conference Series, 1996: 163-170. [6]  Igarashi T, Matsuoka S, Tanaka H. Teddy: a sketching interface for 3D freeform design. ACM SIGGRAPH Computer Graphics proceedings, Annual Conference Series 1999: 409-416. [7] Poletti H. A 3D sketching tool: An MA computing in design research project. Computer Graphics 1995, 25-27. [8] Sachs E, Roberts A, Stoops D. 3-Draw: A tool for designing 3D shapes. IEEE Computer Graphics & Applications 1991, 11(6): 18-25. [9] Deering MF. HoloSketch: A virtual reality sketching/animation tool. ACM Transactions on Computer-Human Interaction 1995, 2,3:220-238. [10] Schkolne S, Pruett M, Schröder P. Surface Drawing: Creating Organic 3D Shapes with the Hand and Tangible Tools. Proceedings of ACM SIGCHI’01  2001, 261-268. [11] Arnheim R. Visual Thinking. University of California Press, 1969. [12] http://www.motionanalysis.com, 2005.  

3-Draw: A tool for designing 3D shapes.

A 3D sketching tool: An MA computing in design research project. Computer Graphics

A conceptual design tool: a sketch and fuzzy logic based system.

A novel method for early formal developments using CAD and RP technology.

a sketching interface for 3D freeform design.

From on-line Sketch to 2D and 3D geometry: A System based on fuzzy knowledge. Computer-Aided Design

HoloSketch: A virtual reality sketching/animation tool.

Reverse engineering of geometric models - An introduction. Computer Aided Design

SKETCH: an interface for sketching 3D scenes.

Surface Drawing: Creating Organic 3D Shapes with the Hand and Tangible Tools.

Visual Thinking.

Use of 3D body motion to freeform surface design

Use of 3D body motion to freeform surface design

Abstract

Similar works

Full text

Available Versions

Brunel University Research Archive