With the increasing prevalence of powerful mobile technology, many applications involve human body measurements, such as online cloth shopping. Aiming at the application of non contact human body measurement and modeling system, this paper presents a new method for extracting human contours in complex background environment. Since the H component on the HSV color space is independent of the light changes, the hair and the lower garment can be divided. Therefore, a method of using the elliptical skin model on YCbCr color space is proposed to detect the skin color, then, automatically extract the skin samples to determine the threshold segmentation range. The combination of the two improves the skin color point recognition rate, gradually separating the body skin, clot and the hair by using these binary values of the linear fusion operation to get the final body contours. Our experiments demonstrates that this algorithm effectively reduces constraints on background environment when extracting contours

Tang, Wen

Wan, T. R.

Wang, L.

Wu, T.

Zhu, Y.

Bournemouth University Research Online

An Efficient Human Body Contour ExtractionMethod for Mobile AppsLi Wang1, Tao Ruan Wan2, Wen Tang3, Yao lin Zhu1, and Tong Wu11 College of Electronic and Information, Xian Polytechnic University, Xian, China2 Faculty of Informatics, University of Bradford, Bradford, UK3 Faculty of Science and Technology, Bournemouth University,Bournemouth, UKAbstract. With the increasing prevalence of powerful mobile technol-ogy, many applications involve human body measurements, such as on-line cloth shopping. Aiming at the application of non-contact humanbody measurement and modeling system, this paper presents a newmethod for extracting human contours in complex background environ-ment. Since the H component on the HSV color space is independent ofthe light changes, the hair and the lower garment can be divided. There-fore, a method of using the elliptical skin model on YCbCr color spaceis proposed to detect the skin color, then, automatically extract the skinsamples to determine the threshold segmentation range. The combina-tion of the two improves the skin color point recognition rate, graduallyseparating the body skin, clot and the hair by using these binary valuesof the linear fusion operation to get the final body contours. Our exper-iments demonstrates that this algorithm effectively reduces constraintson background environment when extracting contours.Keywords: Contour Extraction, Mobile Apps, Online Clothing, Gaus-sian Mixture Model, Threshold Segmentation, Linear Fusion1 IntroductionThe extraction of human contours is an essential part of the development ofvision-based non-contact human body measurements and modeling systems. Ap-plications of such systems are online shopping, online games, virtual reality ap-plications. It directly affects the accuracy of subsequent body size measurements.Therefore, how to accurately extract human contours in images is a top priority[7]. Background environment directly affects the accuracy of human identifica-tion segmentation techniques.Some research work makes a strict background requirements in order to ex-tract accurate human body contours. Lin et al. [6] [4] collected images that werepretreated by re-edge detection to extract more accurate body contours. Therequirement was that the person must wear special clothing to reduce the com-putation workload. Deng et al [3] have used the background difference method toextract the body contours based on the condition that all the pixels were inde-pendent, thus, a fixed camera with a good location on the characters and scenes2 Li Wang et al.was needed. Camera parameters and environmental conditions error are not con-ducive to universal. Yang et al.[13] proposed a body contour extraction methodbased on the HSV color space [10], which reduced the requirements for back-ground, clothing, and lighting. However, for backdrops and floors, the methodrequired a solid color and the color of clothing that were distinctive from thesurrounding environment to segment the skin, clothes, floor, background wall forthe extraction of body contours. But the method did not establish a skin model,because the range of the skin colour difference amongst different people can bevery large, so the algorithm is not highly adaptable in different situations.The main contribution of this paper is to develop a method for body contourextraction without too many restrictions, so that it is widely applicable to variousconditions for online applications such as mobile applications (mobile apps.).2 BackgroundA large number of studies have shown that the main factor in the appearance ofhuman skin color differences is due to the chromaticity. The distribution of skincolors of different people is similar to an elliptical shape in a smaller area [9].Currently, skin detection models can be roughly categorized into threshold basedmethod [12]and elliptical models [9]. The threshold method determines the rangeof the color space component in order to distinguish between the skin color andnon-skin color points, albeit it is fast and efficient, it is difficult to determine therange of parameters. On the other hand, elliptical model establishes a skin colordistribution by using the ellipse model to determine whether the point falls inthe ellipse area, thus the pixel represents the skin colour. However, because thismethod is based on a large number of training samples in order to determine theoverall all boundary of the skin color distribution, obtaining the training datais the core of the method. More recently, Bayesian model [1] has attached muchresearch attentions, which requires the establishment of a skin color classifier,also a large number of training sets is needed to ensure its detection rate. TheGaussian model, including a single Gaussian model [11] and a mixed Gaussianmodel [8], relies on the probability density formula to determine the probabilityof a pixel belonging to the skin point or not. Therefore, Gaussian model canaccurately detect the skin area beyond the boundary of the skin area, but thespeed is slow and is affected by the color space.From the above description, we can see that a large number of body contourextraction algorithms impose strict requirement on the visual source to be pro-cessed, such as the requirement on the scene or the background of the sourcematerials, greatly limiting the adaptive and widespread of the use of these meth-ods. In order to meet the real-time performance of the system, this paper com-bines the threshold method and the ellipse model to extract the contours of thehuman body to achieve efficient real-time performance. The algorithm consistsof three steps: a) Hair and shorts separation using a feature H component inde-pendent of the light changes to differentiate the hair and coat; b) Skin divisionby using a CbCr color space [2] [14] oval skin detection model to separate theHuman Body Contour Extraction 3Fig. 1: Body contour extraction flow chartskin, while automatically extracting human skin samples; extracting the sampleHSV value as a threshold for better skin segmentation effect; c) T-shirt divi-sion using the threshold segmentation method to extract contour, because theshirt color is single. The algorithm is robust to the background walls and floors,and gradually extracts the contours of human hair, skin and cloth to obtain thebody contour. It further reduces the backdrop and the floor, the clothe of thepre-shooting requirements. Thus, the adaptability of the algorithm has also beengreatly improved.3 Improved Body Contour Extraction MethodThis paper introduces a robust approach for extracting contours of human bodyaccurately from the complex background environment. Removing a backgroundscene to sperate the human body from the scene is a difficult task due to therapid change of the background environment. Fig 1 shows the proposed bodycontour extraction flow chart. The focus of this paper is on the characteristicsof the human body and how to extract human body efficiently and effectively.Figure 1 outlines the extraction process, from which we can see that the maincomputation process for the division of clothing, hair, skin, individual differencesin the body colour constitutes a very large part of this process. The use of theoval skin detection model combined with the human skin samples of HSV tomaximize the effect of statistical analysis can significantly improve the skin colorarea recognition rate, hence, the effectiveness of the approach.RGB images taken by a smart phone were used in the experiments. Due tophoto pixels, in order to improve the processing speed and reduce the amountof redundant information, the original images were compressed to 450 X 800resolution. As shown in Figure 2, in the experiment, a general requirement forclothing can be that participant wears a single colour vest and a dark colourshorts, and hair bundles and no shoes. The front posture of the participantwas standing with feet and arms were 45 degrees open (Figure 2 (a)). The sidepose was hands were vertical hanging straight (Figure 2 (b)). Bright lightingconditions was preferable, but the general home environment would be adequate.4 Li Wang et al.(a) (b) (c) (d)Fig. 2: RGB images (a) front image, (b)side image.(c) H binarization . (d) Pro-cessed image.3.1 HSV Colour Space and Segmenting Hair and ClothThe HSV colour space is an oriented colour coordinate system, this colour modelis close to the human colour-aware simulation model [5], where H represents thechromaticity (a measure of the composition of the color spectrum), S representsthe saturation (the pure wavelength ratio in the main wavelength), indicatingthe degree of the same brightness of a color distance, and V represents the purity(relative to the brightness of white light). The equation for RGB transform toHSV component is Equations 1.V = max(R,G,B)S ={V−min(R,G,B)V V 6= 00 V = 0H =60G−BV−minR,G,B V = R120 + 60(B−R)V−min(R,G,B)) V = G240 + 60(B−R))V−min(R,G,B)) V = BH = H + 360, H < 0(1)In order to differentiate various parts of the human body, we must firstlyanalyze the HSV channels of the image and its histogram, as shown in Figure3. The color histogram is the colour features of the image, which describes pro-portions of different colours in the entire image. The advantages of using HSVchannels are that these are that these are suitable for describing images that aredifficult to automatically segment and do not need to consider objects spatialpositions in the image.In Figure 3, histogram peak boundaries were not clear. Therefore, directlyfinding the threshold of the human body from the histograms can be very dif-ficult. Because H is almost independent to the changes in light, it can be usedas colour comparison, so that images can be separated from the colorful color.Figure 3(a) shows hair and cloth colour contrast in H channel image. The three-thirds of the region below the pixel value becomes 0 and the removal of the smallarea can be extracted. Figure 2 (d) shows the hair and shorts binary profiles.Human Body Contour Extraction 5(a) H (b) S (c) V (d) histogramFig. 3: HSV channel image and histogram.3.2 YCbCr Colour Space and Oval Skin Detection ModelThe luminance and chrominance components of the YCbCr colour space areseparated, hence they are often used as colour models for detecting skin colours.In this mode Y is the brightness, Cb is the blue component of the light source, andCr is the red component in the light source. The original RGB image conversionfor YCbCr formula is as follows:Y = 0.299 ∗R+ 0.587 ∗G+ 0.114 ∗BCr = (R− Y ) ∗ 0.713 + dCb = (B − Y ) ∗ 0.564 + dR = Y + 1.403 ∗ (Cr − d)G = Y − 0.714 ∗ (Cr − d)− 0.344 ∗ (Cb− d)B = Y + 1.773 ∗ (Cb− d)with, d = 128, 8bit; = 32768, 16bit; = 0.5, f loat(2)The main factor in the appearance of the difference in colours is the skincolour variations of people represented as a skin colour distribution relative tothe concentration in a smaller area, i.e. similar to the oval shape. The colourof the person can be converted to a YCbCr colour space. In the CbCr planeprojection, we can get the skin colour sample set and KL transformation toestablish the ellipse model. By determining whether the pixel CbCr value isin the ellipse model area, the pixel can be identified as a skin colour pixel ornon-skin color pixel, separating the skin area out. The ellipse model is definedby:(x− ecx)2a2+(y − ecy)2b2= 1 (3)Parameter matrix is given by:[xy]=[cosθ sinθ−sinθ cosθ] [C‘bCxC‘rCy](4)6 Li Wang et al.(a) (b) (c) (d) (e)Fig. 4: Oval skin model: (a) The elliptical model for skin; (b) Ellipse modelsbinary map.(c) Sample skin test chart; (d) Fig. 4 (b) and Fig. 4 (c) linear fusiongraph;(e) Skin segmentation binary map(a) (b) (c) (d) (e) (f)Fig. 5: Automatically extract skin((a)-(c)) and cloth samples((d)-(e))As shown in Figure 4(a), the skin area is found to be defective, and theelliptical model does not fully recognize all the skin pixels, which is due todifferences in the skin colour variations from person to person.3.3 Thresholds Division of The Skin ColourIn order to solve the problem that the ellipse detection model cannot completelyseparate the skin, we extract the skin samples using the HSV value of the sampleto determine the threshold parameters to divide the skin points. This approachhelps identify the problem skin pixels and is able to mitigate the oval skin de-tection model deficiencies. Further, in order to define the threshold range θi forskin division, it is necessary to extract multiple sets of skin samples Si of theperson. Define the sample Region of Interest (ROI) which specifies the upper leftcorner coordinates Pi(mi, ni) of the rectangle and the length and the width Li,Wi of the rectangle. After a large number of experiments, it is concluded thatthe oval skin detection model of the leg and neck segmentations are not idealfor the distribution of human skin colour, we extracted three groups of samplesfrom the left leg, right leg, and neck position to compensate the issues causedby the lighting.After analyzing the sample histogram, the HSV maximum value of each sam-ple is obtained, and it is set to separate the threshold. The pixels in the mostsignificant range are marked as shown in Fig. 4 (c), then linearly merged withFig. 4 (b) as shown in Fig. 4 (d), the extraction process must have some interfer-Human Body Contour Extraction 7(a) (b) (c) (d)Fig. 6: (a)Linear fusion.(b) Processed image, the coat is divided. Extract thecontours of the human body: (c) Integrated map; (d) Morphological operation.(a) (b) (c) (d) (e)Fig. 7: another example: (a)RGB image.(b)Hair and shorts.(c) T-shirt; (d)skin.(e)Results graph.ence points, so that removing the points and some small areas, the integrationstep fills the region to get more accurate results Figure 4 (e) . The method ofextracting the cloth area is similar to that of skin as described above. Since thecloth colour is singular, we also extracted three areas in the middle of the imageas the top cloth as shown in Fig. 5 (b). Through the analysis of the top clothsample, the maximum value was obtained and the pixels in the most valued ar-eas were marked out, and then the three markers were linearly merged as shownin Fig. 6 (a) to obtain the final cloth division as shown in Fig.6(b).The final results was shown in Fig.6 (c), which was obtained by the linearfusion of the results of Fig. 2 (d), Fig. 4 (e) and Fig. 6 (b). Figure 6 (d) wasobtained by morphological closing operations applied to Fig. 6(c). Figure 7 isanother example of this article, the results of the segmentation is good, but alsoproved the feasibility of the proposed method.4 Conclusions and Future WorkThis paper presents an approach of body contour extraction based on the charac-teristics of the surface of the human body. A set of algorithms have been appliedto the main areas of the cloth, hair and skin, taking into account of the large in-dividual differences in the body colour. Based on the algorithm of elliptical skindetection model, our approach combines the two models to automatically ex-8 Li Wang et al.tract skin samples to improve the skin area Recognition rate. Finally, the resultsproduced by the three methods were linearly merged to produce the contoursvia a morphological processing. Experiments show that the approach can extractthe human body contours from relatively complex background scene with goodresults. While this work has served to propose and prototype the body extrac-tion concept, future research will investigate algorithms to further reduce therestrictions on cloth and light and further optimization is also needed.AcknowledgementThis work is supported by Shanxi Province Science and Technology Departmentof International Cooperation Projects (2016JZ026 2016KW-043) and (2016GY-047)References1. Bouzerdoum, D.C..A.: A bayesian approach to skin color classification in ycbcrcolor space. In: TENCON 2000. Proceedings, 24-27 Sept. (2000)2. Dawod A Y, Abdullah J, A.M.J.: Adaptive skin color model for hand segmentation.In: International Conference on Computer Applications and Industrial Electronics.IEEE, 486-489. (2010)3. Deng, W., L.G.W.J.e.a.: Image extraction method of 3d human body feature basedon image. journal of zhejiang university, 2010, 44 (5): 837-840. Journal of ZhejiangUniversity, 44 (5): 837-840. (2010)4. Jiang L, Yao J, L.B.e.a.: Automatic body feature extraction from front and sideimages. Journal of Software Engineering & Applications, 05(12):94-100. (2012)5. Kaehler A, B.G.: Learning OpenCV 3 (2016)6. Lin Y. L., Wang M., e.a.: Constructing 3d human model from front and side images.Expert Systems with Applications, 39(5):5012-5018. (2012)7. Lu, Jun-Ming, W.e.a.: Automated anthropometric data collection using 3d wholebody scanners. Expert Systems with Applications, 35(1-2):407-414. (2008)8. Mokhtar M.Hasan, P.K.M.: Superior skin color model using multiple of gaussianmixture model. British Journal of Science, Vol. 6 (1) (2012)9. Peter Peer, Jure Kovac, F.S.: Human skin color clustering for face detection. In:EUROCON 2003 (2003)10. R, S.: Computer Vision: Algorithms and Applications. 21(8):2601-2605. (2010)11. Subban R, M.R.: Human skin segmentation in color images using gaussian colormode. Recent Advances in Intelligent Informatics. Springer International Publish-ing, 13-21. (2014)12. Teerayoot Sawangsri, Vorapoj Patanavijit, S.J.: Face segmentation using novelskin-color map and morphological technique. In: World Academy of Science, En-gineering and Technology 2 (2005)13. Yang M., Liu G., D.H.: A study on human body contour extraction method basedon hsv color space. 2015 (2): 41-46. JOURNAL OF BEIJING FASHION INSTI-TUTE (NATURAL SCIENCE EDITION), (2015)14. Zhang, Q.: Hand gesture segmentation based on mixed skin-color model and fcm al-gorithm. Journal of Information & Computational Science,12(9):3527-3536. (2015)

An efficient human body contour extraction method for mobile apps.

Abstract

Similar works

Full text

Available Versions

Bournemouth University Research Online