Realistic soft shadows in Augmented Reality (AR) is a fascinating topic in computer graphics. Many researchers are involved to have a significant improvement on this demand. In this paper, we have presented a new technique to produce soft shadows using one of the well-known methods in mathematics called Fuzzy Logic. Fuzzy logic is taken into account to generate the realistic soft shadows in AR. The wide light source is split into some parts that each of them plays the rule of a single light source. The desired soft shadow is generated by splitting the wide light source into multiple parts and considering each part as a single light source. The method which we called Fuzzy Soft Shadow is employed in AR to enhance the quality of semi-soft shadows and soft shadows

Ismail, I

Kolivand, H

Kolivand, M

Sunar, MS

LJMU Research Online (Liverpool John Moores University)

BORNEO SCIENCE 36 (2): SEPTEMBER 2015   FUZZY SOFT SHADOW IN AUGMENTED REALITY SYSTEMS  Hoshang Kolivand, Mohd Shahrizal Sunar, Ismahafezi Ismail, Mahyar Kolivand  MaGIC-X (Media and Games Innovation Centre of Excellence) UTM-IRDA Digital Media Centre  Universiti Teknologi Malaysia 81310 Skudai, Johor, Malaysia.  ABSTRACT. Realistic soft shadows in Augmented Reality (AR) is a fascinating topic in computer graphics. Many researchers are involved to have a significant improvement on this demand. In this paper, we have presented a new technique to produce soft shadows using one of the well-known methods in mathematics called Fuzzy Logic. Fuzzy logic is taken into account to generate the realistic soft shadows in AR. The wide light source is split into some parts that each of them plays the rule of a single light source. The desired soft shadow is generated by splitting the wide light source into multiple parts and considering each part as a single light source. The method which we called Fuzzy Soft Shadow is employed in AR to enhance the quality of semi-soft shadows and soft shadows.  KEYWORDS. Soft Shadows, Augmented Reality, Fuzzy logic   INTRODUCTION  The most sophisticated components in outdoor and indoor rendering are shadows and lighting (Ro ̈nnberg, 2004; Boulanger, 2008). In general, lighting refers to global illumination and interaction between objects where the main object in the outdoor environment is the sky (Madsen & Lal, 2013; Xing et al., 2013). Shadows refer to the dark portions and halftones which include all different categories such as hard, soft or even soft outline shadows and self shadowing.  Many techniques have been used to improve soft shadows but they are usually based on shadow volumes (Crown, 1977) and shadow maps (Williams, 1978). Geometrical based algorithms suffer from high cost of computation while image based algorithms suffer from aliasing.  In this paper, we have tried to enhance the quality of shadows in Augmented Reality (AR) using a new technique based on Fuzzy logic (Zadeh, 1965). The technique is called Fuzzy Soft Shadow (FSS). FSS can be implemented on virtual environments as well as augmented reality environments.  Until very recently only a few researchers focused on shadow generation in AR. Nevertheless, much research is presently being conducted in this realm to improve the knowledge base. An accurate timeline of shadow generation in augmented reality is presented in Figure 1.   8  Hoshang Kolivand, Mohd Shahrizal Sunar, Ismahafezi Ismail, Mahyar Kolivand  To achieve a realistic augmented reality, shadows play an important role and are an essential factor for 3D impression of the scene (Debevec, 2004). AR simulation  Figure 1:  Principle techniques for shadow generation in Augmented Reality  of shadows for a virtual object in real environments is difficult due to deeds reconstruction of the real-world scene, especially when details of approximation of the real scene geometry and the light source are known (Jacobs et al., 2005). Jacobs & Loscos (2004) prepared a classification of the illumination methods into two different groups, common illumination (Jacobs et al., 2005; Madsen & Nielsen, 2008; Haller & Hartmann, 2003) and relighting (Yan, 2008) in augmented reality. The credibility of shadow construction with the correct estimation of light source position can be found in (Yan, 2008; Jensen et al., 2009).  Nowrouzezahrai et al. (2011) applied light factorization for mixed frequency shadows in AR facilitating both hard and soft shadows using shadow mapping algorithm with surrounding scene lighting. Although, they emphasize direct and indirect lighting, they could generate both hard and soft shadows for static and animated virtual objects in AR. The shadow generation is based on shadow maps but there is no evidence to show shadows on other objects.   Soft shadows are the main requirements in the current augmented reality systems to make the environments maximally realistic (Kolivand & Sunar, 2013a; Kolivand & Sunar, 2013b). The widely used techniques are categorized as follows.          9   Fuzzy Soft Shadow in Augmented Reality Systems Aittala (2010) applied Convolution Shadow Maps (Annen et al., 2008) to produce soft shadow in AR which employed both mip-map filtering and fast summed area tables to enhance blurring with variable radius (Hensley et al., 2005). The method is applicable to both scenes external and the internal scenes.  Recently, Madsen & Lal (2013) worked on real shadow on virtual objects and vice versa using color imagery, integrated with depth information from a commercial stereo camera setup. The method is applied with taking in to account the estimating of the radiance of the sky and sun for outdoor scenes. Shadow detection is highlighted in this paper as well as the radiance of the sky and sun for outdoor.  In 2014, Kolivand & Sunar presented a new algorithm on shadows and employed on outdoor augmented reality. They generated realistic virtual objects in AR taking the affect of sky color into account.  METHODS      Definition: Assume that p is a point on the open positive half-space H+(p). Point p is in shadow if, and only if, an open segment τ(pq) in H+(p) from p to a source sample q on the light L intersects the scene geometry Ψ. S(p,Ψ)={q∈L|τ(pq)∩H+(p)∩Ψ ̸=Ø}̸=Ø                      (1) This is true if the light travels straight from the light source (Figure 2). Point p is in the umbra if S(p,Ψ) = L., meaning all the light is occluded by Ψ. Point p is in the penumbra if S(p,Ψ) ̸= L which means. It means that not all light is occluded by Ψ. Finally, point p is in lit if S(p,Ψ)∩L = Ø. In lit, no part of light is occluded by Ψ.                Figure 2:  Shadows’ regions    10  Hoshang Kolivand, Mohd Shahrizal Sunar, Ismahafezi Ismail, Mahyar Kolivand  2.1 Shadows  Hard shadows are produced due to a single point light source. There are three types of shadows. Hard shadows are generated by a single point light source. The silhouette of hard shadows are sharp and as the result they are called Hard Shadows. Figure 3 (left) illustrates a scene includes an abject equipped with hard shadows.  Semi-Soft Shadows are the second type of shadows which are usually created by a wide light source or some close point light sources. The wide light sources that create semi-soft shadows are not much wide. Figure 3 (center) shows a scene with semi-soft shadows. As can be seen from the figure the silhouette of the shadows are almost soft but the center of shadows are hard enough. This type of shadows are suitable for outdoor rendering.  Soft Shadows are the last type of shadows which make the environments more realistic. These kind of shadows are usually generated when the light source are wide enough. They are appropriate for indoor rendering as there are some wide light sources or the light sources are wide enough compare to the distance between the light source and objects. Figure 3 (right) illustrates a scene includes soft shadows. Figure 3:  Shadows, left: Hard Shadows, center: Semi-Soft Shadows, right: Soft-Shadows  2.2  Fuzzy Logic  However, in almost every case we can produce the same product without fuzzy logic, most of the cases fuzzy is faster and cheaper. Fuzzy sets are convenient for doubtful and unreliable systems, especially for systems which involve mathematical modeling and complex extractions.  Crisp Set In classical mathematics there is a set called the definitive set (Crisp set) that allows us to have many subsets with a membership or no membership.  Deem that fA : R3 → {0, 1}, hard shadows can be defined as: ∀x ∈ A : fA(x) = 1 ⇒ x is in Shadow ∀x ∈ A : fA(x) = 0 ⇒ x is in Lit Fuzzy Set Fuzzy logic allows partial membership and it is a methodology to convert a discrete form system to continuous form. The most important concept of fuzzy logic is the controllers. The fuzzy controllers are designed based on expert knowledge. It is possible to define some concepts like small, large, light, heavy, low, slow, fast, medium, high, tall etc.  11   Fuzzy Soft Shadow in Augmented Reality Systems  To have this common linguistic degree, membership can reveal the amount of that characteristic. For example, to create a shadow with some parts in shadow and some parts not, it is considered as hard shadow. However, in the real world, some parts of the scene are located in shadow, and some parts in lit, with yet other parts not exactly in either shadow or lit. Our proposed technique is to define other concepts of these situations using fuzzy logic for mixed environments. Definition: Let U as a non-empty set and A as a fuzzy set, μA : A → [0,1] is degree membership of x ∈ A. A fuzzy set A is determined by A = {(x,μA(x))|x ∈ A}.  In this case, μ is a function that μ : A → [0, 1]. The definition of μ is:   0 x ∈/ A μ → μ(x) x is fuzzy included ∈ A                                        (2)   1 x∈A  FUZZY SOFT SHADOW  In general, a fuzzy inference system consists of six steps. At first crisp input variables is needed to fuzzifier. Fuzzy operators can be used inside the system. Obtaining the inferior from the prior is the next part of the system. Aggregation of results with consideration of rules is next step before defuzzification. A crisp output of defuzzifier is final result of fuzzy system.  Figure 4 illustrates three parts of soft shadow by fuzzy function. Wide light sources or more than one point light source produce soft shadow. So, to have soft shadow there are two cases: a) When the number of point light sources is more than one. Assume that there are n point light sources and the number of visible point light sources from x is m.  b) When the light source is wide. In this case divide light source to n parts. Deem that the number of visible parts from x is m. In the both cases, there is one-to-one relationship between parts of light source and set of [0, 1].           Figure 4:  Fuzzy function for soft shadows       12    Hoshang Kolivand, Mohd Shahrizal Sunar, Ismahafezi Ismail, Mahyar Kolivand  Fuzzy logic is an appropriated tool to determine whether a part of the scene is located in lit, umbra or penumbra especially for uncertain part such as penumbra. The amount of light source that is visible from each point of shadow receiver and the distance from occluder is important to determine in which region it is. Deem that a set of pairs A={(x, μA(x)) :x∈U}whereU isasetpointsin shadow receiver and μA : A → [0, 1]. For each x ∈ A, μA(x) is called the degree  of membership of x ∈ A. For a finite set A = {x1,...,xn} where xi ∈ R3, the fuzzy set of shadow points U is denoted by T (x) = {Tx1, Tx2, ..., Txn} , which T (x) is termed linguistic like much dark, dark, a little dark , so much light and lit and: μ(x) = {μ1x,μ2x,...,μnx} = { μA(x1)/x1, μA(x2)/x2, …., μA(xn)/xn}                                      (3)  μ(x) is the degree membership amount of shadow that indicates how each point in scene (U ) is mapped to membership value in interval [0, 1]. Let x ∈ A. In the fuzzy set (A,A), if μA(x) = 0, x is in lit, if A(x) = 1, x is in umbra, and x is penumbra if 0 < μA(x) < 1. The set {x ∈ A|μA(x) > 0} is soft shadow which is called penumbra and the set {x ∈ A|μA(x) = 1} is the kernel of shadow which is called umbra. Each rule has two parts. Rule’s introduction (IF-part) determines the situation in fuzzy set, rule’s conclusion (THEN-part) determines the output of fuzzy value. The general fuzzy rules form is:          Figure 5:  Fuzzy Inference System           13Fuzzy Soft Shadow in Augmented Reality Systems    Rule: If x1 ∈Ai and(or,xor)x2 ∈Bi Then y=f(x1,x2) Where x1, x2 and y are discrete values, x1 and x2 are input values but y is output value. Ai and Bi are two separate fuzzy sets characterized by and respectively. In simple words, to implement soft shadow in fuzzy set, the rules are: Rule A: If (the light source is invisible from x) then x is in Umbra  Rule B: If (the light source is partial visible from x) then x is in Penumbra Rule C: If (the light source is completely visible from x) then x is in Lit Consider the triple in the scene. Let x = (x1,x2,x3)t ∈ U, m is the number parts of light sources that are visible from x and dist as xs distance from occluder as inputs, and y = (ys1 , ys2 , ys3 )t as a output vector of fuzzy set, T (m) = {Tm0 ,Tm1 ,...,Tmn} term of set n and T(m) = {Ty0,Ty1,...Tyn} term of set y which Ti = i and Ti = iα, where α is ambient.  Rule: if (m is less) and (dist is less) then shadow is darker  Or  Rule: if (m is more) and (dist is long) then shadow is lighter In current method the membership function and after aggregation is one set for output value which needs to be defuzzificated. Figure 5 reveals all inference steps in fuzzy system. This technique is named Shadow Fuzzy.  RESULTS AND DISCUSSION  The most important step in deriving a fuzzy set is the middle process, which are fuzzy rules. After determining all the fuzzy rules output, aggregation is done and a list of truncated output rules will be produced which is a fuzzy set of input variables of the last process, which is defuzzification.     14 Hoshang Kolivand, Mohd Shahrizal Sunar, Ismahafezi Ismail, Mahyar Kolivand    The following figures depict the relationship between that part of light that is visible to the shadow receiver and the amount of shadow. Another parameter that is important when measuring the amount of shadow is the distance between the shadow receiver and the occluder. All current algorithms designed to create soft shadow have this problem, but fuzzy logic reveals this factor as well as possible. Figure 6 is result of fuzzy system for a point which is located very far from light source and can see less parts of light source.                  Figure 6:  Soft shadows generated using proposed method  Defuzzification is the last process of fuzzy logic. In this step the result of fuzzy logic should be determined in classical logic. Aggregation produces a fuzzy set that is an input of defuzzification. If Crisp x and m are fuzzy system inputs, then after processing by relevant rules, the fuzzy system will recognize the status of x, providing information as to how many parts of light source are visible. We have implemented FSS successfully and measured the FPS. The whole integrated work is performed on an Intel(R) Core(TM) i7-4790 CPU@ 3.6 GHz and an NVIDIA GTX 960 GPU. More details is presented in Table 1.            15Fuzzy Soft Shadow in Augmented Reality Systems                   Figure 7:  Soft shadows generated using proposed method comparable with real shadows   Table 1:  Summary of the performance of FSS method          The results show that FSS is reliable and fast enough to be used in any Augmented Reality as well as virtual environments. We hope that technique explore a new view point in computer graphics and especially in augmented reality to be employed in not only PC based version but also in mobile based AR systems.              16   Hoshang Kolivand, Mohd Shahrizal Sunar, Ismahafezi Ismail, Mahyar Kolivand   CONCLUSION AND FUTURE WORKS  As have been proved by many researchers in different categories of science, fuzzy logic can be involved in diversity of knowledge to enhance the current researches. This encouraged us to take fuzzy logic into account for generating soft shadows easier and faster compare to the current algorithms. In this paper a simple soft shadows is generated based on fuzzy logic. The results are acceptable and the idea shows that computer graphics has a potential to be mixed in fuzzy logic. The idea could enhance the quality as well as the speed of rendering. The next step if this work is to consider fuzzy logic in shadow maps to cast shadows on other objects. Kinect camera also can be considered for casting virtual shadows on real objects take fuzzy logic into account. ACKNOWLEDGEMENTS  This research was supported by Vot. Q.J130000.2709.01K26 PAS grant at MaGICX (Media and Games Innovation Centre of Excellence) UTM-IRDA Digital Media Centre Universiti Teknologi Malaysia 81310 Skudai Johor MALAYSIA. REFERENCE Aittala, M. 2010. Inverse lighting and photorealistic rendering for augmented reality. The Visual Computer, 26(6-8):669–678. Annen, T., Dong, A., Mertens, T., Bekaert, P., Seidel, H-P. & Kautz, J. 2008. Real-time, all-frequency shadows in dynamic scenes. ACM Transactions on Graphics (Proceedings of ACM SIGGRAPH 2008), 27(3):1–34. Boulanger, K. 2008. Real-time realistic rendering of nature scenes with dynamic lighting. Ph.D Thesis, University of Central Florida. Crown. F. 1977. Shadow algorithms for computer graphics. Computer Graphics, 11(2):242–247. Debevec, P. 2004. Image-based lighting. IEEE Computer Graphics and Applications, 22:26–34. Haller, M., Drab, S. & Hartmann, W. 2003. A real-time shadow approach for an augmented reality application using shadow volumes. In Proceedings of VRST 2003, pp. 56–65. Hensley, J., Scheuermann, T., Coombe, G., Singh, M. & Lastra, A. 2005. Fast summed-area table generation and its applications. Comput. Graph. Forum, 24(3):547–555.  Jacobs, K. & Loscos, C. 2004. Classification of illumination methods for mixed reality. In Eurographics, State-of-the-Art Report.  Jacobs, K., Nahmias, J-D., Angus, C., Reche, A., Loscos, C. & Steed, A.2005. Automatic generation of consistent shadows for augmented reality. Proceedings of Graphics Interface 2005, pp. 113–120, 2005.      17  Fuzzy Soft Shadow in Augmented Reality Systems Jensen, B. F., Laursen, J. S., Madsen, J. B. & Pedersen, T. W. 2009. Simplifying real time light source tracking and credible shadow generation for augmented reality. Institute for Media Technology, Aalborg University. Kolivand, H. & Sunar, M. 2013a. A survey of shadow volume algorithms in computer graphics. IETE Technical Review, 30(1):38-46. Kolivand, H. & Sunar, M. 2013b. Covering photometric properties of outdoor components with the effects of sky color in mixed reality. Multimedia Tools and Applications, pp. 1–20. Kolivand, H. & Sunar, M. S. 2014. Realistic Real-Time Outdoor Rendering in Augmented Reality. PLoS ONE, 9(9): e108334. doi:10.1371/journal.pone.0108334 Madsen, C. B. & Lal, B. B. 2013. Estimating outdoor illumination conditions based on detection of dynamic shadows. Computer Vision, Imaging and Computer Graphics. Theory and Applications, Springer.  Madsen, C. B. & Nielsen, M. 2008. Towards probe-less augmented reality. A Position Paper, Computer Vision and Media Technology Lab. Aalborg University, Aalborg, Denmark.  Nowrouzezahrai, D., Geiger, S., Mitchell, K., Sumner, R., Jarosz, W. & Gross, M. 2011. Light factorization for mixed-frequency shadows in augmented reality. 10th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 173–179.  Ro ̈nnberg, S. 2004. Real-time rendering of natural illumination. Citeseer. Williams, L. 1978. Casting curved shadows on curved surfaces. SIGGRAPH ’78, 12(3): 270-274 1978.  Xing, G. Y., Zhou, X. H., Liu, Y. L., Qin, X. Y. & Peng. Q.S. 2013. Online illumination estimation of outdoor scenes based on videos containing no shadow area. Science China Information Sciences, 56(3):1–11.  Yan, F. 2008. Estimation of light source environment for illumination consistency of augmented reality. In First International Congress on Image and Signal Processing, 3:771–775. Zadeh, L. A. 1965. Fuzzy sets. Information and control, 8(3):338–353.                     18  

Fuzzy Soft Shadow in Augmented Reality Systems

Abstract

Similar works

Full text

Available Versions

LJMU Research Online (Liverpool John Moores University)