Internal-External Boundary Attention Fusion for Glass Surface Segmentation

Glass surfaces of transparent objects and mirrors are not able to be uniquely and explicitly characterized by their visual appearances because they contain the visual appearance of other reflected or transmitted surfaces as well. Detecting glass regions from a single-color image is a challenging task. Recent deep-learning approaches have paid attention to the description of glass surface boundary where the transition of visual appearances between glass and non-glass surfaces are observed. In this work, we analytically investigate how glass surface boundary helps to characterize glass objects. Inspired by prior semantic segmentation approaches with challenging image types such as X-ray or CT scans, we propose separated internal-external boundary attention modules that individually learn and selectively integrate visual characteristics of the inside and outside region of glass surface from a single color image. Our proposed method is evaluated on six public benchmarks comparing with state-of-the-art methods showing promising results

Enhanced Boundary Learning for Glass-like Object Segmentation

Glass-like objects such as windows, bottles, and mirrors exist widely in the real world. Sensing these objects has many applications, including robot navigation and grasping. However, this task is very challenging due to the arbitrary scenes behind glass-like objects. This paper aims to solve the glass-like object segmentation problem via enhanced boundary learning. In particular, we first propose a novel refined differential module that outputs finer boundary cues. We then introduce an edge-aware point-based graph convolution network module to model the global shape along the boundary. We use these two modules to design a decoder that generates accurate and clean segmentation results, especially on the object contours. Both modules are lightweight and effective: they can be embedded into various segmentation models. In extensive experiments on three recent glass-like object segmentation datasets, including Trans10k, MSD, and GDD, our approach establishes new state-of-the-art results. We also illustrate the strong generalization properties of our method on three generic segmentation datasets, including Cityscapes, BDD, and COCO Stuff. Code and models is available at \url{https://github.com/hehao13/EBLNet}.Comment: ICCV-2021 Code is availabe at https://github.com/hehao13/EBLNe

NeRRF: 3D Reconstruction and View Synthesis for Transparent and Specular Objects with Neural Refractive-Reflective Fields

Neural radiance fields (NeRF) have revolutionized the field of image-based view synthesis. However, NeRF uses straight rays and fails to deal with complicated light path changes caused by refraction and reflection. This prevents NeRF from successfully synthesizing transparent or specular objects, which are ubiquitous in real-world robotics and A/VR applications. In this paper, we introduce the refractive-reflective field. Taking the object silhouette as input, we first utilize marching tetrahedra with a progressive encoding to reconstruct the geometry of non-Lambertian objects and then model refraction and reflection effects of the object in a unified framework using Fresnel terms. Meanwhile, to achieve efficient and effective anti-aliasing, we propose a virtual cone supersampling technique. We benchmark our method on different shapes, backgrounds and Fresnel terms on both real-world and synthetic datasets. We also qualitatively and quantitatively benchmark the rendering results of various editing applications, including material editing, object replacement/insertion, and environment illumination estimation. Codes and data are publicly available at https://github.com/dawning77/NeRRF

Korean mask-dance and Aristotle\u27s poetics

Korean mask-dance is the traditional theatre of Korea. It was formerly the country\u27s most well-known form of drama among traditional theatrical entertainments. This study explores the theatrical structure of Korean mask-dance as well as its historical background. The rise of Korean mask-dance may be traced back to the shamanistic village ritual which gradually became similar to the extant form after absorbing aspects of the Buddhism festival through the Goryeo Dynasty, which lasted from 918-1392). During the Joseon Dynasty (1392-1910), the mask-dance had acquired its basic form with aspects of professional theatrical entertainment. The mask-dances have been performed during traditional holidays and festivals over the past three hundred years. Four types of the mask-dance continue to exist today, all being derived from their geographic origins. Many scholar and artists have explained the value of Korean mask-dance and its own esthetic level. Dance, song, music, masks, costumes, props, stage, and audience participation of Korean mask-dance are obvious theatrical elements and have their own separate meaning. The main purpose of this study is to examine the weak parts of the dramatic structure of the art form and attempt to analyze it using Aristotle\u27s Poetics (384-322 BCE). When Korean mask-dance is analyzed by Aristotle\u27s concept of drama, the mask-dance exactly reverses this order of importance of the dramatic elements. Through recognition of both the uniqueness of Korean mask-dance and the dramatic standards of Aristotle\u27s concept, this study should enable scholars and artists to embrace more fully the universal nature of theatre

Mirror Position Detection in a Catoptric Surface

The Catoptric Surface research project is a pioneering exploration of controlling daylight effects within built environments. In this thesis, we focus on the mirror position detection problem, which plays a vital role in achieving dynamic control over the direction of reflected light within a space. To address the challenge of mirror position detection, we employ computer vision techniques, specifically edge detection and the RANdom SAmple Consensus (RANSAC) algorithm. Edge detection is utilized to identify significant changes in intensity or color, corresponding to object boundaries, while RANSAC is applied for ellipse fitting. By iteratively selecting minimal subsets of points and fitting ellipses that meet geometric constraints, we attempt to accurately determine the position and geometry of mirrors in the catoptric array. We evaluate two different RANSAC libraries for ellipse fitting, and our findings show that the skimage library in Python provides superior results compared to other alternatives. Additionally, we leverage the multiprocessing package to enable parallel processing, improving the efficiency of mirror detection. We conclude that it is possible to detect single steps of mirror movement, however, reliable operation is highly sensitive to parameter settings within the computational pipeline