Neo: Virtual Object Modeling using Commodity Hardware

Recent developments in augmented reality technology have paved way for newapplications in a wide range of areas. These include the commercial markets,medicine applications, military applications and education. The technology pro-vides immersive images to enhance our perception of the world. Augmentedreality addresses challenges related to problem-solving by seamlessly integrat-ing digital images into real-world images.In the context of construction and maintenance industry, project inspections canbe time-consuming and tedious. These inspections involve usages of expensiveand specialized hardware. Some inspections even use physical blueprints anddrawings along with standardized measurement tools. This approach can posepractical challenges and be prone to errors.In this thesis we present Neo, a surface reconstruction system on commodityhardware. It utilizes augmented reality technology by scanning physical sur-roundings and reconstructs them as virtual objects. They are displayed on topof the camera’s live preview of the real world. By using a pipeline architecturewe model the physical surroundings in terms of their shapes and visual appear-ances. Cyber-physical information about the reconstructed virtual models areannotated in real-time. Evaluations of the system show us potentials to createrealistic copies of physical object

3DFusion, A real-time 3D object reconstruction pipeline based on streamed instance segmented data

This paper presents a real-time segmentation and reconstruction system that utilizes RGB-D images to generate accurate and detailed individual 3D models of objects within a captured scene. Leveraging state-of-the-art instance segmentation techniques, the system performs pixel-level segmentation on RGB-D data, effectively separating foreground objects from the background. The segmented objects are then reconstructed into distinct 3D models in a high-performance computation platform. The real-time 3D modelling can be applied across various domains, including augmented/virtual reality, interior design, urban planning, road assistance, security systems, and more. To achieve real-time performance, the paper proposes a method that effectively samples consecutive frames to reduce network load while ensuring reconstruction quality. Additionally, a multi-process SLAM pipeline is adopted for parallel 3D reconstruction, enabling efficient cutting of the clustering objects into individuals. This system employs the industry-leading framework YOLO for instance segmentation. To improve YOLO's performance and accuracy, modifications were made to resolve duplicated or false detection of similar objects, ensuring the reconstructed models align with the targets. Overall, this work establishes a robust real-time system with a significant enhancement for object segmentation and reconstruction in the indoor environment. It can potentially be extended to the outdoor scenario, opening up numerous opportunities for real-world applications

Registration Using Projective Reconstruction for Augmented Reality Systems

In AR systems, registration is one of the most difficult problems currently limiting their applications. In this paper, we proposed a simple registration method using projective reconstruction. This method consists of two steps: embedding and tracking. Embedding involves specifying four points to build the world coordinate system on which a virtual object will be superimposed. In tracking, a projective reconstruction technique in computer vision is used to track the four specified points to compute the modelview transformation for augmentation. This method is simple as only four points need to be specified at the embedding stage, and the virtual object can then be easily augmented in a real video sequence. In addition, it can be extended to a common scenario using a common projective matrix. The proposed method has three advantages: (1) It is fast because the linear least square method can be used to estimate the related matrix in the algorithm and it is not necessary to calculate the fundamental matrix in the extended case; (2) A virtual object can still be superimposed on a related area even if some parts of the specified area are occluded during the augmentation process; and (3) This method is robust because it remains effective even when not all the reference points are detected during the augmentation process (in the rendering process), as long as at least six pairs of related reference point correspondences can be found. Several projective matrices obtained from the authors’ previous work, which are unrelated with the present AR system, were tested on this extended registration method. Experiments showed that these projective matrices can also be utilized for tracking the specified points.Singapore-MIT Alliance (SMA

LEVEL-BASED CORRESPONDENCE APPROACH TO COMPUTATIONAL STEREO

One fundamental problem in computational stereo reconstruction is correspondence. Correspondence is the method of detecting the real world object reflections in two camera views. This research focuses on correspondence, proposing an algorithm to improve such detection for low quality cameras (webcams) while trying to achieve real-time image processing. Correspondence plays an important role in computational stereo reconstruction and it has a vast spectrum of applicability. This method is useful in other areas such as structure from motion reconstruction, object detection, tracking in robot vision and virtual reality. Due to its importance, a correspondence method needs to be accurate enough to meet the requirement of such fields but it should be less costly and easy to use and configure, to be accessible by everyone. By comparing current local correspondence method and discussing their weakness and strength, this research tries to enhance an algorithm to improve previous works to achieve fast detection, less costly and acceptable accuracy to meet the requirement of reconstruction. In this research, the correspondence is divided into four stages. Two stages of preprocessing which are noise reduction and edge detection have been compared with respect to different methods available. In the next stage, the feature detection process is introduced and discussed focusing on possible solutions to reduce errors created by system or problem occurring in the scene such as occlusion. Lastly, in the final stage it elaborates different methods of displaying reconstructed result. Different sets of data are processed based on the steps involved in correspondence and the results are discussed and compared in detail. The finding shows how this system can achieve high speed and acceptable outcome despite of poor quality input. As a conclusion, some possible improvements are proposed based on ultimate outcome