スマートフォンを用いて近距離からディスプレイとポインティング連携するための不可視ARマーカ

学位の種別: 修士University of Tokyo(東京大学

Local Geometric Consensus: A General Purpose Point Pattern-Based Tracking Algorithm

Proceedings of ACM ISMAR 2015, Fukuoka, JapanInternational audienceWe present a method which can quickly and robustly match 2D and 3D point patterns based on their sole spatial distribution , but it can also handle other cues if available. This method can be easily adapted to many transformations such as similarity transformations in 2D/3D, and affine and perspective transformations in 2D. It is based on local geometric consensus among several local matchings and a refinement scheme. We provide two implementations of this general scheme, one for the 2D homography case (which can be used for marker or image tracking) and one for the 3D similarity case. We demonstrate the robustness and speed performance of our proposal on both synthetic and real images and show that our method can be used to augment any (textured/textureless) planar objects but also 3D objects

NeuralMarker: A Framework for Learning General Marker Correspondence

We tackle the problem of estimating correspondences from a general marker, such as a movie poster, to an image that captures such a marker. Conventionally, this problem is addressed by fitting a homography model based on sparse feature matching. However, they are only able to handle plane-like markers and the sparse features do not sufficiently utilize appearance information. In this paper, we propose a novel framework NeuralMarker, training a neural network estimating dense marker correspondences under various challenging conditions, such as marker deformation, harsh lighting, etc. Besides, we also propose a novel marker correspondence evaluation method circumstancing annotations on real marker-image pairs and create a new benchmark. We show that NeuralMarker significantly outperforms previous methods and enables new interesting applications, including Augmented Reality (AR) and video editing.Comment: Accepted by ToG (SIGGRAPH Asia 2022). Project Page: https://drinkingcoder.github.io/publication/neuralmarker

Comparing fiducial markers performance for a task of a humanoid robot self-calibration of manipulators: A pilot experimental study

© Springer Nature Switzerland AG 2018. This paper presents our pilot study of experiments automation with a real robot in order to compare performance of different fiducial marker systems, which could be used in automated camera calibration process. We used Russian humanoid robot AR-601M and automated it’s manipulators for performing joint rotations. This paper is an extension of our previous work on ARTag, AprilTag and CALTag marker comparison in laboratory settings with large-sized markers that had showed significant superiority of CALTag system over the competitors. This time the markers were scaled down and placed on AR-601M humanoid’s palms. We automated experiments of marker rotations, analyzed the results and compared them with the previously obtained results of manual experiments with large-sized markers. The new automated pilot experiments, which were performed both in pure laboratory conditions and pseudo field environments, demonstrated significant differences with previously obtained manual experimental results: AprilTag marker system demonstrated the best performance with a success rate of 97,3% in the pseudo field environment, while ARTag was the most successful in the laboratory conditions

Projection-based visualization of tangential deformation of nonrigid surface by deformation estimation using infrared texture

In this paper, we propose a projection-based mixed reality system that visualizes the tangential deformation of a nonrigid surface by superimposing graphics directly onto the surface by projected imagery. The superimposed graphics are deformed according to the surface deformation. To achieve this goal, we develop a computer vision technique that estimates the tangential deformation by measuring the frame-by-frame movement of an infrared (IR) texture on the surface. IR ink, which can be captured by an IR camera under IR light, but is invisible to the human eye, is used to provide the surface texture. Consequently, the texture does not degrade the image quality of the augmented graphics. The proposed technique measures individually the surface motion between two successive frames. Therefore, it does not suffer from occlusions caused by interactions and allows touching, pushing, pulling, and pinching, etc. The moving least squares technique interpolates the measured result to estimate denser surface deformation. The proposed method relies only on the apparent motion measurement; thus, it is not limited to a specific deformation characteristic, but is flexible for multiple deformable materials, such as viscoelastic and elastic materials. Experiments confirm that, with the proposed method, we can visualize the surface deformation of various materials by projected illumination, even when the user’s hand occludes the surface from the camera.Punpongsanon, P., Iwai, D. & Sato, K. Projection-based visualization of tangential deformation of nonrigid surface by deformation estimation using infrared texture. Virtual Reality 19, 45–56 (2015). https://doi.org/10.1007/s10055-014-0256-y.This is a post-peer-review, pre-copyedit version of an article published in Virtual Reality. The final authenticated version is available online at: https://doi.org/10.1007/s10055-014-0256-y

Improving Markers for Augmented Reality

Tato práce stručně popisuje moderní metody v rozšířené realitě a dále se zabývá generováním a detekcí Uniform Marker Fields. Hlavním cílem je zlepšovat markery známé jako Unifrom Checker-Board Marker Fields. Toho je dosaženo několika vylepšeními návrhu těchto markerů. Mezi hlavní patří využití širší barevné palety, zaměření na podobnost s vybraným obrázkem a změna tvaru modulů. Výsledný algoritmus byl navržen ke generování právě takových markerů. Výstupem jsou esteticky vypadající markery, které umožňují spolehlivou detekci i v případě výrazného překrytí nebo za špatných světelných podmínek.This thesis briefly describes the modern methods in augmented reality and deals with the synthesis of Uniform Marker Fields. The main purpose of this work is to refine the Uniform Checker-Board Marker Fields. This is achieved by proposing several improvements to the marker field design. The main ones are the use of a color palette, targeting resemblance to an image and changing the shape of the modules. An algorithm was implemented for the generation of such marker fields. As a result, the produced markers are aesthetically appealing, besides they can be reliably detected even when heavily occluded and under terrible lighting conditions.

Augmented Reality and Health Informatics: A Study based on Bibliometric and Content Analysis of Scholarly Communication and Social Media

Healthcare outcomes have been shown to improve when technology is used as part of patient care. Health Informatics (HI) is a multidisciplinary study of the design, development, adoption, and application of IT-based innovations in healthcare services delivery, management, and planning. Augmented Reality (AR) is an emerging technology that enhances the user’s perception and interaction with the real world. This study aims to illuminate the intersection of the field of AR and HI. The domains of AR and HI by themselves are areas of significant research. However, there is a scarcity of research on augmented reality as it applies to health informatics. Given both scholarly research and social media communication having contributed to the domains of AR and HI, research methodologies of bibliometric and content analysis on scholarly research and social media communication were employed to investigate the salient features and research fronts of the field. The study used Scopus data (7360 scholarly publications) to identify the bibliometric features and to perform content analysis of the identified research. The Altmetric database (an aggregator of data sources) was used to determine the social media communication for this field. The findings from this study included Publication Volumes, Top Authors, Affiliations, Subject Areas and Geographical Locations from scholarly publications as well as from a social media perspective. The highest cited 200 documents were used to determine the research fronts in scholarly publications. Content Analysis techniques were employed on the publication abstracts as a secondary technique to determine the research themes of the field. The study found the research frontiers in the scholarly communication included emerging AR technologies such as tracking and computer vision along with Surgical and Learning applications. There was a commonality between social media and scholarly communication themes from an applications perspective. In addition, social media themes included applications of AR in Healthcare Delivery, Clinical Studies and Mental Disorders. Europe as a geographic region dominates the research field with 50% of the articles and North America and Asia tie for second with 20% each. Publication volumes show a steep upward slope indicating continued research. Social Media communication is still in its infancy in terms of data extraction, however aggregators like Altmetric are helping to enhance the outcomes. The findings from the study revealed that the frontier research in AR has made an impact in the surgical and learning applications of HI and has the potential for other applications as new technologies are adopted

Eye Tracking During Interaction with a Screen

Tato práce popisuje systém pro sledování obrazovky počítače v reálném čase a určení pozice uživatelova pohledu na tuto obrazovku s využitím dat z eye-trackovacích brýlí a Uniform Marker Fields. Vytvořený systém lokalizuje obrazovku počítače ve snímcích z kamery umístěné na eye-trackeru snímající scénu před uživatelem. V této scéně je detekován marker zobrazený na monitoru počítače. Po jeho úspěšné detekci je marker skryt a ke sledování pozice obrazovky jsou dále využívány významné body detekované ve snímku z kamery ležící na obrazovce, případně z celého snímku. Pokud sledování významných bodů selže, marker je znovu zobrazen a systém je znovu inicializován pomocí detekovaného markeru. Aby byl marker pro uživatele co nejméně nápadný a obtěžující, jsou využity data z eye-trackeru k tomu, aby byl marker zobrazen co nejdále od oblasti uživatelova zájmu. Pro vyhodnocení výkonu a přesnosti vytvořeného systému a rušivosti markeru pro uživatele bylo provedeno několik experimentů a testování s uživateli.This work presents system for real-time tracking of computer screen and estimating location of user's gaze on this screen with the use of eye-tracking data from a head-mounted eye-tracker and Uniform Marker Fields. The designed system localizes the computer screen in the frames from eye-tracker's front camera using a fragment of Uniform Marker Field displayed on the screen. The marker is hidden after its successful detection and natural feature points from the camera frame lying on the computer screen or, optionally, from the whole frame, are used for tracking the screen. If the natural features tracking fails, marker is displayed again and the tracking is re-initialized. In order to make the marker as unobtrusive as possible, gaze information is used to display the marker as far from the area of user's visual attention as possible. To evaluate the performance and accuracy of the implemented system and the influence of marker appearing on the screen on user's ability to perform his work, several experiments and user testing were conducted.