Recognizing the presence of hidden visual markers in digital images

As the promise of Virtual and Augmented Reality (VR and AR) becomes more realistic, an interesting aspect of our enhanced living environment includes the availability — indeed the potential ubiquity — of scannable markers. Such markers could represent an initial step into the AR and VR worlds. In this paper, we address the important question of how to recognise the presence of visual markers in freeform digital photos. We use a particularly challenging marker format that is only minimally constrained in structure, called Artcodes. Artcodes are a type of topological marker system enabling people, by following very simple drawing rules, to design markers that are both aesthetically beautiful and machine readable. Artcodes can be used to decorate the surface of any objects, and yet can also contain a hidden digital meaning. Like some other more commonly used markers (such as Barcodes, QR codes), it is possible to use codes to link physical objects to digital data, augmenting everyday objects. Obviously, in order to trigger the behaviour of scanning and further decoding of such codes, it is first necessary for devices to be aware of the presence of Artcodes in the image. Although considerable literature exists related to the detection of rigidly formatted structures and geometrical feature descriptors such as Harris, SIFT, and SURF, these approaches are not sufficient for describing freeform topological structures, such as Artcode images. In this paper, we propose a new topological feature descriptor that can be used in the detection of freeform topological markers, including Artcodes. This feature descriptor is called a Shape of Orientation Histogram (SOH). We construct this SOH feature vector by quantifying the level of symmetry and smoothness of the orientation histogram, and then use a Random Forest machine learning approach to classify images that contain Artcodes using the new feature vector. This system represents a potential first step for an eventual mobile device application that would detect where in an image such an unconstrained code appears. We also explain how the system handles imbalanced datasets — important for rare, handcrafted codes such as Artcodes — and how it is evaluated. Our experimental evaluation shows good performance of the proposed classification model in the detection of Artcodes: obtaining an overall accuracy of approx. 0.83, F2 measure 0.83, MCC 0.68, AUC-ROC 0.93, and AUC-PR 0.91

Connecting Everyday Objects with the Metaverse: A Unified Recognition Framework

The recent Facebook rebranding to Meta has drawn renewed attention to the metaverse. Technology giants, amongst others, are increasingly embracing the vision and opportunities of a hybrid social experience that mixes physical and virtual interactions. As the metaverse gains in traction, it is expected that everyday objects may soon connect more closely with virtual elements. However, discovering this "hidden" virtual world will be a crucial first step to interacting with it in this new augmented world. In this paper, we address the problem of connecting physical objects with their virtual counterparts, especially through connections built upon visual markers. We propose a unified recognition framework that guides approaches to the metaverse access points. We illustrate the use of the framework through experimental studies under different conditions, in which an interactive and visually attractive decoration pattern, an Artcode, is used as the approach to enable the connection. This paper will be of interest to, amongst others, researchers working in Interaction Design or Augmented Reality who are seeking techniques or guidelines for augmenting physical objects in an unobtrusive, complementary manner.Comment: This paper includes 6 pages, 4 figures, and 1 table, and has been accepted to be published by the 2022 IEEE 46th Annual Computers, Software, and Applications Conference (COMPSAC), Los Alamitos, CA, US

AirCode: Unobtrusive Physical Tags for Digital Fabrication

We present AirCode, a technique that allows the user to tag physically fabricated objects with given information. An AirCode tag consists of a group of carefully designed air pockets placed beneath the object surface. These air pockets are easily produced during the fabrication process of the object, without any additional material or postprocessing. Meanwhile, the air pockets affect only the scattering light transport under the surface, and thus are hard to notice to our naked eyes. But, by using a computational imaging method, the tags become detectable. We present a tool that automates the design of air pockets for the user to encode information. AirCode system also allows the user to retrieve the information from captured images via a robust decoding algorithm. We demonstrate our tagging technique with applications for metadata embedding, robotic grasping, as well as conveying object affordances.Comment: ACM UIST 2017 Technical Paper

Vision-based techniques for gait recognition

Global security concerns have raised a proliferation of video surveillance devices. Intelligent surveillance systems seek to discover possible threats automatically and raise alerts. Being able to identify the surveyed object can help determine its threat level. The current generation of devices provide digital video data to be analysed for time varying features to assist in the identification process. Commonly, people queue up to access a facility and approach a video camera in full frontal view. In this environment, a variety of biometrics are available - for example, gait which includes temporal features like stride period. Gait can be measured unobtrusively at a distance. The video data will also include face features, which are short-range biometrics. In this way, one can combine biometrics naturally using one set of data. In this paper we survey current techniques of gait recognition and modelling with the environment in which the research was conducted. We also discuss in detail the issues arising from deriving gait data, such as perspective and occlusion effects, together with the associated computer vision challenges of reliable tracking of human movement. Then, after highlighting these issues and challenges related to gait processing, we proceed to discuss the frameworks combining gait with other biometrics. We then provide motivations for a novel paradigm in biometrics-based human recognition, i.e. the use of the fronto-normal view of gait as a far-range biometrics combined with biometrics operating at a near distance