Visual Human Tracking and Group Activity Analysis: A Video Mining System for Retail Marketing

Thesis (PhD) - Indiana University, Computer Sciences, 2007In this thesis we present a system for automatic human tracking and activity recognition from video sequences. The problem of automated analysis of visual information in order to derive descriptors of high level human activities has intrigued computer vision community for decades and is considered to be largely unsolved. A part of this interest is derived from the vast range of applications in which such a solution may be useful. We attempt to find efficient formulations of these tasks as applied to the extracting customer behavior information in a retail marketing context. Based on these formulations, we present a system that visually tracks customers in a retail store and performs a number of activity analysis tasks based on the output from the tracker. In tracking we introduce new techniques for pedestrian detection, initialization of the body model and a formulation of the temporal tracking as a global trans-dimensional optimization problem. Initial human detection is addressed by a novel method for head detection, which incorporates the knowledge of the camera projection model.The initialization of the human body model is addressed by newly developed shape and appearance descriptors. Temporal tracking of customer trajectories is performed by employing a human body tracking system designed as a Bayesian jump-diffusion filter. This approach demonstrates the ability to overcome model dimensionality ambiguities as people are leaving and entering the scene. Following the tracking, we developed a two-stage group activity formulation based upon the ideas from swarming research. For modeling purposes, all moving actors in the scene are viewed here as simplistic agents in the swarm. This allows to effectively define a set of inter-agent interactions, which combine to derive a distance metric used in further swarm clustering. This way, in the first stage the shoppers that belong to the same group are identified by deterministically clustering bodies to detect short term events and in the second stage events are post-processed to form clusters of group activities with fuzzy memberships. Quantitative analysis of the tracking subsystem shows an improvement over the state of the art methods, if used under similar conditions. Finally, based on the output from the tracker, the activity recognition procedure achieves over 80% correct shopper group detection, as validated by the human generated ground truth results

小売店舗における消費者行動の柔軟な認識システムに関する研究

Tohoku University博士（情報科学）thesi

Analyzing computer vision models for detecting customers: a practical experience in a mexican retail

Computer vision has become an important technology for obtaining meaningful data from visual content and providing valuable information for enhancing security controls, marketing, and logistic strategies in diverse industrial and business sectors. The retail sector constitutes an important part of the worldwide economy. Analyzing customer data and shopping behaviors has become essential to deliver the right products to customers, maximize profits, and increase competitiveness. In-person shopping is still a predominant form of retail despite the appearance of online retail outlets. As such, in-person retail is adopting computer vision models to monitor store products and customers. This research paper presents the development of a computer vision solution by Lytica Company to detect customers in Steren’s physical retail stores in Mexico. Current computer vision models such as SSD Mobilenet V2, YOLO-FastestV2, YOLOv5, and YOLOXn were analyzed to find the most accurate system according to the conditions and characteristics of the available devices. Some of the challenges addressed during the analysis of videos were obstruction and proximity of the customers, lighting conditions, position and distance of the camera concerning the customer when entering the store, image quality, and scalability of the process. Models were evaluated with the F1-score metric: 0.64 with YOLO FastestV2, 0.74 with SSD Mobilenetv2, 0.86 with YOLOv5n, 0.86 with YOLOv5xs, and 0.74 with YOLOXn. Although YOLOv5 achieved the best performance, YOLOXn presented the best balance between performance and FPS (frames per second) rate, considering the limited hardware and computing power conditions

Tecnología para Tiendas Inteligentes

Trabajo de Fin de Grado en Doble Grado en Ingeniería Informática y Matemáticas, Facultad de Informática UCM, Departamento de Ingeniería del Software e Inteligencia Artificial, Curso 2020/2021Smart stores technologies exemplify how Artificial Intelligence and Internet of Things can effectively join forces to shape the future of retailing. With an increasing number of companies proposing and implementing their own smart store concepts, such as Amazon Go or Tao Cafe, a new field is clearly emerging. Since the technologies used to build their infrastructure offer significant competitive advantages, companies are not publicly sharing their own designs. For this reason, this work presents a new smart store model named Mercury, which aims to take the edge off of the lack of public and accessible information and research documents in this field. We do not only introduce a comprehensive smart store model, but also work-through a feasible detailed implementation so that anyone can build their own system upon it.Las tecnologías utilizadas en las tiendas inteligentes ejemplifican cómo la Inteligencia Artificial y el Internet de las Cosas pueden unir, de manera efectiva, fuerzas para transformar el futuro de la venta al por menor. Con un creciente número de empresas proponiendo e implementando sus propios conceptos de tiendas inteligentes, como Amazon Go o Tao Cafe, un nuevo campo está claramente emergiendo. Debido a que las tecnologías utilizadas para construir sus infraestructuras ofrecen una importante ventaja competitiva, las empresas no están compartiendo públicamente sus diseños. Por esta razón, este trabajo presenta un nuevo modelo de tienda inteligente llamado Mercury, que tiene como objetivo mitigar la falta de información pública y accesible en este campo. No solo introduciremos un modelo general y completo de tienda inteligente, sino que también proponemos una implementación detallada y concreta para que cualquier persona pueda construir su propia tienda inteligente siguiendo nuestro modelo.Depto. de Ingeniería de Software e Inteligencia Artificial (ISIA)Fac. de InformáticaTRUEunpu

Social presence and dishonesty:perceptions from security guards

Self-service technologies within retail enable customers to scan, bag and pay for their items independent from staff involvement. The use of self-service, due to its nature of reducing social interaction between customers and staff, has been implicated in creating opportunities for thefts to occur. However, the perception of social presence, such as induced by surveillance, induces customers to show more prosocial behavior. As security personnel are at the forefront to deal with dishonest customers, we conducted semi-structured interviews with security guards in two major supermarkets in the UK to assess factors surrounding theft, with a view to identify operational or technological opportunities to address theft. Our findings show that the perceived motivational and situational factors contributing to theft are complex. We conclude that surveillance in its current form does not appear to provide a sufficient social presence to prevent potential theft at self-service checkouts (SCOs). Future research could focus on additional surveillance measures to induce social presence, such as technological implementations in the SCO itself.</p

A First Look Into Users’ Perceptions of Facial Recognition in the Physical World

Facial recognition (FR) technology is being adopted in both private and public spheres for a wide range of reasons, from ensuring physical safety to providing personalized shopping experiences. It is not clear yet, though, how users perceive this emerging technology in terms of usefulness, risks, and comfort. We begin to address these questions in this paper. In particular, we conducted a vignette-based study with 314 participants on Amazon Mechanical Turk to investigate their perceptions of facial recognition in the physical world, based on thirty-five scenarios across eight different contexts of FR use. We found that users do not have a binary answer towards FR adoption. Rather, their perceptions are grounded in the specific contexts in which FR will be applied. The participants considered a broad range of factors, including control over facial data, the utility of FR, the trustworthiness of organizations using FR, and the location and surroundings of FR use to place the corresponding privacy risks in context. They weighed the privacy risks with the usability, security, and economic gain of FR use as they reported their perceptions. Participants also noted the reasons and rationals behind their perceptions of facial recognition, which let us conduct an in-depth analysis of their perceived benefits, concerns, and comfort with using this technology in various scenarios. Through this first systematic look into users’ perceptions of facial recognition in the physical world, we shed light on the tension between FR adoption and users’ concerns. Taken together, our findings have broad implications that advance the Privacy and Security community’s understanding of FR through the lens of users, where we presented guidelines for future research in these directions