Planogram Compliance Checking Based on Detection of Recurring Patterns

In this paper, a novel method for automatic planogram compliance checking in retail chains is proposed without requiring product template images for training. Product layout is extracted from an input image by means of unsupervised recurring pattern detection and matched via graph matching with the expected product layout specified by a planogram to measure the level of compliance. A divide and conquer strategy is employed to improve the speed. Specifically, the input image is divided into several regions based on the planogram. Recurring patterns are detected in each region respectively and then merged together to estimate the product layout. Experimental results on real data have verified the efficacy of the proposed method. Compared with a template-based method, higher accuracies are achieved by the proposed method over a wide range of products.Comment: Accepted by MM (IEEE Multimedia Magazine) 201

Concept-based Anomaly Detection in Retail Stores for Automatic Correction using Mobile Robots

Tracking of inventory and rearrangement of misplaced items are some of the most labor-intensive tasks in a retail environment. While there have been attempts at using vision-based techniques for these tasks, they mostly use planogram compliance for detection of any anomalies, a technique that has been found lacking in robustness and scalability. Moreover, existing systems rely on human intervention to perform corrective actions after detection. In this paper, we present Co-AD, a Concept-based Anomaly Detection approach using a Vision Transformer (ViT) that is able to flag misplaced objects without using a prior knowledge base such as a planogram. It uses an auto-encoder architecture followed by outlier detection in the latent space. Co-AD has a peak success rate of 89.90% on anomaly detection image sets of retail objects drawn from the RP2K dataset, compared to 80.81% on the best-performing baseline of a standard ViT auto-encoder. To demonstrate its utility, we describe a robotic mobile manipulation pipeline to autonomously correct the anomalies flagged by Co-AD. This work is ultimately aimed towards developing autonomous mobile robot solutions that reduce the need for human intervention in retail store management.Comment: 8 pages, 9 figures, 2 tables, IEEE Transactions on Systems, Man and Cybernetic

Reversing ShopView analysis for planogram creation

Com o aumento da preocupação dos retalhistas na melhora dos resultados das vendas e da experiência dos consumidores, existe uma necessidade de desenvolver tecnologia que auxilie a otimização desses objetivos. Está provado que uma correta colocação dos produtos nas prateleiras pode aumentar significativamente as vendas e a experiência dos consumidores [1]. Com isto em mente, a Fraunhofer Portugal desenvolveu o ShopView [2], uma solução que tem como objetivo ajudar os retalhistas a extrair, validar e manipular os planogramas a partir de imagens de alta resolução das lojas. Desta forma, esta tese tem como foco a criação de um algoritmo, com auxilio a algoritmos de visão computacional, de forma a extrair informação de images de alta resolução de gondolas de supermercados obtidas pelo ShopView. Particularmente, foram implementados passos de pré-processamento de forma a melhorara a eficiência e precisão de um motor de OCR (Optical Character Recognition) no reconhecimento de texto nos produtos da gondola. Estes algoritmos de pré-processamento são compostos por técnicas de segmentação e remoção de ruido. O uso deste motor de OCR permite obter informações adicionais sobre os produtos e separações das prateleiras, esta informação é posteriormente usada em algoritmos de agrupamento para extrair automáticamente um planograma preciso a partir das imagens. O algoritmo apresentado é capaz de extrair informação relevante das imagens das gondolas, de forma a identificar os produtos existentes e criar metadados válidos sobre esses produtos e a sua localização. Com estes metadados é possível criar, validar e modificar o planograma no ShopView. O uso de OCR neste algoritmo oferece vantagens sobre as outras abordagens disponíveis devido à capacidade de diferenciar produtos com diferenças mínimas entre si, e a sua imunidade a alterações no aspeto das embalagens dos produtos. Além disso, a metodologia proposta não requer nenhuma interação prévia do utilizador para funcionar corretamente.[1] Chanjin Chung, Todd M. Schmit, Diansheng Dong, and Harry M. Kaiser. Economic evaluation of shelf-space management in grocery stores. Agribusiness, 23(4):583-597, September 2007.[2] L. Rosado, J. Gonçalves, J. Costa, D. Ribeiro, and F. Soares. Supervised learning for out-of-stock detection in panoramas of retail shelves. In 2016 IEEE International Conference on Imaging Systems and Techniques (IST), pages 406-411.With the increasing care of retail shop owners in improving sales and costumer experience, there is a need to develop technology in order to optimize their goals. It's proven that a planned product placement can boost sales and improve costumer experience [1]. With this in mind, Fraunhofer Portugal came up with ShopView [2] solution to help retail shops extract, validate and manipulate planograms from high-resolution images of the real shelves in the store.In this sense, this thesis focused on the creation of an algorithm using computer vision algorithms to extract information from high resolution images of retail shelves taken with ShopView solution. Particularly, it was implemented pre-processing steps to improve the efficiency and accuracy of an OCR (Optical Character Recognition) engine in recognizing the text in the shelves products. These pre-processing algorithms comprise of denoising and segmentation techniques. The use of this OCR engine brings additional information about products and shelves separation, this information is later used in clustering algorithms to automatically extract an accurate planogram from shelves photos. The presented algorithm is capable of extracting relevant information from the shelves images, to identify the existing products and create a valid metadata about them and their location. With this metadata, it is possible to create, validate and modify the planogram in ShopView. The use of OCR on this algorithm has advantages over other available approaches due to its capability to differentiate products with minimal visual differences and its immunity to appearance changes on the products packaging. Moreover, the methodology proposed does not require any previous user interaction to work properly.[1] Chanjin Chung, Todd M. Schmit, Diansheng Dong, and Harry M. Kaiser. Economic evaluation of shelf-space management in grocery stores. Agribusiness, 23(4):583-597, September 2007.[2] L. Rosado, J. Gonçalves, J. Costa, D. Ribeiro, and F. Soares. Supervised learning for out-of-stock detection in panoramas of retail shelves. In 2016 IEEE International Conference on Imaging Systems and Techniques (IST), pages 406-411

Improving replenishment practices at the store level to minimize out-of-stock levels: a case study in a portuguese grocery retailer

The extent of the out-of-stocks (OOS) in the European grocery retail industry continues to be a major problem, ranging from 7% to 10%, reflecting poor on-shelf availability (OSA), which is a key performance indicator of customer service. This calls for the need for solutions to overcome this pressing issue, as it has consequences for all supply chain actors. The main causes of OOS are originated at the store level, being those related to ordering, replenishment and planning practices. The following research emphasizes the importance of addressing the leading causes of poor replenishment practices in the context of a hypermarket of a Portuguese grocery retailer, to which lack of replenishment represents the cause that contributes the most to OOS. The case study followed a qualitative approach, where multiple data collection techniques were used, such as direct observation, interviews, and data collected from the company, aiming at identifying the main problems and inefficiencies with the current procedures taking place at the store, in order to make improvement suggestions to address the problems identified, so the store can reduce the current OOS rate motivated by lack of replenishment. After understanding the store’s operations and the causes of the poor replenishment practices, some solutions were proposed. These solutions can be summarized in the need of improving planogram compliance by creating awareness of the importance of OSA, training the employees to improve their performance over the daily activities, redesigning the shelf replenishment process, and introducing technology-based solutions for OOS detection.Na Europa, o problema das roturas na indústria do retalho alimentar continua a ser bastante significativo, variando entre os 7% e 10%. Este é um reflexo da indisponibilidade de producto nas prateleiras, um indicador-chave na avaliação do serviço ao cliente. Este problema remete para a necessidade de encontrar soluções devido às consequências negativas para todos os elementos da cadeia de abastecimento. As principais causas das roturas encontram-se na loja, estando relacionadas com as práticas de colocação de encomenda, reposição e planeamento. A presente investigação enfatiza a importância de abordar as principais causas dos problemas relacionados com as práticas de reposição num hipermercado de um retalhista português, para o qual a falta de reposição representa a principal causa de roturas. O caso de estudo caso seguiu uma abordagem qualitativa, onde foram utilizadas várias técnicas de recolha de dados, como observação direta, entrevistas, e dados da empresa, com o objetivo de identificar os principais problemas e ineficiências nos processos em vigor, e propor respetivas melhorias, de modo a que a loja possa reduzir a taxa de roturas por falta de reposição. Após a compreensão das operações da loja e das causas relacionadas com as práticas de reposição, foram propostas algumas soluções. Estas soluções passam pela necessidade de melhorar o cumprimento do planograma através da sensibilização para a importância da disponibilidade de produto, da formação dos funcionários para melhorar o seu desempenho, do redesenho do processo de reposição, e da introdução de soluções tecnológicas para a deteção de roturas

Machine Learning in Sensors and Imaging

Machine learning is extending its applications in various fields, such as image processing, the Internet of Things, user interface, big data, manufacturing, management, etc. As data are required to build machine learning networks, sensors are one of the most important technologies. In addition, machine learning networks can contribute to the improvement in sensor performance and the creation of new sensor applications. This Special Issue addresses all types of machine learning applications related to sensors and imaging. It covers computer vision-based control, activity recognition, fuzzy label classification, failure classification, motor temperature estimation, the camera calibration of intelligent vehicles, error detection, color prior model, compressive sensing, wildfire risk assessment, shelf auditing, forest-growing stem volume estimation, road management, image denoising, and touchscreens

Managing out-of-stocks and over-stock occurrences in supermarket stores: a case study in Singapore

Despite over 40 years of research on out-of-stock (OOS) and over-stock (OS) occurrences, OOS rates remain at an average of 8%. Further, while the store has been found to be a major contributor to OOS situations, it continues to remain a ‘black-box’ in OOS research, especially at the operational level. This thesis examines how supermarket stores execute in-store processes to manage OOS and OS events before, during and after their occurrences. It adopted the case study approach to investigate four specific in-store operations practices – planning and ordering, receiving and checkout, storage, and shelf replenishment - of 19 stores of a major supermarket chain in Singapore. Using semi-structured interviews supplemented by unobtrusive on-site observations of live in-store processes, this study found that OOS and OS occurrences were generally attributable to mismanagement of logistical processes, especially failure to deal with trivial operational issues and minor human errors on-time. Store managers’ attitudes toward enforcement of standard operations procedures (SOPs) also played a significant role in minimizing OOS and OS occurrences in-store. Contrasting the manner in which low-OOS and high-OOS stores handled OOS and OS events, this study unearthed five specific approaches the case supermarket stores used, depending on the in-store retailing dynamics at the time and store management’s knowledge of the causes of their occurrence. From a theoretical perspective, findings from this study have provided a theoretical thread, linking the relationships between store management commitment toward OOS and OS events and OOS and OS performance. They also bring many of the well-documented OOS and OS measures from a broad strategic dimension to the detailed operational level. From a practical standpoint, these findings offer four major Despite over 40 years of research on out-of-stock (OOS) and over-stock (OS) occurrences, OOS rates remain at an average of 8%. Further, while the store has been found to be a major contributor to OOS situations, it continues to remain a ‘black-box’ in OOS research, especially at the operational level. This thesis examines how supermarket stores execute in-store processes to manage OOS and OS events before, during and after their occurrences. It adopted the case study approach to investigate four specific in-store operations practices – planning and ordering, receiving and checkout, storage, and shelf replenishment - of 19 stores of a major supermarket chain in Singapore. Using semi-structured interviews supplemented by unobtrusive on-site observations of live in-store processes, this study found that OOS and OS occurrences were generally attributable to mismanagement of logistical processes, especially failure to deal with trivial operational issues and minor human errors on-time. Store managers’ attitudes toward enforcement of standard operations procedures (SOPs) also played a significant role in minimizing OOS and OS occurrences in-store. Contrasting the manner in which low-OOS and high-OOS stores handled OOS and OS events, this study unearthed five specific approaches the case supermarket stores used, depending on the in-store retailing dynamics at the time and store management’s knowledge of the causes of their occurrence. From a theoretical perspective, findings from this study have provided a theoretical thread, linking the relationships between store management commitment toward OOS and OS events and OOS and OS performance. They also bring many of the well-documented OOS and OS measures from a broad strategic dimension to the detailed operational level. From a practical standpoint, these findings offer four major sets of best-practice guidelines on OOS and OS management that relates to the role of store managers, adherence to SOPs, supplier relationship management and effects of contextual factors

Emerging technology influences on the merchandise practises of a retailer: a study of Massbuild South Africa.

Masters Degree. University of KwaZulu-Natal, Durban.Background: Retailers across the globe are embracing technological advancements in their merchandise and distribution processes. Technology is changing the way every retailer conducts business by helping to create efficiencies, save money, and provide better products and services. Retail companies are also adopting technology to their advantage. Purpose: The purpose of this research is to understand the impact newer technology implementation has on a retailer’s processes, specifically regarding merchandise and distribution. The study examined the current technology available to Massbuild and how these factors impact its daily processes. A prominent challenge in retail is the implementation phase of adopting newer technology, which requires management decision-making. Methodology: This research study is exploratory. The methodology was qualitative and utilised a semi-structured in-depth interview approach with twelve senior management employees at Massbuild. A purposive sampling method was used to help select participants who fit the criterion. The empirical findings provide insightful and vital information on the benefits and challenges of technology on merchandise and distribution processes. Findings: The research findings highlight the emerging technologies that will help a retailer focus on improving existing merchandise and distribution processes. The research participants interviewed emphasised particular technologies used by Massbuild and its evolutionary change over the past ten years. There has been a strong emphasis on automation, artificial intelligence and assortment optimisation within Massbuild. Regarding implementation of technology, interview participants provided insight into possible solutions to challenges they encounter within their respective employment roles. Contribution: Technology is at the forefront of retail and is continually evolving. There are in-depth studies available regarding technology in retail, especially with the influence of the fourth industrial revolution. This research provides fresh insights into the retail field of merchandise and distribution, and provides fruitful insight for future researchers

The adoption of lean techniques to optimise the on-shelf availability of products and drive business performance in the food industry: a South African manufacturing and retail case study

Includes bibliographical references.The degree of sustaining business performance, while maintaining competitive costs, satisfied consumers and customers has become more difficult and harder to achieve. To date, both retailers and manufacturers are economically challenged as they enter into a new age and era that is characterised by a restructuring of the supply and demand known today, the one in which the consumer demand chain will both lead and direct all organisational processes. The greatest challenge in manufacturing and retail supply chains today continue to be the inconsistency of product availability. Both retailers and their manufacturers frequently find themselves in positions where they either have too much stock of specific stock-keeping units (SKUs) or insufficient stock levels of a particular SKU, Steve (2010). Retailers and their suppliers both seek to avoid the costly out-of-stock (OOS) situations, which result in lost revenue opportunity for both parties. OOS can also damage shopper loyalty as frustrated consumers might seek out alternative retailers for the same merchandise, while on the other hand suppliers' brand loyalty can be impacted if a competitor's product is substituted instead. It remains true that the two pillars of business, namely demand and supply, still rule. Traditionally, putting supply before demand, with its implied precedence, was the correct approach to apply, but in today's business environment, there is a major shift taking place, predominantly driven by the cycles in globalisation that would be faster than in the traditional way, oversupply in the fast -moving consumer goods industry, a parallel loss of pricing power, consumers with a twenty-four hours access to precise pricing information, which terminates the power of information scarcity, and shorter product life cycles. The global economic crash that represented a global economic storm led many organisations to rethink the manner in which organisations are led. A consensus exists among many authors and commentators that the emerging economic order has imposed changes to the very way companies are doing business

Planogram compliance checking using recurring patterns

This paper proposes a novel automated planogram compliance checking method for retail chains without requiring product template images for modeling or training. Product layout information is extracted from one single input image by means of unsupervised recurring pattern detection and matched via graph matching with the expected product layout specified by a planogram. To improve the efficiency, a divide-conquer strategy is employed. Specifically, the input image is divided into several regions based on the planogram. Recurring patterns are detected in each region respectively and then merged together to estimate product layout information. Experimental results on real data from a supermarket chain have verified the effectiveness and efficiency of the proposed method