A review of the current trends and future directions of camera barcode reading

Modern mobile phones or smartphones have become a pervasive and affordable device for users at different levels of age around the world. Smartphones equipped with many useful sensors, including camera, barometer, accelerometer, and digital compass. The sensors on smartphones attracted researchers and developers to develop mobile applications (apps) and study the potential use of the sensors to support daily life activities. Unlike other types of sensor, the smartphone camera has been underutilized. Analysis of the literature suggested that smartphone camera mainly serves for personal and social photography. Practically, a smartphone camera can be used as an imaging device for reading a barcode. Although barcode has been used for identifying products and items, the use of a smartphone camera as a reading device has not been explored thoroughly. Further, scholarly resources describing the fundamental knowledge of smartphone camera barcode reading is not available in the literature which could be the reason contributed to slow research progress of the domain. Therefore, this study aims to review the current trends and future directions of smartphone camera for barcode reading. Specifically, the study reviews the literature on the types of applications that are currently available and run on the standard mobile platform for reading a barcode. It also analyzes the necessary components that made up barcode reading apps. Further, the review identifies technical and non-technical issues that are critical for the development of the apps. The contributions of this work are twofold, first, it provides the fundamental knowledge on the building blocks of camera barcode reading apps, and second, it explores the issues in the current camera barcode reading apps that could encourage exploration towards addressing the issues. Practically, the findings could spark new research ideas to address the current issues related to the use of smartphone camera for barcode reading in the near future

Usage of Hough Transform for Expiry Date Extraction via Optical Character Recognition

In this article we evaluate the impact of using two image pre-processing approaches with the objective of aiding an Optical Character Recognition (OCR) software in correctly retrieving an expiry date from an image of a product containing it. In particular, we analyze the impact of finding the rotation angle of an image using the Hough transform and the impact of image binarization using adaptive Gaussian threshold. We attempt to further increase OCR accuracy through a sliding window approach. Our results show that applying the Hough transform noticeably improves OCR performance with minimal impact on the execution time

Enhancing pharmaceutical packaging through a technology ecosystem to facilitate the reuse of medicines and reduce medicinal waste

The idea of reusing dispensed medicines is appealing to the general public provided its benefits are illustrated, its risks minimized, and the logistics resolved. For example, medicine reuse could help reduce medicinal waste, protect the environment and improve public health. However, the associated technologies and legislation facilitating medicine reuse are generally not available. The availability of suitable technologies could arguably help shape stakeholders’ beliefs and in turn, uptake of a future medicine reuse scheme by tackling the risks and facilitating the practicalities. A literature survey is undertaken to lay down the groundwork for implementing technologies on and around pharmaceutical packaging in order to meet stakeholders’ previously expressed misgivings about medicine reuse (’stakeholder requirements’), and propose a novel ecosystem for, in effect, reusing returned medicines. Methods: A structured literature search examining the application of existing technologies on pharmaceutical packaging to enable medicine reuse was conducted and presented as a narrative review. Results: Reviewed technologies are classified according to different stakeholders’ requirements, and a novel ecosystem from a technology perspective is suggested as a solution to reusing medicines. Conclusion: Active sensing technologies applying to pharmaceutical packaging using printed electronics enlist medicines to be part of the Internet of Things network. Validating the quality and safety of returned medicines through this network seems to be the most effective way for reusing medicines and the correct application of technologies may be the key enabler

Accessible Cosmetic Packaging for Visually Impaired and Blind Users

Visually impaired and blind users often find it challenging to identify cosmetic products, and colours and read the label information because of the lack of accessible cosmetic packaging. The goal of this study was to understand the characteristics of and design opportunities for accessible packaging. Four research questions were examined: 1) What features of the packaging design will make it easier to distinguish between various cosmetic products? 2) How may packaging be made to provide Visually impaired/Blind consumers with access to product label information? 3) How do Visually impaired/Blind individuals conceptualize colour and how may this be incorporated into product design/packaging? 4) What application can be developed to aid and enhance the user experience of existing cosmetic packaging? To answer these questions inclusive design framework was adopted, and a co-design methodology was used. Three co-designers took part in the study. Findings were grouped into two themes 1) Characteristics of packaging required to improve user's experience and 2)Characteristics that would minimize the need for users to seek advice from others prior to product selection or use. Findings were used 1) to create an accessible packaging checklist and 2) to design an app that helps visually impaired and blind users identify products, and colours and get expert feedback on how the make-up is applied. The application is named Beauty Ally. The checklist will help brands develop accessible cosmetic packaging for visually impaired and blind consumers. The app will enhance the user experience of visually impaired and blind consumers with existing packaging by helping them identify products, read label information, and colours and seek advice from volunteers

Vision Based Extraction of Nutrition Information from Skewed Nutrition Labels

An important component of a healthy diet is the comprehension and retention of nutritional information and understanding of how different food items and nutritional constituents affect our bodies. In the U.S. and many other countries, nutritional information is primarily conveyed to consumers through nutrition labels (NLs) which can be found in all packaged food products. However, sometimes it becomes really challenging to utilize all this information available in these NLs even for consumers who are health conscious as they might not be familiar with nutritional terms or find it difficult to integrate nutritional data collection into their daily activities due to lack of time, motivation, or training. So it is essential to automate this data collection and interpretation process by integrating Computer Vision based algorithms to extract nutritional information from NLs because it improves the user’s ability to engage in continuous nutritional data collection and analysis. To make nutritional data collection more manageable and enjoyable for the users, we present a Proactive NUTrition Management System (PNUTS). PNUTS seeks to shift current research and clinical practices in nutrition management toward persuasion, automated nutritional information processing, and context-sensitive nutrition decision support. PNUTS consists of two modules, firstly a barcode scanning module which runs on smart phones and is capable of vision-based localization of One Dimensional (1D) Universal Product Code (UPC) and International Article Number (EAN) barcodes with relaxed pitch, roll, and yaw camera alignment constraints. The algorithm localizes barcodes in images by computing Dominant Orientations of Gradients (DOGs) of image segments and grouping smaller segments with similar DOGs into larger connected components. Connected components that pass given morphological criteria are marked as potential barcodes. The algorithm is implemented in a distributed, cloud-based system. The system’s front end is a smartphone application that runs on Android smartphones with Android 4.2 or higher. The system’s back end is deployed on a five node Linux cluster where images are processed. The algorithm was evaluated on a corpus of 7,545 images extracted from 506 videos of bags, bottles, boxes, and cans in a supermarket. The DOG algorithm was coupled to our in-place scanner for 1D UPC and EAN barcodes. The scanner receives from the DOG algorithm the rectangular planar dimensions of a connected component and the component’s dominant gradient orientation angle referred to as the skew angle. The scanner draws several scan lines at that skew angle within the component to recognize the barcode in place without any rotations. The scanner coupled to the localizer was tested on the same corpus of 7,545 images. Laboratory experiments indicate that the system can localize and scan barcodes of any orientation in the yaw plane, of up to 73.28 degrees in the pitch plane, and of up to 55.5 degrees in the roll plane. The videos have been made public for all interested research communities to replicate our findings or to use them in their own research. The front end Android application is available for free download at Google Play under the title of NutriGlass. This module is also coupled to a comprehensive NL database from which nutritional information can be retrieved on demand. Currently our NL database consists of more than 230,000 products. The second module of PNUTS is an algorithm whose objective is to determine the text skew angle of an NL image without constraining the angle’s magnitude. The horizontal, vertical, and diagonal matrices of the (Two Dimensional) 2D Haar Wavelet Transform are used to identify 2D points with significant intensity changes. The set of points is bounded with a minimum area rectangle whose rotation angle is the text’s skew. The algorithm’s performance is compared with the performance of five text skew detection algorithms on 1001 U.S. nutrition label images and 2200 single- and multi-column document images in multiple languages. To ensure the reproducibility of the reported results, the source code of the algorithm and the image data have been made publicly available. If the skew angle is estimated correctly, optical character recognition (OCR) techniques can be used to extract nutrition information

Recent advancements in machine vision methods for product code recognition: A systematic review [version 1; peer review: 2 approved]

Background: Manufacturing markings printed on products play an important role in the handling and use of pharmaceuticals and perishable foods. Currently, optical character recognition, neural networks, deep learning-based methods, and combinations of these methods are used to recognize these codes. Methods: This systematic review was performed to find papers that can answer the following research questions: How have machine vision methods that can recognize product texts evolved over the past eight years? What are the most common difficulties in recognizing product texts? Articles published between 2012 and 2020 were systematically searched from Science Direct/SCOPUS, and Google Scholar in November-December 2020. Ten studies were eligible, with inclusion criteria: details about the recognition method used, performance analysis result, imaging method, product and the text printed on it. Results: Product text recognition methods have evolved significantly over the last two years to tolerate the most common difficulties in the field. This is due to the ability of the deep learning neural network (DNN) architectures such as convolutional neural networks (CNN) to extract and learn salient character features straight from packaging surface images. Four of the most recent methods use two consecutive deep learning networks, one detecting the text area based on an image captured from the product package's surface and the other recognizing the characters within. Furthermore, this paper presents solutions to the most common product text recognition difficulties. Conclusions: There were a limited number of studies that met the eligibility criteria for this systematic review. The study's aim was to evaluate the development of machine vision methods for recognizing manufacturing marking texts printed on the surface of products. The study results demonstrated how methods have evolved over time, beginning with optical character recognition, and advancing to methods which can recognize texts despite the field's most common problems

Near Field Communication Applications

Near Field Communication (NFC) is a short-range, low power contactless communication between NFC-enabled devices that are held in the closed proximity to each other. NFC technology has been moving rapidly from its initial application areas of mobile payment services and contactless ticketing to the diversity of new areas. Three specific NFC tags highlighted in the thesis have different structures in terms of memory, security and usage in different applications. NFC information tags exploit the data exchange format NDEF standardized by NFC Forum. NFC applications are rapidly stepping into novel and diverse application areas. Often they are deployed in combination with different devices and systems through their integrability and adaptability features. The diverse application areas where NFC tags and cards are used cover smart posters, contactless ticketing, keys and access control, library services, entertainment services, social network services, education, location based services, work force and retail management and healthcare. In designing different NFC applications, it is necessary to take into consideration different design issues such as to choosing the NFC tools and devices according to the technical requirements of the application, considering especially the memory, security and price factors as well as their relation to the purpose and usage of the final product. The security aspect of the NFC tags is remarkably important in selecting the proper NFC device. The race between hackers attacking and breaking the security systems of programmable high level products and manufacturers to produce reliable secure systems and products seems to never end. This has proven to be case, for example, for trying MIFARE Ultralight and DESFire MF3ICD40 tags. An important consideration of studying the different applications of NFC tags and cards during the thesis work was to understand the ubiquitous character of NFC technology.Lähitunnistus yhteys tekniikka (NFC) on lyhyen tähtäimen, pienitehoinen, kontaktiton yhteydenpito NFC yhteensopivien laitteiden välillä, jossa laitteet pidetään toistensä välittömässä läheisyydessä tiedon siirtämiseksi niiden välillä. NFC-teknologia on siirtynyt nopeasti sen alkuperäisiltä toimialueilta eli mobiili maksupalvelujen ja kontaktittomien lippujen sovellusalueilta moninaisille uusille alueille. Kolmella NFC tagillä, joita on käsitelty tässä tutkielmassa, on muistin, turvallisuuden ja käytön kannalta erilaisiä rakenteita, joita käytetään eri sovelluksissa. NFC-tagit käyttävät tiedonvälityksessä NFC Forumin standardoimaa NDEF-tiedonvaihtoformaattia. NFC sovellukset esiintyvät yhä enenevässä määrin nopeasti kehyttyvillä, uudenlaisilla ja monipuolisilla sovellusalueilla, usein yhdessä eri laitteiden ja järjestelmien kanssa. NFC on käytettävissä erinäisten laitteiden kanssa erilaisissa järjestelmäympäristöissä. Monipuoliset sovellusalueet, joissa muun muassa NFC-tagejä ja -kortteja käytetään sisältävät seuraavanlaisia sovelluksia: älykkäät julisteet, kontaktittomat liput, avaimet ja pääsynvalvonta, kirjastopalvelut, viihdepalvelut, sosiaalisen verkoston palvelut, kasvatukseen ja koulutukseen liittyvät palvelut, sijaintiperustaiset palvelut, työvoiman ja vähittäiskaupan hallinto-palvelut ja terveyspalvelut. Erilaisten NFC-sovelluksien suunnittelussa on väistämätöntä ottaa erilaisia suunnitteluasioita huomioon kuten valita NFC-työkalut ja laitteet sovelluksen teknisten vaatimusten mukaan. Erilaiset tärkeät tekijät kuten muisti, tietoturvallisuusominaisuudet ja hinta ja niiden kaikkien toimivuus lopputuotteen kannalta on otettava huomioon. Tietoturvallisuusnäkökohta on erityisen tärkeä oikean NFC laitteen valitsemisessa, sillä käynnissä on loputon kilpajuoksu hakkerien, jotka yrittävät rikkoa ohjelmoitavien korkeatasoisten laitteiden ja tuotteiden tietoturvajärjestelmiä, ja valmistajien, jotka pyrkivät tuottamaan luotettavia varmoja järjestelmiä, välillä. Tietoturvariskiin liittyviä ongelmia on löydetty esimerkiksi MIFARE Ultralight ja DESFire MF3ICD40 tageista. Tärkeä havainto, joka saatiin erilaisten NFC sovelluksien tutkimisesta, oli oivaltaa NFCteknologian potentiaalinen kaikkialle ulottuva, yleiskäyttöinen luonne

Medicines Reuse

This reprint examines the concept of medicines reuse, the idea that unused medication returned by one patient can be re-dispensed for use by another. Ten papers written by over 20 authors examine a range of issues related to medicines reuse including the circular economy of the pharmaceutical supply chain; the prevalence of unused medicines or medication waste within patients' homes; people's views about the causes of medication waste and the potential for medicines reuse; what might influence people to reuse medicines in the future; how sensing technologies might facilitate medicines reuse; and the effect of including sensing technologies on people's willingness to consider medicines reuse