AI-Based Analytics for Hawkers Identification in Video Surveillance for Smart Community

Street hawking is a widespread phenomenon in urban areas globally, presenting challenges for local authorities such as traffic congestion, waste management, and negative impacts on the city's image. This research addresses key issues faced by authorities in managing hawkers, including the resistance to formalization, maintaining urban aesthetics, waste disposal, and understanding user preferences. The study investigates the performance of the You Only Look Once (YOLO) algorithm, utilizing Convolutional Neural Networks (CNN) for real-time object detection. To achieve thisobjective, the YOLOv5 algorithm is trained with a custom image dataset collected from the same camera along the street in the city area to detect five classes of objects, namely umbrella, table, stool, car, and people. Real images that were captured via camera and video surveillance were compiled as datasets which are then used to train and test the algorithm. The study aims to provide insights into the data collection process of hawkers along the street around the areas and the development of real-time hawker detection for the smart city application

Research on product detection and recognition methods for intelligent vending machines

With the continuous development of China's economy and the improvement of residents' living standards, it also brings increasing costs of labor and rent. In addition, the impact of the pandemic on the entity industry has brought opportunities for the development of new retail models. Based on the booming development of artificial intelligence, big data, and mobile payment in the new era, the new retail industry using artificial intelligence technology has shown outstanding performance in the market. Among them, intelligent vending machines have emerged in the new retail model. In order to provide users with a good shopping experience, the product detection speed and accuracy of intelligent vending machines must be high enough. We adopt Faster R-CNN, a mature object detection algorithm in deep learning, to solve the commodity settlement scenario of intelligent vending machines

Analyzing computer vision models for detecting customers: a practical experience in a mexican retail

Computer vision has become an important technology for obtaining meaningful data from visual content and providing valuable information for enhancing security controls, marketing, and logistic strategies in diverse industrial and business sectors. The retail sector constitutes an important part of the worldwide economy. Analyzing customer data and shopping behaviors has become essential to deliver the right products to customers, maximize profits, and increase competitiveness. In-person shopping is still a predominant form of retail despite the appearance of online retail outlets. As such, in-person retail is adopting computer vision models to monitor store products and customers. This research paper presents the development of a computer vision solution by Lytica Company to detect customers in Steren’s physical retail stores in Mexico. Current computer vision models such as SSD Mobilenet V2, YOLO-FastestV2, YOLOv5, and YOLOXn were analyzed to find the most accurate system according to the conditions and characteristics of the available devices. Some of the challenges addressed during the analysis of videos were obstruction and proximity of the customers, lighting conditions, position and distance of the camera concerning the customer when entering the store, image quality, and scalability of the process. Models were evaluated with the F1-score metric: 0.64 with YOLO FastestV2, 0.74 with SSD Mobilenetv2, 0.86 with YOLOv5n, 0.86 with YOLOv5xs, and 0.74 with YOLOXn. Although YOLOv5 achieved the best performance, YOLOXn presented the best balance between performance and FPS (frames per second) rate, considering the limited hardware and computing power conditions

Centralised and decentralised sensor fusion‐based emergency brake assist

Copyright: © 2021 by the authors. Many advanced driver assistance systems (ADAS) are currently trying to utilise multi-sensor architectures, where the driver assistance algorithm receives data from a multitude of sen-sors. As mono‐sensor systems cannot provide reliable and consistent readings under all circum-stances because of errors and other limitations, fusing data from multiple sensors ensures that the environmental parameters are perceived correctly and reliably for most scenarios, thereby substan-tially improving the reliability of the multi‐sensor‐based automotive systems. This paper first high-lights the significance of efficiently fusing data from multiple sensors in ADAS features. An emergency brake assist (EBA) system is showcased using multiple sensors, namely, a light detection and ranging (LiDAR) sensor and camera. The architectures of the proposed ‘centralised’ and ‘decentral-ised’ sensor fusion approaches for EBA are discussed along with their constituents, i.e., the detection algorithms, the fusion algorithm, and the tracking algorithm. The centralised and decentralised architectures are built and analytically compared, and the performance of these two fusion architectures for EBA are evaluated in terms of speed of execution, accuracy, and computational cost. While both fusion methods are seen to drive the EBA application at an acceptable frame rate (~20fps or higher) on an Intel i5‐based Ubuntu system, it was concluded through the experiments and analyt-ical comparisons that the decentralised fusion‐driven EBA leads to higher accuracy; however, it has the downside of a higher computational cost. The centralised fusion‐driven EBA yields compara-tively less accurate results, but with the benefits of a higher frame rate and lesser computational cost

Dominant Feature Pooling for Multi Camera Object Detection and Optimization of Retinex Algorithm

학위논문(박사) -- 서울대학교대학원 : 공과대학 전기·정보공학부, 2021.8. 이혁재.본 논문은 멀티 카메라 object detection CNN을 위한 detection 단계에서 활용하는 새로운 dominant feature pooling 방법을 제안한다. 멀티 카메라 시스템은 다양한 관점에서 물체의 이미지를 캡처하고, 물체의 더 많은 주요 feature를 detection에 활용할 수 있다. 따라서 여러 카메라에서 feature를 pooling하면 detection 정확도를 향상시킬 수 있다. 제안된 방법은 객체의 다양한 뷰포인트에서 얻은 feature vector 중에서 더 많은 정보를 제공하는 주요 feature을 선택하고 선택한 feature vector를 pooling하여 새로운 feature map을 구성한다. 제안된 방법은 단일 카메라에 대한 YOLOv3 네트워크를 기반으로 하며, 멀티 카메라 시스템에 대한 추가 학습 과정이 필요하지 않다. Dominant feature pooling의 효과를 주장하기 위해, 이 연구에서는 feature vector를 시각화하는 새로운 방법도 제안된다. 또한 object detection CNN은 저조도 환경에 대응이 취약하므로 이를 개선할 수 있는 Retinex 알고리즘의 활용 방법을 제안한다. 저조도 영상을 그대로 학습하여 개선을 할 수 있지만, 실 사용 환경에서 조도 정도를 예측할 수 없기 때문에 Retinex 개선이 필수적임을 실험을 통해 나타내었다. 또한 개선 효과가 뚜렷하지만 복잡도가 높은 Retinex 알고리즘을 HW 설계를 통해 최적화 하는 방법을 제안한다. Retinex 알고리즘 연산에 필수적인 exponentiation과 Gaussian filtering을 효율적으로 구현하는 방법을 제안하여 높은 해상도에서도 실시간으로 동작이 가능한 HW를 구현하였다.This paper proposes a novel dominant feature pooling method utilized in the detection phase for multi-camera object detection CNNs. Multi-camera systems can capture images of objects from various perspectives and utilize more of the important features of objects for detection. Thus, the detection accuracy can be improved by pooling the features of the multiple cameras. The proposed method constructs a new feature patch by selecting and pooling the dominant features that provides more information among the feature vectors obtained from various viewpoints of objects. The proposed method is based on the YOLOv3 network for a single camera, and does not require additional learning processes for multi-camera systems. To show the effectiveness of dominant feature pooling, a novel method of visualizing feature vectors is also proposed in this work. Furthermore, a method of utilizing Retinex algorithms that can improve response to low-light environments for object detection CNN is proposed. Although improvements can be made by learning low-light images as they are, experimental results show that Retinex improvements are essential because the degree of illumination cannot be predicted accurately to create new datasets in practical environments. This work proposes a method to optimize Retinex algorithms through HW designs. An efficient implementation of the exponentiation operation and the Gaussian filtering, which are essential for Retinex algorithm operations is proposed to implement HW that can operate in real time at high resolution.제 1 장 서 론 1 1.1 연구 배경 1 1.2 연구 내용 2 1.3 논문 구성 4 제 2 장 배경 이론 및 관련 연구 5 2.1 Object Detection CNN 5 2.2 Multi View CNN 6 2.3 Retinex 알고리즘 7 2.3.1 Retinex Algorithm using Gaussian Filter 8 2.3.2 Multiscale Retinex Algorithm 9 2.3.3 Efficient Naturalness Restoration 10 제 3 장 무인 판매대 시스템 12 3.1 무인 판매대 시스템 개요 12 3.2 Object Detection CNN을 활용한 상품 인식 16 3.3 Multi-Object Tracking을 활용한 상품 구매 판단 18 3.4 무인 판매대의 실시간 동작을 위한 최적화 방안 20 3.4.1 카메라 선택 알고리즘 20 3.4.2 Multithreading 24 3.4.3 Pruning 25 3.5 무인 판매대 시스템 성능 평가 27 3.5.1 Object Detection 성능 평가 27 3.5.2 무인 판매대 시스템 전체 결과 29 제 4 장 멀티 카메라 Dominant Feature Pooling 32 4.1 Object Detection CNN과 멀티 카메라 Object Clustering 33 4.1.1 Object Detection CNN 33 4.1.2 멀티 카메라 Object Clustring 35 4.2 Dominant Feature Pooling 방법 37 4.2.1 Dominant Feature Scoring 40 4.2.2 Dominant Feature Pooling 47 4.2.3 YOLOv3의 Detection Layer 재사용 50 4.3 Feature 시각화를 통한 제안 방법 분석 52 4.3.1 제안하는 Feature 시각화 방법 52 4.3.2 기존 단일 카메라 YOLOv3의 Feature 시각화 55 4.3.3 제안하는 방법의 멀티카메라 Feature 시각화 57 4.4 Dominant Feature Pooling 결과 및 분석 59 4.4.1 COCO Dataset에서의 결과 60 4.4.2 Custom Dataset에서의 결과 62 4.4.3 Scoring Method 별 결과 63 4.4.3 Dominant Feature Pooling의 수행시간 결과 64 제 5 장 Retinex Applied Object Detection 및 하드웨서 가속시스템 65 5.1 기존 Retinex 적용 연구 66 5.2 Retinex Applied Object Detection 68 5.2.1 Retinex Applied Object Detection 학습 68 5.2.2 Retinex Applied Object Detection 결과 72 5.3 Object Detection을 위한 Retinex 최적화 76 5.3.1 Gaussian Filter 크기에 따른 Retinex 효과 분석 76 5.3.2 Gaussain Filter 크기에 따른 Object Detection 결과 80 5.4 Retinex 하드웨어 시스템의 필요성 및 기존 연구 82 5.5 제안 하드웨어 시스템 구현 개요 85 5.6 제안 하드웨어 시스템 구현 특장점 89 5.6.1 Gaussian filter의 구현 89 5.6.2 Exponentiation의 구현 96 5.6.3 HDMI/DVI 지원 및 영상 latency 최소화 103 5.7 제안 하드웨어 시스템 구현 결과 및 분석 106 5.7.1 실시간 동작 및 낮은 latency에 대한 분석 106 5.7.2 제안한 시스템의 영상 처리 성능 결과 분석 109 5.7.3 제안한 시스템의 FPGA Resource Utilization 112 5.7.4 다른 시스템과의 Resource Utilization 비교 114 5.7.5 제안한 시스템의 영상 처리 성능 결과 분석 119 제 6 장 결론 120 참고문헌 121 Abstract 131박

Analyzing computer vision models for detecting customers: a practical experience in a mexican retail

Computer vision has become an important technology for obtaining meaningful data from visual content and providing valuable information for enhancing security controls, marketing, and logistic strategies in diverse industrial and business sectors. The retail sector constitutes an important part of the worldwide economy. Analyzing customer data and shopping behaviors has become essential to deliver the right products to customers, maximize profits, and increase competitiveness. In-person shopping is still a predominant form of retail despite the appearance of online retail outlets. As such, in-person retail is adopting computer vision models to monitor store products and customers. This research paper presents the development of a computer vision solution by Lytica Company to detect customers in Steren’s physical retail stores in Mexico. Current computer vision models such as SSD Mobilenet V2, YOLO-FastestV2, YOLOv5, and YOLOXn were analyzed to find the most accurate system according to the conditions and characteristics of the available devices. Some of the challenges addressed during the analysis of videos were obstruction and proximity of the customers, lighting conditions, position and distance of the camera concerning the customer when entering the store, image quality, and scalability of the process. Models were evaluated with the F1-score metric: 0.64 with YOLO FastestV2, 0.74 with SSD Mobilenetv2, 0.86 with YOLOv5n, 0.86 with YOLOv5xs, and 0.74 with YOLOXn. Although YOLOv5 achieved the best performance, YOLOXn presented the best balance between performance and FPS (frames per second) rate, considering the limited hardware and computing power conditions

Deep Learning Detected Nutrient Deficiency in Chili Plant

Chili is a staple commodity that also affects the Indonesian economy due to high market demand. Proven in June 2019, chili is a contributor to Indonesia's inflation of 0.20% from 0.55%. One factor is crop failure due to malnutrition. In this study, the aim is to explore Deep Learning Technology in agriculture to help farmers be able to diagnose their plants, so that their plants are not malnourished. Using the RCNN algorithm as the architecture of this system. Use 270 datasets in 4 categories. The dataset used is primary data with chili samples in Boyolali Regency, Indonesia. The chili we use are curly chili. The results of this study are computers that can recognize nutrient deficiencies in chili plants based on image input received with the greatest testing accuracy of 82.61% and has the best mAP value of 15.57%

Radial Basis Function Neural Network in Identifying The Types of Mangoes

Mango (Mangifera Indica L) is part of a fruit plant species that have different color and texture characteristics to indicate its type. The identification of the types of mangoes uses the manual method through direct visual observation of mangoes to be classified. At the same time, the more subjective way humans work causes differences in their determination. Therefore in the use of information technology, it is possible to classify mangoes based on their texture using a computerized system. In its completion, the acquisition process is using the camera as an image processing instrument of the recorded images. To determine the pattern of mango data taken from several samples of texture features using Gabor filters from various types of mangoes and the value of the feature extraction results through artificial neural networks (ANN). Using the Radial Base Function method, which produces weight values, is then used as a process for classifying types of mangoes. The accuracy of the test results obtained from the use of extraction methods and existing learning methods is 100%