Autonomous Removal of Perspective Distortion for Robotic Elevator Button Recognition

Elevator button recognition is considered an indispensable function for enabling the autonomous elevator operation of mobile robots. However, due to unfavorable image conditions and various image distortions, the recognition accuracy remains to be improved. In this paper, we present a novel algorithm that can autonomously correct perspective distortions of elevator panel images. The algorithm first leverages the Gaussian Mixture Model (GMM) to conduct a grid fitting process based on button recognition results, then utilizes the estimated grid centers as reference features to estimate camera motions for correcting perspective distortions. The algorithm performs on a single image autonomously and does not need explicit feature detection or feature matching procedure, which is much more robust to noises and outliers than traditional feature-based geometric approaches. To verify the effectiveness of the algorithm, we collect an elevator panel dataset of 50 images captured from different angles of view. Experimental results show that the proposed algorithm can accurately estimate camera motions and effectively remove perspective distortions

A Large-Scale Dataset for Benchmarking Elevator Button Segmentation and Character Recognition

Human activities are hugely restricted by COVID-19, recently. Robots that can conduct inter-floor navigation attract much public attention, since they can substitute human workers to conduct the service work. However, current robots either depend on human assistance or elevator retrofitting, and fully autonomous inter-floor navigation is still not available. As the very first step of inter-floor navigation, elevator button segmentation and recognition hold an important position. Therefore, we release the first large-scale publicly available elevator panel dataset in this work, containing 3,718 panel images with 35,100 button labels, to facilitate more powerful algorithms on autonomous elevator operation. Together with the dataset, a number of deep learning based implementations for button segmentation and recognition are also released to benchmark future methods in the community. The dataset will be available at \url{https://github.com/zhudelong/elevator_button_recognitio

Autonomous Removal of Perspective Distortion based on Detection Results of Robotic Elevator Button Corner

Elevator button recognition is an important function to realize the autonomous operation of elevators. However, challenging image conditions and various image distortions make it difficult to accurately recognize buttons. In this work, We propose a novel algorithm that can automatically correct perspective distortions of elevator panel images based on button corner detection results. The algorithm first leverages DeepLabv3+ model and Hough Transform method to obtain button segmentation results and button corner detection results, then utilizes pixel coordinates of standard button corners as reference features to estimate camera motions for correcting perspective distortions. The algorithm is much more robust to outliers and noise on the removal of perspective distortion than traditional geometric approaches as it only performs on a single image autonomously. 15 elevator panel images are captured from different angles of view as the dataset. The experimental results show that our approach significantly outperforms traditional geometric techniques in accuracy and robustness. Rectification results of the proposed algorithm is 77.4% better than the results of traditional geometric algorithm in average