Artificial intelligence approach for tomato detection and mass estimation in precision agriculture

Funding: This study was carried out with the support of “Research Program for Agricultural Science & Technology Development” (Project No: PJ013891012020), National Institute of Agricultural Sciences, Rural Development Administration, Republic of Korea.Application of computer vision and robotics in agriculture requires sufficient knowledge and understanding of the physical properties of the object of interest. Yield monitoring is an example where these properties affect the quantified estimation of yield mass. In this study, we propose an image-processing and artificial intelligence-based system using multi-class detection with instance-wise segmentation of fruits in an image that can further estimate dimensions and mass. We analyze a tomato image dataset with mass and dimension values collected using a calibrated vision system and accurate measuring devices. After successful detection and instance-wise segmentation, we extract the real-world dimensions of the fruit. Our characterization results exhibited a significantly high correlation between dimensions and mass, indicating that artificial intelligence algorithms can effectively capture this complex physical relation to estimate the final mass. We also compare different artificial intelligence algorithms to show that the computed mass agrees well with the actual mass. Detection and segmentation results show an average mask intersection over union of 96.05%, mean average precision of 92.28%, detection accuracy of 99.02%, and precision of 99.7%. The mean absolute percentage error for mass estimation was 7.09 for 77 test samples using a bagged ensemble tree regressor. This approach could be applied to other computer vision and robotic applications such as sizing and packaging systems and automated harvesting or to other measuring instruments.Publisher PDFPeer reviewe

Optimal Total Mesorectal Excision for Rectal Cancer: the Role of Robotic Surgery from an Expert's View

Total mesorectal excision (TME) has gained worldwide acceptance as a standard surgical technique in the treatment of rectal cancer. Ever since laparoscopic surgery was first applied to TME for rectal cancer, with increasing penetration rates, especially in Asia, an unstable camera platform, the limited mobility of straight laparoscopic instruments, the two-dimensional imaging, and a poor ergonomic position for surgeons have been regarded as limitations. Robotic technology was developed in an attempt to reduce the limitations of laparoscopic surgery. The robotic system has many advantages, including a more ergonomic position, stable camera platform and stereoscopic view, as well as elimination of tremor and subsequent improved dexterity. Current comparison data between robotic and laparoscopic rectal cancer surgery show similar intraoperative results and morbidity, postoperative recovery, and short-term oncologic outcomes. Potential benefits of a robotic system include reduction of surgeon's fatigue during surgery, improved performance and safety for intracorporeal suture, reduction of postoperative complications, sharper and more meticulous dissection, and completion of autonomic nerve preservation techniques. However, the higher cost for a robotic system still remains an obstacle to wide application, and many socioeconomic issues remain to be solved in the future. In addition, we need more concrete evidence regarding the merits for both patients and surgeons, as well as the merits compared to conventional laparoscopic techniques. Therefore, we need large-scale prospective randomized clinical trials to prove the potential benefits of robot TME for the treatment of rectal cancer

An N-Modular Redundancy Framework Incorporating Response-Time Analysis on Multiprocessor Platforms

A timing constraint and a high level of reliability are the fundamental requirements for designing hard real-time systems. To support both requirements, the N modular redundancy (NMR) technique as a fault-tolerant real-time scheduling has been proposed, which executes identical copies for each task simultaneously on multiprocessor platforms, and a single correct one is voted on, if any. However, this technique can compromise the schedulability of the target system during improving reliability because it produces N identical copies of each job that execute in parallel on multiprocessor platforms, and some tasks may miss their deadlines due to the enlarged computing power required for completing their executions. In this paper, we propose task-level N modular redundancy (TL-NMR), which improves the system reliability of the target system of which tasks are scheduled by any fixed-priority (FP) scheduling without schedulability loss. Based on experimental results, we demonstrate that TL-NMR maintains the schedulability, while significantly improving average system safety compared to the existing NMR

Efficient Pedestrian Detection at Nighttime Using a Thermal Camera

Most of the commercial nighttime pedestrian detection (PD) methods reported previously utilized the histogram of oriented gradient (HOG) or the local binary pattern (LBP) as the feature and the support vector machine (SVM) as the classifier using thermal camera images. In this paper, we propose a new feature called the thermal-position-intensity-histogram of oriented gradient (TPIHOG or T π HOG) and developed a new combination of the T π HOG and the additive kernel SVM (AKSVM) for efficient nighttime pedestrian detection. The proposed T π HOG includes detailed information on gradient location; therefore, it has more distinctive power than the HOG. The AKSVM performs better than the linear SVM in terms of detection performance, while it is much faster than other kernel SVMs. The combined T π HOG-AKSVM showed effective nighttime PD performance with fast computational time. The proposed method was experimentally tested with the KAIST pedestrian dataset and showed better performance compared with other conventional methods

New Vehicle Detection Method with Aspect Ratio Estimation for Hypothesized Windows

All kinds of vehicles have different ratios of width to height, which are called the aspect ratios. Most previous works, however, use a fixed aspect ratio for vehicle detection (VD). The use of a fixed vehicle aspect ratio for VD degrades the performance. Thus, the estimation of a vehicle aspect ratio is an important part of robust VD. Taking this idea into account, a new on-road vehicle detection system is proposed in this paper. The proposed method estimates the aspect ratio of the hypothesized windows to improve the VD performance. Our proposed method uses an Aggregate Channel Feature (ACF) and a support vector machine (SVM) to verify the hypothesized windows with the estimated aspect ratio. The contribution of this paper is threefold. First, the estimation of vehicle aspect ratio is inserted between the HG (hypothesis generation) and the HV (hypothesis verification). Second, a simple HG method named a signed horizontal edge map is proposed to speed up VD. Third, a new measure is proposed to represent the overlapping ratio between the ground truth and the detection results. This new measure is used to show that the proposed method is better than previous works in terms of robust VD. Finally, the Pittsburgh dataset is used to verify the performance of the proposed method

Identifying and extracting bark key features of 42 tree species using convolutional neural networks and class activation mapping

© 2022, The Author(s).The significance of automatic plant identification has already been recognized by academia and industry. There were several attempts to utilize leaves and flowers for identification; however, bark also could be beneficial, especially for trees, due to its consistency throughout the seasons and its easy accessibility, even in high crown conditions. Previous studies regarding bark identification have mostly contributed quantitatively to increasing classification accuracy. However, ever since computer vision algorithms surpassed the identification ability of humans, an open question arises as to how machines successfully interpret and unravel the complicated patterns of barks. Here, we trained two convolutional neural networks (CNNs) with distinct architectures using a large-scale bark image dataset and applied class activation mapping (CAM) aggregation to investigate diagnostic keys for identifying each species. CNNs could identify the barks of 42 species with > 90% accuracy, and the overall accuracies showed a small difference between the two models. Diagnostic keys matched with salient shapes, which were also easily recognized by human eyes, and were typified as blisters, horizontal and vertical stripes, lenticels of various shapes, and vertical crevices and clefts. The two models exhibited disparate quality in the diagnostic features: the old and less complex model showed more general and well-matching patterns, while the better-performing model with much deeper layers indicated local patterns less relevant to barks. CNNs were also capable of predicting untrained species by 41.98% and 48.67% within the correct genus and family, respectively. Our methodologies and findings are potentially applicable to identify and visualize crucial traits of other plant organs.Y