Fast and Reliable Autonomous Surgical Debridement with Cable-Driven Robots Using a Two-Phase Calibration Procedure

Automating precision subtasks such as debridement (removing dead or diseased tissue fragments) with Robotic Surgical Assistants (RSAs) such as the da Vinci Research Kit (dVRK) is challenging due to inherent non-linearities in cable-driven systems. We propose and evaluate a novel two-phase coarse-to-fine calibration method. In Phase I (coarse), we place a red calibration marker on the end effector and let it randomly move through a set of open-loop trajectories to obtain a large sample set of camera pixels and internal robot end-effector configurations. This coarse data is then used to train a Deep Neural Network (DNN) to learn the coarse transformation bias. In Phase II (fine), the bias from Phase I is applied to move the end-effector toward a small set of specific target points on a printed sheet. For each target, a human operator manually adjusts the end-effector position by direct contact (not through teleoperation) and the residual compensation bias is recorded. This fine data is then used to train a Random Forest (RF) to learn the fine transformation bias. Subsequent experiments suggest that without calibration, position errors average 4.55mm. Phase I can reduce average error to 2.14mm and the combination of Phase I and Phase II can reduces average error to 1.08mm. We apply these results to debridement of raisins and pumpkin seeds as fragment phantoms. Using an endoscopic stereo camera with standard edge detection, experiments with 120 trials achieved average success rates of 94.5%, exceeding prior results with much larger fragments (89.4%) and achieving a speedup of 2.1x, decreasing time per fragment from 15.8 seconds to 7.3 seconds. Source code, data, and videos are available at https://sites.google.com/view/calib-icra/.Comment: Code, data, and videos are available at https://sites.google.com/view/calib-icra/. Final version for ICRA 201

Surface Defect Classification for Hot-Rolled Steel Strips by Selectively Dominant Local Binary Patterns

Developments in defect descriptors and computer vision-based algorithms for automatic optical inspection (AOI) allows for further development in image-based measurements. Defect classification is a vital part of an optical-imaging-based surface quality measuring instrument. The high-speed production rhythm of hot continuous rolling requires an ultra-rapid response to every component as well as algorithms in AOI instrument. In this paper, a simple, fast, yet robust texture descriptor, namely selectively dominant local binary patterns (SDLBPs), is proposed for defect classification. First, an intelligent searching algorithm with a quantitative thresholding mechanism is built to excavate the dominant non-uniform patterns (DNUPs). Second, two convertible schemes of pattern code mapping are developed for binary encoding of all uniform patterns and DNUPs. Third, feature extraction is carried out under SDLBP framework. Finally, an adaptive region weighting method is built for further strengthening the original nearest neighbor classifier in the feature matching stage. The extensive experiments carried out on an open texture database (Outex) and an actual surface defect database (Dragon) indicates that our proposed SDLBP yields promising performance on both classification accuracy and time efficiencyPeer reviewe

Detection of Grape Clusters in Images using Convolutional Neural Network

Convolutional Neural Networks and Deep Learning have revolutionized every field since their inception. Agriculture has also been reaping the fruits of developments in mentioned fields. Technology is being revolutionized to increase yield, save water wastage, take care of diseased weeds, and also increase the profit of farmers. Grapes are among the highest profit-yielding and important fruit related to the juice industry. Pakistan being an agricultural country, can widely benefit by cultivating and improving grapes per hectare yield. The biggest challenge in harvesting grapes to date is to detect their cluster successfully; many approaches tend to answer this problem by harvest and sort technique where the foreign objects are separated later from grapes after harvesting them using an automatic harvester. Currently available systems are trained on data that is from developed or grape-producing countries, thus showing data biases when used at any new location thus it gives rise to a need of creating a dataset from scratch to verify the results of research. Grape is available in different sizes, colors, seed sizes, and shapes which makes its detection, through simple Computer vision, even more challenging. This research addresses this issue by bringing the solution to this problem by using CNN and Neural Networks using the newly created dataset from local farms as the other research and the methods used don’t address issues faced locally by the farmers. YOLO has been selected to be trained on the locally collected dataset of grapes

RecPOID: POI Recommendation with Friendship Aware and Deep CNN

In location-based social networks (LBSNs), exploit several key features of points-of-interest (POIs) and users on precise POI recommendation be significant. In this work, a novel POI recommenda-tion pipeline based on the convolutional neural network named RecPOID is proposed, which can recommend an accurate sequence of top-k POIs and considers only the effect of the most similar pattern friendship rather than all user’s friendship. We use the fuzzy c-mean clustering method to find the similarity. Temporal and spatial features of similar friends are fed to our Deep CNN model. The 10-layer convolutional neural network can predict longitude and latitude and the Id of the next proper locations; after that, based on the shortest time distance from a similar pattern’s friendship, select the smallest distance locations. The proposed structure uses six features, includ-ing user’s ID, month, day, hour, minute, and second of visiting time by each user as inputs. RecPOID based on two accessible LBSNs datasets is evaluated. Experimental outcomes illustrate considering most similar friendship could improve the accuracy of recommendations and the proposed RecPOID for POI recommendation outperforms state-of-the-art approaches

DeePOF: A hybrid approach of deep convolutional neural network and friendship to Point‐of‐Interest (POI) recommendation system in location‐based social networks

Today, millions of active users spend a percentage of their time on location-based social networks like Yelp and Gowalla and share their rich information. They can easily learn about their friends\u27 behaviors and where they are visiting and be influenced by their style. As a result, the existence of personalized recommendations and the investigation of meaningful features of users and Point of Interests (POIs), given the challenges of rich contents and data sparsity, is a substantial task to accurately recommend the POIs and interests of users in location-based social networks (LBSNs). This work proposes a novel pipeline of POI recommendations named DeePOF based on deep learning and the convolutional neural network. This approach only takes into consideration the influence of the most similar pattern of friendship instead of the friendship of all users. The mean-shift clustering technique is used to detect similarity. The most similar friends\u27 spatial and temporal features are fed into our deep CNN technique. The output of several proposed layers can predict latitude and longitude and the ID of subsequent appropriate places, and then using the friendship interval of a similar pattern, the lowest distance venues are chosen. This combination method is estimated on two popular datasets of LBSNs. Experimental results demonstrate that analyzing similar friendships could make recommendations more accurate and the suggested model for recommending a sequence of top-k POIs outperforms state-of-the-art approaches

Activity Recognition in Smart Homes via Feature-Rich Visual Extraction of Locomotion Traces

The proliferation of sensors in smart homes makes it possible to monitor human activities, routines, and complex behaviors in an unprecedented way. Hence, human activity recognition has gained increasing attention over the last few years as a tool to improve healthcare and well-being in several applications. However, most existing activity recognition systems rely on cameras or wearable sensors, which may be obtrusive and may invade the user's privacy, especially at home. Moreover, extracting expressive features from a stream of data provided by heterogeneous smart-home sensors is still an open challenge. In this paper, we investigate a novel method to detect activities of daily living by exploiting unobtrusive smart-home sensors (i.e., passive infrared position sensors and sensors attached to everyday objects) and vision-based deep learning algorithms, without the use of cameras or wearable sensors. Our method relies on depicting the locomotion traces of the user and visual clues about their interaction with objects on a floor plan map of the home, and utilizes pre-trained deep convolutional neural networks to extract features for recognizing ongoing activity. One additional advantage of our method is its seamless extendibility with additional features based on the available sensor data. Extensive experiments with a real-world dataset and a comparison with state-of-the-art approaches demonstrate the effectiveness of our method