SimPS-Net: Simultaneous Pose & Segmentation Network of Surgical Tools

The ability to detect and localise surgical tools using RGB cameras during robotic assisted surgery can allow for the development of various implementations, such as vision- based active constraints and refinements in robot path planning, which can ultimately lead in improved patient safety during operation. For this purpose, the proposed network, SimPS-Net capable of both detection and 3D pose estimation of standard surgical tools using a single RGB camera, is introduced. In addition to the network, a novel dataset generated for training and testing is presented. The proposed network achieved a mean DICE coefficient of 85.0%, while also exhibiting a low average error of 5.5mm and 3.3◦ for 3D position and orientation respectively, thus outperforming the competing networks.</jats:p

Real-time active constraint generation and enforcement for surgical tools using 3D detection and localisation network

Introduction: Collaborative robots, designed to work alongside humans for manipulating end-effectors, greatly benefit from the implementation of active constraints. This process comprises the definition of a boundary, followed by the enforcement of some control algorithm when the robot tooltip interacts with the generated boundary. Contact with the constraint boundary is communicated to the human operator through various potential forms of feedback. In fields like surgical robotics, where patient safety is paramount, implementing active constraints can prevent the robot from interacting with portions of the patient anatomy that shouldn’t be operated on. Despite improvements in orthopaedic surgical robots, however, there exists a gap between bulky systems with haptic feedback capabilities and miniaturised systems that only allow for boundary control, where interaction with the active constraint boundary interrupts robot functions. Generally, active constraint generation relies on optical tracking systems and preoperative imaging techniques.Methods: This paper presents a refined version of the Signature Robot, a three degrees-of-freedom, hands-on collaborative system for orthopaedic surgery. Additionally, it presents a method for generating and enforcing active constraints “on-the-fly” using our previously introduced monocular, RGB, camera-based network, SimPS-Net. The network was deployed in real-time for the purpose of boundary definition. This boundary was subsequently used for constraint enforcement testing. The robot was utilised to test two different active constraints: a safe region and a restricted region.Results: The network success rate, defined as the ratio of correct over total object localisation results, was calculated to be 54.7% ± 5.2%. In the safe region case, haptic feedback resisted tooltip manipulation beyond the active constraint boundary, with a mean distance from the boundary of 2.70 mm ± 0.37 mm and a mean exit duration of 0.76 s ± 0.11 s. For the restricted-zone constraint, the operator was successfully prevented from penetrating the boundary in 100% of attempts.Discussion: This paper showcases the viability of the proposed robotic platform and presents promising results of a versatile constraint generation and enforcement pipeline.</jats:p

SimPS-Net: Simultaneous Pose &amp; Segmentation Network of Surgical Tools

Localisation of surgical tools during operation is of paramount importance in the context of robotic assisted surgery. 3D pose estimation can be utilised to explore the interaction of tools with registered tissue and improve the motion planning of robotic platforms, thus avoiding potential collisions with external agents. With the problems of traditional tracking systems being cost and the need to redesign surgical tools to accommodate markers, there has been a shift towards image-based, markerless tracking techniques. This study introduces a network capable of detecting and localising tools in 3D using a monocular setup. For training and validation, a novel dataset, 3dStool, was produced, and the network was trained to obtain a mean Dice coefficient of 85.0% for detection, along with a mean position and orientation error of 5.5mm and 3.3. respectively. The presented method is significantly more versatile than various state of the art solutions, as it requires no prior knowledge regarding the 3D structure of the tracked tools. The results were compared to standard pose estimation networks using the same dataset and demonstrated lower errors along most metrics. In addition, the generalisation capabilities of the proposed network were explored by performing inference on a previously unseen pair of scissors

Using Post-Emergence Herbicides in Combination with the Sowing Date to Suppress Sinapis arvensis and Silybum marianum in Durum Wheat

Wild mustard (Sinapis arvensis L.) and milk thistle (Silybum marianum (L.) Gaertn.) are two competitive broad-leaved weeds commonly found in cereals in Europe, while several weed species have developed resistance to the main herbicides that are applied on these crops. Thus, the implementation of integrated weed management (IWM) programs is of great importance. Field experiments were conducted based on a split-plot design with two factors (sowing date and herbicides). Our results showed that the density of wild mustard and milk thistle was higher in the early sowing compared to the late sowing, while the total weed density was up to 75% higher in early sowing. Moreover, the herbicides florasulam + 2.4-D and bromoxynil + 2.4-D exhibited high efficacy (&gt;98%) against milk thistle and wild mustard, while tribenuron-methyl and florasulam + clopyralid provided greater efficacy in the late sowing compared to the early sowing. Among the four herbicides, the lowest dry biomass and grain yield of wheat were observed in tribenuron-methyl and florasulam + clopyralid, while in the weed-infested treatment, the highest values of both parameters were recorded in late sowing. Finally, the results showed that the sowing date is a cultural weed control method that should be implemented in IWM programs, since it can affect both weed density and herbicide efficacy

Effects of Post-Emergence Herbicides and Period of Johnsongrass (<i>Sorghum halepense</i> (L.) Pers.) Control on Growth and Yield of Sunflower Crops

Sunflower is an important industrial crop since it is grown all over the world for oil production, while Johnsongrass (Sorghum halepense (L.) Pers.) is characterized by great competitiveness and can severely impair plant growth and crop productivity. Thus, a two-year field experiment was conducted to evaluate the impact of Johnsongrass control practices on plant growth, seed yield, and oil content of sunflower crop. The results indicated that Johnsongrass competition negatively affected sunflower growth and productivity as the lowest values of height, dry biomass, seed, and oil yields were recorded at the weed-infested treatment, followed by the weed infested for 30 days after sowing. All the other treatments had a positive effect on vegetative and yield parameters. Moreover, fluazifop-p-butyl, quizalofop-p-ethyl, and the combination of fluazifop-p-butyl and imazamox effectively controlled Johnsongrass. Specifically, in 2020, the lowest dry weight of Johnsongrass was observed in the plots where fluazifop-p-butyl + imazamox were applied. Thus, the results of this study clearly showed that the use of the above-mentioned herbicides can improve the seed and oil yield of a sunflower crop by managing Johnsongrass, while the competition of this rapidly growing weed for a short period of 30 days can significantly reduce crop yield