7 research outputs found
Deep learning investigation for chess player attention prediction using eye-tracking and game data
This article reports on an investigation of the use of convolutional neural
networks to predict the visual attention of chess players. The visual attention
model described in this article has been created to generate saliency maps that
capture hierarchical and spatial features of chessboard, in order to predict
the probability fixation for individual pixels Using a skip-layer architecture
of an autoencoder, with a unified decoder, we are able to use multiscale
features to predict saliency of part of the board at different scales, showing
multiple relations between pieces. We have used scan path and fixation data
from players engaged in solving chess problems, to compute 6600 saliency maps
associated to the corresponding chess piece configurations. This corpus is
completed with synthetically generated data from actual games gathered from an
online chess platform. Experiments realized using both scan-paths from chess
players and the CAT2000 saliency dataset of natural images, highlights several
results. Deep features, pretrained on natural images, were found to be helpful
in training visual attention prediction for chess. The proposed neural network
architecture is able to generate meaningful saliency maps on unseen chess
configurations with good scores on standard metrics. This work provides a
baseline for future work on visual attention prediction in similar contexts
Evaluation of 3D Vision Systems for Detection of Small Objects in Agricultural Environments
3D information provides unique information about shape, localisation and relations between objects, not found
in standard 2D images. This information would be very beneficial in a large number of applications in agriculture such as fruit picking, yield monitoring, forecasting and phenotyping. In this paper, we conducted a
study on the application of modern 3D sensing technology together with the state-of-the-art machine learning
algorithms for segmentation and detection of strawberries growing in real farms. We evaluate the performance
of two state-of-the-art 3D sensing technologies and showcase the differences between 2D and 3D networks
trained on the images and point clouds of strawberry plants and fruit. Our study highlights limitations of the
current 3D vision systems for the detection of small objects in outdoor applications and sets out foundations for
future work on 3D perception for challenging outdoor applications such as agriculture
Real-time detection of broccoli crops in 3D point clouds for autonomous robotic harvesting
Real-time 3D perception of the environment is crucial for the adoption and deployment of reliable autonomous harvesting robots in agriculture. Using data collected with RGB-D cameras under farm field conditions, we present two methods for processing 3D data that reliably detect mature broccoli heads. The proposed systems are efficient and enable real-time detection on depth data of broccoli crops using the organised structure of the point clouds delivered by a depth sensor. The systems are tested with datasets of two broccoli varieties collected in planted fields from two different countries. Our evaluation shows the new methods outperform state-of-the- art approaches for broccoli detection based on both 2D vision- based segmentation techniques and depth clustering using the Euclidean proximity of neighbouring points. The results show the systems are capable of accurately detecting the 3D locations of broccoli heads relative to the vehicle at high frame rates
Segmentation and detection from organised 3D point clouds: a case study in broccoli head detection
Autonomous harvesting is becoming an important challenge and necessity in agriculture, because of the lack of labour and the growth of population needing to be fed. Perception is a key aspect of autonomous harvesting and is very challenging due to difficult lighting conditions, limited sensing technologies, occlusions, plant growth, etc. 3D vision approaches can bring several benefits addressing the aforementioned challenges such as localisation, size estimation, occlusion handling and shape analysis. In this paper, we propose a novel approach using 3D information for detecting broccoli heads based on Convolutional Neural Networks (CNNs), exploiting the organised nature of the point clouds originating from the RGBD sensors. The proposed algorithm, tested on real-world datasets, achieves better performances than the state-of-the-art, with better accuracy and generalisation in unseen scenarios, whilst significantly reducing inference time, making it better suited for real-time in-field applications
Gaussian map predictions for 3D surface feature localisation and counting
In this paper, we propose to employ a Gaussian map representation to estimate precise location and count of 3D surface features, addressing the limitations of state-of-the-art methods based on density estimation which struggle in presence of local disturbances. Gaussian maps indicate probable object location and can be generated directly from keypoint annotations avoiding laborious and costly per-pixel annotations. We apply this method to the 3D spheroidal class of objects which can be projected into 2D shape representation enabling efficient processing by a neural network GNet, an improved UNet architecture, which generates the likely locations of surface features and their precise count. We demonstrate a practical use of this technique for counting strawberry achenes which is used as a fruit quality measure in phenotyping applications. The results of training the proposed system on several hundreds of 3D scans of strawberries from a publicly available dataset demonstrate the accuracy and precision of the system which outperforms the state-of-the-art density-based methods for this application
3D shape sensing and deep learning-based segmentation of strawberries
Automation and robotisation of the agricultural sector are seen as a viable solution to socio-economic challenges
faced by this industry. This technology often relies on intelligent perception systems providing information about
crops, plants and the entire environment. The challenges faced by traditional 2D vision systems can be addressed
by modern 3D vision systems which enable straightforward localisation of objects, size and shape estimation, or
handling of occlusions. So far, the use of 3D sensing was mainly limited to indoor or structured environments. In
this paper, we evaluate modern sensing technologies including stereo and time-of-flight cameras for 3D
perception of shape in agriculture and study their usability for segmenting out soft fruit from background based
on their shape. To that end, we propose a novel 3D deep neural network which exploits the organised nature of
information originating from the camera-based 3D sensors. We demonstrate the superior performance and ef
ficiency of the proposed architecture compared to the state-of-the-art 3D networks. Through a simulated study,
we also show the potential of the 3D sensing paradigm for object segmentation in agriculture and provide in
sights and analysis of what shape quality is needed and expected for further analysis of crops. The results of this
work should encourage researchers and companies to develop more accurate and robust 3D sensing technologies
to assure their wider adoption in practical agricultural applications
Key Point-Based Orientation Estimation of Strawberries for Robotic Fruit Picking
Selective robotic harvesting can help address labour shortages affecting modern global agriculture. For an accurate and efficient picking process, a robotic harvester requires the precise location and orientation of the fruit to effectively plan the trajectory of the end effector. The current methods for estimating fruit orientation employ either complete 3D information registered from multiple views or rely on fully-supervised learning techniques, requiring difficult-to-obtain manual annotation of the reference orientation. In this paper, we introduce a novel key-point-based fruit orientation estimation method for the prediction of 3D orientation from 2D images directly. The proposed technique can work without full 3D orientation annotations but can also exploit such information for improved accuracy. We evaluate our work on two separate datasets of strawberry images obtained from real-world scenarios. Our method achieves state-of-the-art performance with an average error as low as 8◦, improving predictions by ∼30% compared to previous work presented in [18]. Furthermore, our method is suited for real-time robotic applications with fast inference times of ∼30ms