305 research outputs found

    ABC: Adaptive, Biomimetic, Configurable Robots for Smart Farms - From Cereal Phenotyping to Soft Fruit Harvesting

    Get PDF
    Currently, numerous factors, such as demographics, migration patterns, and economics, are leading to the critical labour shortage in low-skilled and physically demanding parts of agriculture. Thus, robotics can be developed for the agricultural sector to address these shortages. This study aims to develop an adaptive, biomimetic, and configurable modular robotics architecture that can be applied to multiple tasks (e.g., phenotyping, cutting, and picking), various crop varieties (e.g., wheat, strawberry, and tomato) and growing conditions. These robotic solutions cover the entire perception–action–decision-making loop targeting the phenotyping of cereals and harvesting fruits in a natural environment. The primary contributions of this thesis are as follows. a) A high-throughput method for imaging field-grown wheat in three dimensions, along with an accompanying unsupervised measuring method for obtaining individual wheat spike data are presented. The unsupervised method analyses the 3D point cloud of each trial plot, containing hundreds of wheat spikes, and calculates the average size of the wheat spike and total spike volume per plot. Experimental results reveal that the proposed algorithm can effectively identify spikes from wheat crops and individual spikes. b) Unlike cereal, soft fruit is typically harvested by manual selection and picking. To enable robotic harvesting, the initial perception system uses conditional generative adversarial networks to identify ripe fruits using synthetic data. To determine whether the strawberry is surrounded by obstacles, a cluster complexity-based perception system is further developed to classify the harvesting complexity of ripe strawberries. c) Once the harvest-ready fruit is localised using point cloud data generated by a stereo camera, the platform’s action system can coordinate the arm to reach/cut the stem using the passive motion paradigm framework, as inspired by studies on neural control of movement in the brain. Results from field trials for strawberry detection, reaching/cutting the stem of the fruit with a mean error of less than 3 mm, and extension to analysing complex canopy structures/bimanual coordination (searching/picking) are presented. Although this thesis focuses on strawberry harvesting, ongoing research is heading toward adapting the architecture to other crops. The agricultural food industry remains a labour-intensive sector with a low margin, and cost- and time-efficiency business model. The concepts presented herein can serve as a reference for future agricultural robots that are adaptive, biomimetic, and configurable

    Developmental Bootstrapping of AIs

    Full text link
    Although some current AIs surpass human abilities in closed artificial worlds such as board games, their abilities in the real world are limited. They make strange mistakes and do not notice them. They cannot be instructed easily, fail to use common sense, and lack curiosity. They do not make good collaborators. Mainstream approaches for creating AIs are the traditional manually-constructed symbolic AI approach and generative and deep learning AI approaches including large language models (LLMs). These systems are not well suited for creating robust and trustworthy AIs. Although it is outside of the mainstream, the developmental bootstrapping approach has more potential. In developmental bootstrapping, AIs develop competences like human children do. They start with innate competences. They interact with the environment and learn from their interactions. They incrementally extend their innate competences with self-developed competences. They interact and learn from people and establish perceptual, cognitive, and common grounding. They acquire the competences they need through bootstrapping. However, developmental robotics has not yet produced AIs with robust adult-level competences. Projects have typically stopped at the Toddler Barrier corresponding to human infant development at about two years of age, before their speech is fluent. They also do not bridge the Reading Barrier, to skillfully and skeptically draw on the socially developed information resources that power current LLMs. The next competences in human cognitive development involve intrinsic motivation, imitation learning, imagination, coordination, and communication. This position paper lays out the logic, prospects, gaps, and challenges for extending the practice of developmental bootstrapping to acquire further competences and create robust, resilient, and human-compatible AIs.Comment: 102 pages, 29 figure

    Learning-Based Control Strategies for Soft Robots: Theory, Achievements, and Future Challenges

    Get PDF
    In the last few decades, soft robotics technologies have challenged conventional approaches by introducing new, compliant bodies to the world of rigid robots. These technologies and systems may enable a wide range of applications, including human-robot interaction and dealing with complex environments. Soft bodies can adapt their shape to contact surfaces, distribute stress over a larger area, and increase the contact surface area, thus reducing impact forces

    International Academic Symposium of Social Science 2022

    Get PDF
    This conference proceedings gathers work and research presented at the International Academic Symposium of Social Science 2022 (IASSC2022) held on July 3, 2022, in Kota Bharu, Kelantan, Malaysia. The conference was jointly organized by the Faculty of Information Management of Universiti Teknologi MARA Kelantan Branch, Malaysia; University of Malaya, Malaysia; Universitas Pembangunan Nasional Veteran Jakarta, Indonesia; Universitas Ngudi Waluyo, Indonesia; Camarines Sur Polytechnic Colleges, Philippines; and UCSI University, Malaysia. Featuring experienced keynote speakers from Malaysia, Australia, and England, this proceeding provides an opportunity for researchers, postgraduate students, and industry practitioners to gain knowledge and understanding of advanced topics concerning digital transformations in the perspective of the social sciences and information systems, focusing on issues, challenges, impacts, and theoretical foundations. This conference proceedings will assist in shaping the future of the academy and industry by compiling state-of-the-art works and future trends in the digital transformation of the social sciences and the field of information systems. It is also considered an interactive platform that enables academicians, practitioners and students from various institutions and industries to collaborate

    From visuomotor control to latent space planning for robot manipulation

    Get PDF
    Deep visuomotor control is emerging as an active research area for robot manipulation. Recent advances in learning sensory and motor systems in an end-to-end manner have achieved remarkable performance across a range of complex tasks. Nevertheless, a few limitations restrict visuomotor control from being more widely adopted as the de facto choice when facing a manipulation task on a real robotic platform. First, imitation learning-based visuomotor control approaches tend to suffer from the inability to recover from an out-of-distribution state caused by compounding errors. Second, the lack of versatility in task definition limits skill generalisability. Finally, the training data acquisition process and domain transfer are often impractical. In this thesis, individual solutions are proposed to address each of these issues. In the first part, we find policy uncertainty to be an effective indicator of potential failure cases, in which the robot is stuck in out-of-distribution states. On this basis, we introduce a novel uncertainty-based approach to detect potential failure cases and a recovery strategy based on action-conditioned uncertainty predictions. Then, we propose to employ visual dynamics approximation to our model architecture to capture the motion of the robot arm instead of the static scene background, making it possible to learn versatile skill primitives. In the second part, taking inspiration from the recent progress in latent space planning, we propose a gradient-based optimisation method operating within the latent space of a deep generative model for motion planning. Our approach bypasses the traditional computational challenges encountered by established planning algorithms, and has the capability to specify novel constraints easily and handle multiple constraints simultaneously. Moreover, the training data comes from simple random motor-babbling of kinematically feasible robot states. Our real-world experiments further illustrate that our latent space planning approach can handle both open and closed-loop planning in challenging environments such as heavily cluttered or dynamic scenes. This leads to the first, to our knowledge, closed-loop motion planning algorithm that can incorporate novel custom constraints, and lays the foundation for more complex manipulation tasks

    Robot tool use: A survey

    Get PDF
    Using human tools can significantly benefit robots in many application domains. Such ability would allow robots to solve problems that they were unable to without tools. However, robot tool use is a challenging task. Tool use was initially considered to be the ability that distinguishes human beings from other animals. We identify three skills required for robot tool use: perception, manipulation, and high-level cognition skills. While both general manipulation tasks and tool use tasks require the same level of perception accuracy, there are unique manipulation and cognition challenges in robot tool use. In this survey, we first define robot tool use. The definition highlighted the skills required for robot tool use. The skills coincide with an affordance model which defined a three-way relation between actions, objects, and effects. We also compile a taxonomy of robot tool use with insights from animal tool use literature. Our definition and taxonomy lay a theoretical foundation for future robot tool use studies and also serve as practical guidelines for robot tool use applications. We first categorize tool use based on the context of the task. The contexts are highly similar for the same task (e.g., cutting) in non-causal tool use, while the contexts for causal tool use are diverse. We further categorize causal tool use based on the task complexity suggested in animal tool use studies into single-manipulation tool use and multiple-manipulation tool use. Single-manipulation tool use are sub-categorized based on tool features and prior experiences of tool use. This type of tool may be considered as building blocks of causal tool use. Multiple-manipulation tool use combines these building blocks in different ways. The different combinations categorize multiple-manipulation tool use. Moreover, we identify different skills required in each sub-type in the taxonomy. We then review previous studies on robot tool use based on the taxonomy and describe how the relations are learned in these studies. We conclude with a discussion of the current applications of robot tool use and open questions to address future robot tool use

    Artificial self-awareness for robots

    Get PDF
    Robots are evolving and entering into various sectors and aspects of life. While humans are aware of their bodies and capabilities, which help them work on a task in different environments, robots are not. This thesis is about defining and developing a robotic artificial self-awareness framework. The aim is to allow robots to adapt to their environment and better manage their task. The robot’s artificial self-aware knowledge is captured based on levels where each level helps a robot acquire higher self-awareness competence. These levels are inspired by Rochat [1] self-awareness development levels in humans, where each level is associated with a complexity of self-knowledge. Self-awareness in humans leads to distinguishing themselves from the environment, allowing humans to understand themselves and control their capabilities. This work focuses on the first and second levels of self awareness through differentiation and situation (minimal self). The artificial self-awareness level-1 proposes the first step towards a basic, minimal self-awareness in a robot. The artificial self-awareness level-2 proposes an increasing capacity of self-awareness knowledge in the robot. That is, this thesis posits an experimental methodology to evaluate whether the robot can differentiate and situate itself from the environment and to test whether artificial self-awareness level-1 and level-2 increase a robot’s self-certainty in an unseen environment. The research utilises deep neural network techniques to allow a dual-arm robot to identify itself within different environments. The robot vision and proprioception are captured using a camera and robot sensors to build a model that allows a robot to differentiate itself from the environment. The level-1 results indicate that a robot can distinguish itself with an accuracy of 80.3% on average in different environmental settings and under confounding input signals. Also, the level-2 results show that a robot can situate itself in different environments with an accuracy of 86.01% yielding a higher artificial self-certainty of 5.71%. This thesis work helps a robot be aware of itself in different environments

    Static Shape Control of Soft Continuum Robots using Deep Visual Inverse Kinematic Models

    Get PDF

    Neural and Cognitive Mechanisms of Real-World Interaction during Adult Learning

    Get PDF
    The goal of this thesis is to understand what makes a social interaction successful, and whether it supports learning of conceptual knowledge. Crucially, it distinguishes learning via the social from learning about the social, and asks the question of how social interaction supports declarative processing of non-social material. In doing so, it priorities ecological validity: all experiments involve relatively unconstrained teacher-learner interaction, and learning material resembled documentary-like content. The first half of the thesis shows a series of studies on how adults learn in online contexts (Study 1 and 2): Study 1 presents two online experiments, where social contingency (i.e. being part of a live interaction vs observing a pre-recorded one) and social cues (i.e. teacher’s webcam on vs off vs showing a slide only) were manipulated. Results showed that learning in live interaction was associated with the best performance, and live social interaction with a full view of the teacher provided the optimal setting for learning, while seeing a slide had greater benefit during recorded sessions specifically. Study 2 replicates the live-learning advantage across three experiments and a large sample of adults with Autistic Spectrum Condition (ASC). The second half of this thesis (Study 3 and 4) investigates face-to-face interaction, using functional Near-Infrared Spectroscopy (fNIRS) hyperscanning and wavelet transform coherence (WTC) analysis, to measure brain synchrony in naturalistic interactions. Study 3 tests the hypothesis that being in the same room and engaging in conversation affects people’s brain response to later novel stimuli. Study 4 asks whether teacher-student brain synchrony can be a marker of learning success and, if so, how it is modulated by social behaviours. Findings reveal a complex dynamic between neural responses and behavioural metrics, in particular mutual gaze and joint attention. Results are discussed in the frame of the mutual-prediction hypothesis, and advocate for a multi-modal investigation of social learning to fully understand its underlying cognitive mechanisms. Overall, this work advances the current understanding of naturalistic social interaction and has theoretical implications for cognitive models of information exchange and mutual prediction, as well as practical significance for educational policies. The novel multi-modal and highly ecological approach used in this thesis makes this work an important example for real-world second person social neuroscience
    • …
    corecore