Design and Implementation of a GPS Guidance System for Agricultural Tractors Using Augmented Reality Technology

Current commercial tractor guidance systems present to the driver information to perform agricultural tasks in the best way. This information generally includes a treated zones map referenced to the tractor’s position. Unlike actual guidance systems where the tractor driver must mentally associate treated zone maps and the plot layout, this paper presents a guidance system that using Augmented Reality (AR) technology, allows the tractor driver to see the real plot though eye monitor glasses with the treated zones in a different color. The paper includes a description of the system hardware and software, a real test done with image captures seen by the tractor driver, and a discussion predicting that the historical evolution of guidance systems could involve the use of AR technology in the agricultural guidance and monitoring systems

Steering a Tractor by Means of an EMG-Based Human-Machine Interface

An electromiographic (EMG)-based human-machine interface (HMI) is a communication pathway between a human and a machine that operates by means of the acquisition and processing of EMG signals. This article explores the use of EMG-based HMIs in the steering of farm tractors. An EPOC, a low-cost human-computer interface (HCI) from the Emotiv Company, was employed. This device, by means of 14 saline sensors, measures and processes EMG and electroencephalographic (EEG) signals from the scalp of the driver. In our tests, the HMI took into account only the detection of four trained muscular events on the driver’s scalp: eyes looking to the right and jaw opened, eyes looking to the right and jaw closed, eyes looking to the left and jaw opened, and eyes looking to the left and jaw closed. The EMG-based HMI guidance was compared with manual guidance and with autonomous GPS guidance. A driver tested these three guidance systems along three different trajectories: a straight line, a step, and a circumference. The accuracy of the EMG-based HMI guidance was lower than the accuracy obtained by manual guidance, which was lower in turn than the accuracy obtained by the autonomous GPS guidance; the computed standard deviations of error to the desired trajectory in the straight line were 16 cm, 9 cm, and 4 cm, respectively. Since the standard deviation between the manual guidance and the EMG-based HMI guidance differed only 7 cm, and this difference is not relevant in agricultural steering, it can be concluded that it is possible to steer a tractor by an EMG-based HMI with almost the same accuracy as with manual steering

Trolls: a novel low-cost controlling system platform for walk-behind tractor

A novel low-cost controlling system platform for walk-behind hand tractors (Quick G3000 and G1000) was designed and developed to solve the fatigue problem faced by farmers when ploughing the rice field. This platform is dedicated to designing and manufacturing mechanical, electrical, and software components. The tractor was modified and added with an embedded control system that functioned as the slave, while the direction of the tractor movement was controlled remotely by humans through Bluetooth communication with the smartphone application as the master. Several servos and direct currents (DCs) were used as the actuator to move some levers and clutches instead of the tractor to make it remotely controllable. This system has been directly tested in the paddy farming land through two tractors: Quick G3000 and G1000. The testing results showed that this system could be used within more or less six hours; there is a cost-efficiency of 21.74% and 84.62% battery usage efficiency. More efficient mechanics caused this cost efficiency, and the reduction in electronic devices affects battery efficiency. A low-cost platform for controlling walk-behind tractors has been successfully developed; this platform assists farmers in ploughing their fields

Reitinsuunnittelu määrätyssä järjestyksessä tehtäville peltotöille usean työkoneen yhteistyönä

Coverage path planning is the task of finding a collision free path that passes over every point of an area or volume of interest. In agriculture, the coverage task is encountered especially in the process of crop cultivation. Several tasks are performed on the field, one after the other, during the cultivation cycle. Cooperation means that multiple agents, in this case vehicles, are working together towards a common goal. Several studies consider the problem where a single task is divided and assigned among the agents. In this thesis, however, the vehicles have different tasks that are sequentially dependent, that is, the first task must be completed before the other. The tasks are performed simultaneously on the same area. The literature review suggests that there is a lack of previous research on this topic. The objective of this thesis was to develop an algorithm to solve the cooperative coverage path planning problem for sequentially dependent tasks. A tool chain that involves Matlab, Simulink and Visual Studio was adapted for the development and testing of the solution. A development and testing architecture was designed including a compatible interface to a simulation and a real-life test environment. Two different algorithms were implemented based on the idea of computing short simultaneous paths at a time and scheduling them in real-time. The results were successfully demonstrated in a real-life test environment with two tractors equipped with a disc cultivator and a seeder. The objective was to sow the test area. The test drives show that with the algorithms that were developed in this thesis it is possible to perform two sequentially dependent agricultural coverage tasks simultaneously on the same area.Kattavassa reitinsuunnittelussa yritetään löytää polku, jonka aikana määritelty ala tai tilavuus tulee käytyä läpi niin että alueen jokainen piste on käsitelty. Maataloudessa tämä tehtävä on merkityksellinen erityisesti peltoviljelyssä. Useita peltotöitä suoritetaan yksi toisensa jälkeen samalla alueella viljelyvuoden aikana. Useissa tutkimuksissa käsitellään yhteistyönä tehtävää reitinsuunnittelua, jossa yksi tehtävä on jaettu osiin ja osat jaetaan useiden tekijöiden kuten robottien kesken. Tässä diplomityössä peltotyökoneilla on kuitenkin omat erilliset tehtävänsä, joilla on määrätty järjestys, eli niiden suorittaminen riippuu työjärjestyksestä. Työkoneet työskentelevät samanaikaisesti samalla alueella. Diplomityössä tehty kirjallisuuskatsaus viittaa siihen, että vastaavaa aihetta ei ole aiemmin tutkittu. Tämän diplomityön tavoitteena on kehittää algoritmi, jolla voidaan toteuttaa reitinsuunnittelu määrätyssä järjestyksessä tehtäville peltotöille usean peltotyökoneen yhteistyönä. Algoritmikehitystä ja testausta varten suunniteltiin yhtenäinen rajapinta, jolla algoritmia voitaisiin testata sekä simulaatiossa että todellisessa testitilanteessa. Algoritmikehityksessä käytettiin työkaluina Matlab, Simulink ja Visual Studio -ohjelmia. Työssä toteutettiin kaksi algoritmia, jotka perustuvat samaan ideaan: suunnitellaan kerrallaan kaksi lyhyttä samanaikaista polkua, jotka ajoitetaan reaaliajassa. Algoritmeja testattiin todellisessa testiympäristössä kahden työkoneen yhteistyönä, kun tavoitteena on kylvää koko testialue. Ensimmäinen työvaihe suoritettiin lautasmuokkaimella ja toinen kylvökoneella. Testiajot osoittavat, että diplomityössä kehitetyillä algoritmeilla voidaan ohjata kahden toisistaan riippuvaisen peltotyön toteutus samanaikaisesti samalla peltoalueella

An Augmented Interaction Strategy For Designing Human-Machine Interfaces For Hydraulic Excavators

Lack of adequate information feedback and work visibility, and fatigue due to repetition have been identified as the major usability gaps in the human-machine interface (HMI) design of modern hydraulic excavators that subject operators to undue mental and physical workload, resulting in poor performance. To address these gaps, this work proposed an innovative interaction strategy, termed “augmented interaction”, for enhancing the usability of the hydraulic excavator. Augmented interaction involves the embodiment of heads-up display and coordinated control schemes into an efficient, effective and safe HMI. Augmented interaction was demonstrated using a framework consisting of three phases: Design, Implementation/Visualization, and Evaluation (D.IV.E). Guided by this framework, two alternative HMI design concepts (Design A: featuring heads-up display and coordinated control; and Design B: featuring heads-up display and joystick controls) in addition to the existing HMI design (Design C: featuring monitor display and joystick controls) were prototyped. A mixed reality seating buck simulator, named the Hydraulic Excavator Augmented Reality Simulator (H.E.A.R.S), was used to implement the designs and simulate a work environment along with a rock excavation task scenario. A usability evaluation was conducted with twenty participants to characterize the impact of the new HMI types using quantitative (task completion time, TCT; and operating error, OER) and qualitative (subjective workload and user preference) metrics. The results indicated that participants had a shorter TCT with Design A. For OER, there was a lower error probability due to collisions (PER1) with Design A, and lower error probability due to misses (PER2)with Design B. The subjective measures showed a lower overall workload and a high preference for Design B. It was concluded that augmented interaction provides a viable solution for enhancing the usability of the HMI of a hydraulic excavator

Event and Time-Triggered Control Module Layers for Individual Robot Control Architectures of Unmanned Agricultural Ground Vehicles

Automation in the agriculture sector has increased to an extent where the accompanying methods for unmanned field management are becoming more economically viable. This manifests in the industry’s recent presentation of conceptual cab-less machines that perform all field operations under the high-level task control of a single remote operator. A dramatic change in the overall workflow for field tasks that historically assumed the presence of a human in the immediate vicinity of the work is predicted. This shift in the entire approach to farm machinery work provides producers increased control and productivity over high-level tasks and less distraction from operating individual machine actuators and implements. The final implication is decreased mechanical complexity of the cab-less field machines from their manned counter types. An Unmanned Agricultural Ground Vehicle (UAGV) electric platform received a portable control module layer (CML) which was modular and able to accept higher-level mission commands while returning system states to high-level tasks. The simplicity of this system was shown by its entire implementation running on microcontrollers networked on a Time-Triggered Controller Area Network (TTCAN) bus. A basic form of user input and output was added to the system to demonstrate a simple instance of sub-system integration. In this work, all major levels of design and implementation are examined in detail, revealing the ‘why’ and ‘how’ of each subsystem. System design philosophy is highlighted from the beginning. A state-space feedback steering controller was implemented on the machine utilizing a basic steering model found in literature. Finally, system performance is evaluated from the perspectives of a number of disciplines including: embedded systems software design, control systems, and robot control architecture. Recommendations for formalized UAGV system modeling, estimation, and control are discussed for the continuation of research in simplified low-cost machines for in-field task automation. Additional recommendations for future time-triggered CML experiments in bus robustness and redundancy are discussed. The work presented is foundational in the shift from event-triggered communications towards time-triggered CMLs for unmanned agricultural machinery and is a front-to-back demonstration of time-triggered design. Advisor: Santosh K. Pitl

Mobile Service Support based on Smart Glasses

Emerging technologies, such as smart glasses, offer new possibilities to support service processes. Specifically, in situations where a person providing a service, such as a technician, needs both hands to complete a complex set of tasks, hands-free speech-controlled information systems can offer support with additional information. We investigated this research field in a three-year consortium with partners from the agricultural technology sector. During the course of our research, we 1) analyzed the domain in a multi-method approach to develop (meta-)requirements, 2) proposed design principles, 3) instantiated them in a prototype, and 4) evaluated the prototype. We followed a design science research approach in which we combined the build phase with four evaluation cycles that comprised focus groups, a prototype demonstration, and, based on that demonstration, a survey with 105 domain experts. We address real-world problems in providing information at the point of service and contribute to the methodological knowledge base of IS design and service systems engineering by developing and implementing design requirements and principles for smart glasses-based service support systems

Automation and Robotics in Forest Harvesting Operations: Identifying Near-Term Opportunities

Technology development, in terms of both capability and cost-effective integration, is moving at a fast pace. While advanced robotic systems are already commonplace in controlled workspaces such as factories, the use of remote controlled or autonomous machines in more complex environments, such as for forest operations, is in its infancy. There is little doubt autonomous machinery will play an important role in forest operations in the future. Many machine functions already have the support of automation, and the implementation of remote control of the machine where an operator can operate a piece of equipment, typically in clear line-of sight, at least is commonly available. Teleoperation is where the operator works from a virtual environment with live video and audio feedback from the machine. Since teleoperation provides a similar operator experience to working in the machine, it is relatively easy for an operator to use teleoperation. Autonomous systems are defined by being able to perform certain functions without direct control of a human operator. This paper presents opportunities for remote control, teleoperated machines in forest operations and presents examples of existing developments and ideas from both forestry and other industries. It identified the extraction phase of harvesting as the most logical placement of autonomous machines in the near-term. The authors recognise that, as with all emerging technologies and sectors, there is ample scope for differences in opinions as to what will be commercially successful in the future

CREATING AN INNOVATION OPPORTUNITY SPACE FOR BROADACRE SMART FARMING: A CASE STUDY OF AUTONOMOUS FARM EQUIPMENT

Advances in digital technologies are transforming the agriculture and agri-food system. The technological changes are represented in many forms, ranging from software-based prescriptions for optimal rate application of farm inputs, advanced imagery of fields and plants collected by sensors, satellites and drones, to new forms of human-to-machine interactions and machine learning This thesis is a case study of one type of a smart farming innovation, a field robot., originating from a small-to-medium sized enterprise (SME) that designs and manufacturers machinery used in broadacre, conservation tillage farming. The innovation, known as DOT™, is an entrepreneur’s response to problems in the agriculture industry, and a solution to a critical constraint of labour shortages in the sector. By gathering qualitative data through interviews, news items and academic publications, observing the farming community’s engagement with digital technology innovation at farm show, and applying the Innovation Opportunity Space (IOS) analytical framework, this study identified that an autonomous DOT™ offers a solution for farming problems. Other firms are incorporating the DOT™ technology into their manufacturing operations through licensing agreements and early farmer adoption is positive. The process of innovation was based on synthesis of tacit knowledge (experience-based knowledge of farming and agribusiness) and codified knowledge (drawing on computer programing), while public policy facilitated the hiring of trained university students who remain with the SME as advocates for smart farming. There remain some gaps: public policy for safe deployment of smart farming innovation is lagging behind invention and commercialization; new business models for manufacture and commercialization of high-tech equipment are just emerging and data ownership and control remains unresolved; and evidence of the value of smart farming technologies to farmers and the larger social system remains scant

Digital Transformation in Smart Farm and Forest Operations Needs Human-Centered AI: Challenges and Future Directions

The main impetus for the global efforts toward the current digital transformation in almost all areas of our daily lives is due to the great successes of artificial intelligence (AI), and in particular, the workhorse of AI, statistical machine learning (ML). The intelligent analysis, modeling, and management of agricultural and forest ecosystems, and of the use and protection of soils, already play important roles in securing our planet for future generations and will become irreplaceable in the future. Technical solutions must encompass the entire agricultural and forestry value chain. The process of digital transformation is supported by cyber-physical systems enabled by advances in ML, the availability of big data and increasing computing power. For certain tasks, algorithms today achieve performances that exceed human levels. The challenge is to use multimodal information fusion, i.e., to integrate data from different sources (sensor data, images, *omics), and explain to an expert why a certain result was achieved. However, ML models often react to even small changes, and disturbances can have dramatic effects on their results. Therefore, the use of AI in areas that matter to human life (agriculture, forestry, climate, health, etc.) has led to an increased need for trustworthy AI with two main components: explainability and robustness. One step toward making AI more robust is to leverage expert knowledge. For example, a farmer/forester in the loop can often bring in experience and conceptual understanding to the AI pipeline—no AI can do this. Consequently, human-centered AI (HCAI) is a combination of “artificial intelligence” and “natural intelligence” to empower, amplify, and augment human performance, rather than replace people. To achieve practical success of HCAI in agriculture and forestry, this article identifies three important frontier research areas: (1) intelligent information fusion; (2) robotics and embodied intelligence; and (3) augmentation, explanation, and verification for trusted decision support. This goal will also require an agile, human-centered design approach for three generations (G). G1: Enabling easily realizable applications through immediate deployment of existing technology. G2: Medium-term modification of existing technology. G3: Advanced adaptation and evolution beyond state-of-the-art