FPGA-based real-time moving target detection system for unmanned aerial vehicle application

Moving target detection is the most common task for Unmanned Aerial Vehicle (UAV) to find and track object of interest from a bird's eye view in mobile aerial surveillance for civilian applications such as search and rescue operation. The complex detection algorithm can be implemented in a real-time embedded system using Field Programmable Gate Array (FPGA). This paper presents the development of real-time moving target detection System-on-Chip (SoC) using FPGA for deployment on a UAV. The detection algorithm utilizes area-based image registration technique which includes motion estimation and object segmentation processes. The moving target detection system has been prototyped on a low-cost Terasic DE2-115 board mounted with TRDB-D5M camera. The system consists of Nios II processor and stream-oriented dedicated hardware accelerators running at 100 MHz clock rate, achieving 30-frame per second processing speed for 640 × 480 pixels' resolution greyscale videos

Utilizing Dual Information for Moving Target Search Trajectory Optimization

Various recent events have shown the enormous importance of maritime search-and-rescue missions. By reducing the time to find floating victims at sea, the number of casualties can be reduced. A major improvement can be achieved by employing autonomous aerial systems for autonomous search missions, allowed by the recent rise in technological development. In this context, the need for efficient search trajectory planning methods arises. The objective is to maximize the probability of detecting the target at a certain time k, which depends on the estimation of the position of the target. For stationary target search, this is a function of the observation at time k. When considering the target movement, this is a function of all previous observations up until time k. This is the main difficulty arising in solving moving target search problems when the duration of the search mission increases. We present an intermediate result for the single searcher single target case towards an efficient algorithm for longer missions with multiple aerial vehicles. Our primary aim in the development of this algorithm is to disconnect the networks of the target and platform, which we have achieved by applying Benders decomposition. Consequently, we solve two much smaller problems sequentially in iterations. Between the problems, primal and dual information is exchanged. To the best of our knowledge, this is the first approach utilizing dual information within the category of moving target search problems. We show the applicability in computational experiments and provide an analysis of the results. Furthermore, we propose well-founded improvements for further research towards solving real-life instances with multiple searchers

Detection and tracking for radar simulation using MATLAB

The objective of the project is to simulate the real time Radar detection and tracking operations using MATLAB software. Radar system use modulated waveforms and directive antennas to transmit electromagnetic energy into a specific volume in space to search for targets. Objects (targets) within a search volume will reflect portions of this energy (radar returns or echoes) back to the radar. These echoes are then processed by radar receiver to extract target information such as range. Velocity, angular position, and other target identifying characteristics. The project mainly concentrates on the radar displays and different radar types to collect the information of the flying objects, such as the range, speed, distance, angles. The display types are A-scope, B-scope, C-scope, PPI, and RHI, which are used in modern radars. While others are either obsolete or are found only in very specialized applications. Signals displayed on these scopes can be raw video, synthetic video (detected video) or computer-generated symbols. The radar types consider in the project are CWT (Continuous Wave Transmission), Pulse, Doppler, and MTI (Moving Target Indicator). For each display, all the values related to the object are calculated in different patterns and graphs for the corresponding formulated values and angles

Time-based selection in complex displays : visual marking does not occur in multi-element asynchronous dynamic (MAD) search

In visual search, a preview benefit occurs when half of the distractor items (the preview set) are presented before the remaining distractor items and the target (the search set). Separating the display across time allows participants to prioritize the search set, leading to increased search efficiency. To date, such time-based selection has been examined using relatively simple types of search displays. However, recent research has shown that when displays better mimic real-world scenes by including a combination of stationary, moving and luminance-changing items (Multi-element Asynchronous Dynamic [MAD] displays), previous search principles reported in the literature no longer apply. In the current work, we examined time-base selection in MAD search conditions. Overall the findings illustrated an advantage for processing new items based on overall RTs but no advantage in terms of search rates. In the absence of a speed–accuracy trade-off no preview benefit emerged when using more complex MAD stimuli

Optimization based trajectory planning for real-time 6DoF robotic patient motion compensation systems

Purpose: Robotic stabilization of a therapeutic radiation beam with respect to a dynamically moving tumor target can be accomplished either by moving the radiation source, the patient, or both. As the treatment beam is on during this process, the primary goal is to minimize exposure of normal tissue to radiation as much as possible when moving the target back to the desired position. Due to the complex mechanical structure of 6 degree-of-freedom (6DoF) robots, it is not intuitive as to what 6 dimensional (6D) correction trajectory is optimal in achieving such a goal. With proportional-integrative-derivative (PID) and other controls, the potential exists that the controller may generate a trajectory that is highly curved, slow, or suboptimal in that it leads to unnecessary exposure of healthy tissue to radiation. This work investigates a novel feedback planning method that takes into account a robot’s mechanical joint structure, patient safety tolerances, and other system constraints, and performs real-time optimization to search the entire 6D trajectory space in each time cycle so it can respond with an optimal 6D correction trajectory. Methods: Computer simulations were created for two 6DoF robotic patient support systems: a Stewart-Gough platform for moving a patient’s head in frameless maskless stereotactic radiosurgery, and a linear accelerator treatment table for moving a patient in prostate cancer radiation therapy. Motion planning was formulated as an optimization problem and solved at real-time speeds using the L-BFGS algorithm. Three planning methods were investigated, moving the platform as fast as possible (platform-D), moving the target along a straight-line (target-S), and moving the target based on the fastest descent of position error (target-D). Both synthetic motion and prior recorded human motion were used as input data and output results were analyzed. Results: For randomly generated 6D step-like and sinusoidal synthetic input motion, target-D planning demonstrated the smallest net trajectory error in all cases. On average, optimal planning was found to have a 45% smaller target trajectory error than platform-D control, and a 44% smaller target trajectory error than target-S planning. For patient head motion compensation, only target-D planning was able to maintain a ≤0.5mm and ≤0.5deg clinical tolerance objective for 100% of the treatment time. For prostate motion, both target-S planning and target-D planning outperformed platform-D control. Conclusions: A general 6D target trajectory optimization framework for robotic patient motion compensation systems was investigated. The method was found to be flexible as it allows control over various performance requirements such as mechanical limits, velocities, acceleration, or other system control objectives.</p

Minimum time search in real-world scenarios using multiple UAVs with onboard orientable cameras

This paper proposes a new evolutionary planner to determine the trajectories of several Unmanned Aerial Vehicles (UAVs) and the scan direction of their cameras for minimizing the expected detection time of a nondeterministically moving target of uncertain initial location. To achieve this, the planner can reorient the UAVs cameras and modify the UAVs heading, speed, and height with the purpose of making the UAV reach and the camera observe faster the areas with high probability of target presence. Besides, the planner uses a digital elevation model of the search region to capture its influence on the camera likelihood (changing the footprint dimensions and the probability of detection) and to help the operator to construct the initial belief of target presence and target motion model. The planner also lets the operator include intelligence information in the initial target belief and motion model, in order to let him/her model real-world scenarios systematically. All these characteristics let the planner adapt the UAV trajectories and sensor poses to the requirements of minimum time search operations over real-world scenarios, as the results of the paper, obtained over 3 scenarios built with the modeling aid-tools of the planner, show.This work was supported by Airbus under SAVIER AER30459 projec

Semantic Mechanical Search with Large Vision and Language Models

Moving objects to find a fully-occluded target object, known as mechanical search, is a challenging problem in robotics. As objects are often organized semantically, we conjecture that semantic information about object relationships can facilitate mechanical search and reduce search time. Large pretrained vision and language models (VLMs and LLMs) have shown promise in generalizing to uncommon objects and previously unseen real-world environments. In this work, we propose a novel framework called Semantic Mechanical Search (SMS). SMS conducts scene understanding and generates a semantic occupancy distribution explicitly using LLMs. Compared to methods that rely on visual similarities offered by CLIP embeddings, SMS leverages the deep reasoning capabilities of LLMs. Unlike prior work that uses VLMs and LLMs as end-to-end planners, which may not integrate well with specialized geometric planners, SMS can serve as a plug-in semantic module for downstream manipulation or navigation policies. For mechanical search in closed-world settings such as shelves, we compare with a geometric-based planner and show that SMS improves mechanical search performance by 24% across the pharmacy, kitchen, and office domains in simulation and 47.1% in physical experiments. For open-world real environments, SMS can produce better semantic distributions compared to CLIP-based methods, with the potential to be integrated with downstream navigation policies to improve object navigation tasks. Code, data, videos, and the appendix are available: https://sites.google.com/view/semantic-mechanical-searc

Convolutional neural network-based real-time object detection and tracking for parrot AR drone 2.

Recent advancements in the field of Artificial Intelligence (AI) have provided an opportunity to create autonomous devices, robots, and machines characterized particularly with the ability to make decisions and perform tasks without human mediation. One of these devices, Unmanned Aerial Vehicles (UAVs) or drones are widely used to perform tasks like surveillance, search and rescue, object detection and target tracking, parcel delivery (recently started by Amazon), and many more. The sensitivity in performing said tasks demands that drones must be efficient and reliable. For this, in this paper, an approach to detect and track the target object, moving or still, for a drone is presented. The Parrot AR Drone 2 is used for this application. Convolutional Neural Network (CNN) is used for object detection and target tracking. The object detection results show that CNN detects and classifies object with a high level of accuracy (98%). For real-time tracking, the tracking algorithm responds faster than conventionally used approaches, efficiently tracking the detected object without losing it from sight. The calculations based on several iterations exhibit that the efficiency achieved for target tracking is 96.5%

Pergerakan Pasukan Untuk Mengejar Musuh Bergerak Menggunakan D* Lite Berbasis Algoritma Pathfinding

The movement of agents in the Real Time Strategy game influenced by several factors, one of which is the technique of agent movement within the game environment. Pathfinding in a video game is an artificial intelligence algorithm how to find an agent moves the optimal way in which there are obstacles in the environment. This can be achieved by implementing a pathfinding algorithm to the game. This study of the D* Lite algorithm is able to plan a search path in an game environment, change the environment and moving target to be efficient, optimal and complete for agents and will describe some way of planning applications and provides a solid foundation for further research on methods of search re in artificial intelligence.Keywords: Agent Movement, Real Time Strategy, Pathfinding, D* Lite AbstrakPergerakan agen pada permainan Real Time Strategy dipengaruhi oleh beberapa faktor salah satunya adalah teknik pergerakan agen didalam lingkungan permainan. Pathfinding dalam video game merupakan algoritma kecerdasan buatan bagaimana cara sebuah agen bergerak menemukan jalan optimal dengan usaha minimal sampai pada tujuan. Hal ini bisa dicapai dengan mengimplementasikan suatu algoritma pathfinding pada game. Penelitian ini mengenai algoritma D* Lite yang mampu merencanakan pencarian jalur di lingkungan game dengan environment yang berubah sekaligus objek yang sebagai target bergerak dan menjadikan proses pengejaran target menjadi efisien bagi agen serta memberikan dasar yang kuat untuk penelitian lebih lanjut tentang metode pencarian ulang dalam kecerdasan buatanKata kunci: Agent Movement, Real Time Strategy, Pathfinding, D* Lit

Sensory Island Task (SIT): A New Behavioral Paradigm to Study Sensory Perception and Neural Processing in Freely Moving Animals

A central function of sensory systems is the gathering of information about dynamic interactions with the environment during self-motion. To determine whether modulation of a sensory cue was externally caused or a result of self-motion is fundamental to perceptual invariance and requires the continuous update of sensory processing about recent movements. This process is highly context-dependent and crucial for perceptual performances such as decision-making and sensory object formation. Yet despite its fundamental ecological role, voluntary self-motion is rarely incorporated in perceptual or neurophysiological investigations of sensory processing in animals. Here, we present the Sensory Island Task (SIT), a new freely moving search paradigm to study sensory processing and perception. In SIT, animals explore an open-field arena to find a sensory target relying solely on changes in the presented stimulus, which is controlled by closed-loop position tracking in real-time. Within a few sessions, animals are trained via positive reinforcement to search for a particular area in the arena (“target island”), which triggers the presentation of the target stimulus. The location of the target island is randomized across trials, making the modulated stimulus feature the only informative cue for task completion. Animals report detection of the target stimulus by remaining within the island for a defined time (“sit-time”). Multiple “non-target” islands can be incorporated to test psychometric discrimination and identification performance. We exemplify the suitability of SIT for rodents (Mongolian gerbil, Meriones unguiculatus) and small primates (mouse lemur, Microcebus murinus) and for studying various sensory perceptual performances (auditory frequency discrimination, sound source localization, visual orientation discrimination). Furthermore, we show that pairing SIT with chronic electrophysiological recordings allows revealing neuronal signatures of sensory processing under ecologically relevant conditions during goal-oriented behavior. In conclusion, SIT represents a flexible and easily implementable behavioral paradigm for mammals that combines self-motion and natural exploratory behavior to study sensory sensitivity and decision-making and their underlying neuronal processing