245 research outputs found
MISFIT-V: Misaligned Image Synthesis and Fusion using Information from Thermal and Visual
Detecting humans from airborne visual and thermal imagery is a fundamental
challenge for Wilderness Search-and-Rescue (WiSAR) teams, who must perform this
function accurately in the face of immense pressure. The ability to fuse these
two sensor modalities can potentially reduce the cognitive load on human
operators and/or improve the effectiveness of computer vision object detection
models. However, the fusion task is particularly challenging in the context of
WiSAR due to hardware limitations and extreme environmental factors. This work
presents Misaligned Image Synthesis and Fusion using Information from Thermal
and Visual (MISFIT-V), a novel two-pronged unsupervised deep learning approach
that utilizes a Generative Adversarial Network (GAN) and a cross-attention
mechanism to capture the most relevant features from each modality.
Experimental results show MISFIT-V offers enhanced robustness against
misalignment and poor lighting/thermal environmental conditions compared to
existing visual-thermal image fusion methods
WiSARD: A Labeled Visual and Thermal Image Dataset for Wilderness Search and Rescue
Sensor-equipped unoccupied aerial vehicles (UAVs) have the potential to help
reduce search times and alleviate safety risks for first responders carrying
out Wilderness Search and Rescue (WiSAR) operations, the process of finding and
rescuing person(s) lost in wilderness areas. Unfortunately, visual sensors
alone do not address the need for robustness across all the possible terrains,
weather, and lighting conditions that WiSAR operations can be conducted in. The
use of multi-modal sensors, specifically visual-thermal cameras, is critical in
enabling WiSAR UAVs to perform in diverse operating conditions. However, due to
the unique challenges posed by the wilderness context, existing dataset
benchmarks are inadequate for developing vision-based algorithms for autonomous
WiSAR UAVs. To this end, we present WiSARD, a dataset with roughly 56,000
labeled visual and thermal images collected from UAV flights in various
terrains, seasons, weather, and lighting conditions. To the best of our
knowledge, WiSARD is the first large-scale dataset collected with multi-modal
sensors for autonomous WiSAR operations. We envision that our dataset will
provide researchers with a diverse and challenging benchmark that can test the
robustness of their algorithms when applied to real-world (life-saving)
applications
Road Condition Detection and Emergency Rescue Recognition Using On-Board UAV in the Wildness
Unmanned aerial vehicle (UAV) vision technology is becoming increasingly important, especially in wilderness rescue. For humans in the wilderness with poor network conditions and bad weather, this paper proposes a technique for road extraction and road condition detection from video captured by UAV multispectral cameras in real-time or pre-downloaded multispectral images from satellites, which in turn provides humans with optimal route planning. Additionally, depending on the flight altitude of the UAV, humans can interact with the UAV through dynamic gesture recognition to identify emergency situations and potential dangers for emergency rescue or re-routing. The purpose of this work is to detect the road condition and identify emergency situations in order to provide necessary and timely assistance to humans in the wild. By obtaining a normalized difference vegetation index (NDVI), the UAV can effectively distinguish between bare soil roads and gravel roads, refining the results of our previous route planning data. In the low-altitude human–machine interaction part, based on media-pipe hand landmarks, we combined machine learning methods to build a dataset of four basic hand gestures for sign for help dynamic gesture recognition. We tested the dataset on different classifiers, and the best results show that the model can achieve 99.99% accuracy on the testing set. In this proof-of-concept paper, the above experimental results confirm that our proposed scheme can achieve our expected tasks of UAV rescue and route planning
An Appearance-Based Tracking Algorithm for Aerial Search and Rescue Purposes
The automation of the Wilderness Search and Rescue (WiSAR) task aims for high levels of understanding of various scenery. In addition, working in unfriendly and complex environments may cause a time delay in the operation and consequently put human lives at stake. In order to
address this problem, Unmanned Aerial Vehicles (UAVs), which provide potential support to the
conventional methods, are used. These vehicles are provided with reliable human detection and
tracking algorithms; in order to be able to find and track the bodies of the victims in complex
environments, and a robust control system to maintain safe distances from the detected bodies.
In this paper, a human detection based on the color and depth data captured from onboard sensors
is proposed. Moreover, the proposal of computing data association from the skeleton pose and a
visual appearance measurement allows the tracking of multiple people with invariance to the scale,
translation and rotation of the point of view with respect to the target objects. The system has been
validated with real and simulation experiments, and the obtained results show the ability to track
multiple individuals even after long-term disappearances. Furthermore, the simulations present the
robustness of the implemented reactive control system as a promising tool for assisting the pilot to
perform approaching maneuvers in a safe and smooth manner.This research is supported by Madrid Community project SEGVAUTO 4.0 P2018/EMT-4362)
and by the Spanish Government CICYT projects (TRA2015-63708-R and TRA2016-78886-C3-1-R), and Ministerio
de EducaciĂłn, Cultura y Deporte para la FormaciĂłn de Profesorado Universitario (FPU14/02143). Also,
we gratefully acknowledge the support of the NVIDIA Corporation with the donation of the GPUs used for
this research
Automated Complexity-Sensitive Image Fusion
To construct a complete representation of a scene with environmental obstacles such as fog, smoke, darkness, or textural homogeneity, multisensor video streams captured in diferent modalities are considered. A computational method for automatically fusing multimodal image streams into a highly informative and unified stream is proposed. The method consists of the following steps: 1. Image registration is performed to align video frames in the visible band over time, adapting to the nonplanarity of the scene by automatically subdividing the image domain into regions approximating planar patches
2. Wavelet coefficients are computed for each of the input frames in each modality
3. Corresponding regions and points are compared using spatial and temporal information across various scales
4. Decision rules based on the results of multimodal image analysis are used to combine thewavelet coefficients from different modalities
5. The combined wavelet coefficients are inverted to produce an output frame containing useful information gathered from the available modalities
Experiments show that the proposed system is capable of producing fused output containing the characteristics of color visible-spectrum imagery while adding information exclusive to infrared imagery, with attractive visual and informational properties
Facilitating Internet of Things on the Edge
The evolution of electronics and wireless technologies has entered a new era, the Internet of Things (IoT). Presently, IoT technologies influence the global market, bringing benefits in many areas, including healthcare, manufacturing, transportation, and entertainment.
Modern IoT devices serve as a thin client with data processing performed in a remote computing node, such as a cloud server or a mobile edge compute unit. These computing units own significant resources that allow prompt data processing. The user experience for such an approach relies drastically on the availability and quality of the internet connection. In this case, if the internet connection is unavailable, the resulting operations of IoT applications can be completely disrupted. It is worth noting that emerging IoT applications are even more throughput demanding and latency-sensitive which makes communication networks a practical bottleneck for the service provisioning. This thesis aims to eliminate the limitations of wireless access, via the improvement of connectivity and throughput between the devices on the edge, as well as their network identification, which is fundamentally important for IoT service management.
The introduction begins with a discussion on the emerging IoT applications and their demands. Subsequent chapters introduce scenarios of interest, describe the proposed solutions and provide selected performance evaluation results. Specifically, we start with research on the use of degraded memory chips for network identification of IoT devices as an alternative to conventional methods, such as IMEI; these methods are not vulnerable to tampering and cloning. Further, we introduce our contributions for improving connectivity and throughput among IoT devices on the edge in a case where the mobile network infrastructure is limited or totally unavailable. Finally, we conclude the introduction with a summary of the results achieved
Telemedicine and its application in telemedicine management
Telemedicine can be defined as the extensive depiction of
providing medical and healthcare services by using telecommunications structures. Information Technology (IT) which covers controlling, interactive media, pattern recognition, knowledge management, image and signal processing: have empowered an extensive array of telemedicine applications to be supported.
The joined consequence of the expansion of the global population and maturing populace in most advanced countries offersascent to an increasing interest on the public health system. The effect on public health systems in various nations were further empowered by a change in the lifestyle and environmental contamination
which further increases the demand for health systems.
This is obvious from the pattern of perpetual ailments and complication arising from obesity-related conditions which attack youthful individuals over the previous decade. Currently, the financial prosperity which blesses the present generation is a result of the diligent work done by our fore fathers and the rapacious
exploitation of the natural resources that will eventually cause various issues to the upcoming generation. Therefore, we should seize the responsibility of caring for the elderly who tirelessly
sacrificed their time for the betterment of the current generation. Nevertheless, we are attempting to upgrade medicinal technology to enhance our well-being, and to furnish a supportable healthcare system for the upcoming era. Telemedicine is poised as a means of fulfilling our obligations to the adolescents and the elderly
- …