This paper presents the hardware/software
architecture of the Sky-Eye UAS prototype. In particular
it details the hardware of the prototype, its operational
concept and the software avionics architecture. The
software architecture is named UAS Service Abstraction
Layer (USAL) and consists on the set of standard services
required for most UAS missions. The USAL is a
distributed architecture which follows the publish /
subscribe communication paradigm, allowing fast
development of new functionalities. We describe the
USAL services required to properly manage the remote
sensing mission of hot spot detection. This includes the
sensor management, data storage, communications,
image processing, flight plan management and mission
management.Postprint (published version

Barrado Muxí, Cristina

Lema Rosas, Juan Manuel

López Rubio, Juan

Pastor Llorens, Enric

Royo Chic, Pablo

Solé, M.

English

UPCommons. Portal del coneixement obert de la UPC

UAS ARCHITECTURE FOR FOREST FIRE REMOTE SENSING*  P. ROYOa, E. PASTORa, M.  SOLÉa, J.M. LEMAa, J. LÓPEZa, C. BARRADOa  aTechnical University of Catalonia, 08860 Castelldefels, SPAIN – proyo@ac.upc.edu                                                             * This work has been partially supported by Spanish Ministry of Education under contract TIN2010-18989 and by AGAUR under contract VALTEC-09-2-0099. Abstract – This paper presents the hardware/software architecture of the Sky-Eye UAS prototype.  In particular it details the hardware of the prototype, its operational concept and the software avionics architecture. The software architecture is named UAS Service Abstraction Layer (USAL) and consists on the set of standard services required for most UAS missions.  The USAL is a distributed architecture which follows the publish / subscribe communication paradigm, allowing fast development of new functionalities.  We describe the USAL services required to properly manage the remote sensing mission of hot spot detection.  This includes the sensor management, data storage, communications, image processing, flight plan management and mission management.     Keywords: Robotics, Forest Fire, Service Oriented Architecture, UAS, Distributed Architecture  1. INTRODUCTION  The use of Unmanned Aircraft System (UAS) in remote sensing works is becoming a reality. In forest fire scenarios we propose the use of UAS for semi-real-time hot spot detection. In this paper we present Sky-Eye, a rotorcraft prototype designed to provide significant advantages in terms of flexibility, precision, cost, and safety.  Sky-Eye is specifically designed for Mediterranean forest fires, as a support tool for the Fire Services to improve tactical decisions.  More than 50,000 fires are detected in South Europe each year which burnt hundreds of thousands of hectares every year.  Figure 1 shows the last decade most critical fires location and year in South Europe (Castellnou, 2009.  For example, more than 270,000 hectares of vegetation burned and more than 110 villages were affected directly by the fire front only in Greece during the catastrophic fire season of 2007 (Xanthopoulos, 2008).      This paper proposes the use of an Unmanned Aircraft Systems (UAS) as a valid alternative for semi-real-time wild fire surveillance. UAS have the flexibility of an aircraft for working as fast remote sensing tool.  The cost of operating UAS, once the legal aspects are defined, will be on a tenth of operating and aircraft.  And more important, the absence of a pilot on board is absolutely safer mean than traditional aircraft for obtaining real time remote sensing data in dangerous scenarios like a wild fire.   Several projects have proposed the use of UAS in forest fires: COMETS (Martínez-de-Dios, 2007) had as main objective the design and implementation of a distributed heterogeneous fleet of Unmanned Aerial Vehicles (UAV) for cooperative detection of forest fires and their monitoring using.  Both helicopters and airships were included.  COMETS demonstrated the system in forest fire simulated scenarios.  Towards the end of 2007, the UK West Midlands Fire Service became the first urban fire and rescue service to successfully bring into actual operation an UAS (WMFS, 2008).  The UAS, named as Incident Support Imaging System (ISIS), was deployed in a major fire, resulted in the deaths of four firefighters, to provide live video for fire investigators.  The AWARE European project (AWARE, 2009) had as general objective the design, development and experimentation of a software platform providing the functionalities required for the cooperation among aerial flying objects. The cooperating objects were autonomous helicopters, but also ground sensors and actuators carried by people and vehicles connected in a wireless network.  The platform enabled the operation in difficult access scenarios and with no infrastructure for communications.  The project demonstration was performed in May 2009 in which three autonomous vehicles were involved and fire monitoring was tested using cameras on board.  The Wildfire Research and Applications Partnership (WRAP) was successfully launched in 2003 by the NASA and highlights the collaborative partnership with the US Forest Service in wildfire science and applications (Ambrosia, 2008).  The mission’s project is to support efforts to the fire and disaster community and has been extended through 2010.  The objective of the project is to prove new technologies for increasing the information content and remote sensing data in the shortest time.  The outcomes are UAS operational wildfire monitoring technologies, in particular, the Ikhana, its on board scanner AMS, and the novel tool Wildfire Decision Support System used within the US Forest Service and the other fire mitigation and management agencies.  The Ikhana has supported fire surveillance missions covering the western US wildfire season.  The UAS was target to detect post-fire hot-spots located at the perimeter of fire. Figure 1 Largest Mediterranean forest fires in last decade   In this paper we present the UAS prototype being built as part of the Sky-Eye project. Its main application is foreseen in the European Mediterranean area and for post-fire hot spot detection.  Just after a wild fire has been controlled, two or three days of fire monitoring task has to be maintained in order to prevent fire reactivation.  The cost of post fire monitoring is expensive because needs large number of resources.  A UAS equipped with thermal cameras and flying over the area can provide a precise map of hot spots in few hours.  Early reported hot spots will avoid having ground brigades operating blindly.  The paper is organized as follows:  Section 2 presents an overview of the Sky-Eye hardware platform prototype.  Section 3 reviews the concept of operation of the prototype. Section 4 provides the details of the software architecture, based on the Service Oriented Architecture (SOA) paradigm.  Finally in Section 5 conclusions and future work are given.   2. SKY-EYE HARDWARE PROTOTYPE  A UAS is composed of three parts: The Air Segment (AS) which includes the airframe and all the devices on-board, the payload. The Ground Control Station (GCS) used to control the air segment flight and the mission from the ground. The Communication Channel (CC) between the AS and the GCS. This channel is also used to distribute the acquired data in real time.  Sky-Eye is a UAS prototype which has been built around existing Commercial Off-The-Shelf (COTS) technology that can be immediately deployed on the field at a reasonable cost and time.  Information is gathered by the on-board sensors, processed and then displayed in such a way it can be immediately exploited by the GCS but also to individual operative squads.  The design of the system has been done to be able to operate at day and at night, and with smoke conditions.   Figure 2 shows the current hardware development state of the Sky-Eye prototype AS.  The avionics include the autopilot, a number of embedded computers to properly manage the mission, a local area network, several sensors, data storage, communications devices, image processing units and mission management capacities.  The frame is a VARIO RC helicopter enlarged with larger carbon fiber skids to hold the payload.  Sensors are basically the two cameras installed on board: a high definition visual camera, and a thermal camera.  Additionally, there is a video camera placed in front of the AS for the pilot in command.  This platform has been selected according to the targeted Mediterranean wild fires and to the cost objectives.  A low/medium altitude tactic UAS was preferred for cost availability and due to the wild fire sizes.  The CC is for the command and control, and for the payload.  A WLAN communication (802.11a, 5GHz) is used.  On the other side the GCS (see Figure 3) has been developed following concepts such as the easy development, flexibility, interoperability and cost reduction.  Therefore, the design of a GCS was for a clear purpose: centralizing all the elements normally required for the control and monitoring of a UAS, such as computers, data transceivers, video receivers, etc., in a single transportable module.  Its internal structure permits easy manipulation and upgrades while being resilient enough to stand heavy duty transportation. The GCS uses two computers running the control applications and video monitoring in parallel. It has a slot for the autopilot ground unit, a slot for the analog video receiver and an analog video capture card.   3. OPERATIONAL WORKFLOW  The goal of the prototype is to reuse the same operational and functional concept for remote sensing in different post-fire areas with just some user/system-parameterization. The operational concept has to be flexible enough and reusable in order to use equivalent tasks for similar missions such as fire prevention activities, fire detection, monitoring and, as in this case, post-fire analysis.  Figure 2 Sky-Eye Air segment hardware platform overview      The system operates as shown in Figure 4:  When the exploration perimeter and parameters are defined, the system is ready to start the hot-spots analysis. The next step is to determine the optimal way to explore the post-fire area. The hot-spots analysis can be divided into several scan patterns depending on the perimeter shape and exploration parameters.  Once the optimal number of scans has been identified it is time to define the scan parameters basically in terms of flight planning. Parameters such as minimum row separation, speed and altitude are needed for each scan pattern. Once these parameters are computed, the UAS is ready to perform the scan pattern over the first analysis area to get and store the terrain samples.      These images have to be stored with the UAS telemetry attached in order to post-process and correct thermal and visual images on the ground or on board. After the samples have been stored, the basic data such as hot-spots coordinates are retrieved automatically by the operator. Each hot-spot is reviewed by a person who has enough fire knowledge to discriminate important data from irrelevant data. Depending on the hot-spot relevance the fire operator needs to be able to download thermal, visual1 or fuse2 images of the important                                                            1 Visual images are very sensitive to changes in illumination, the weather conditions, or the presence of other external factors such as smoke; but they are needed to discard false alarms caused by other hot objects, such as vehicles, or solar reflections, such as water.  The aim of the visual image in the proposed system is only to provide additional information about the surrounding of the hot spot.  hot-spots. At the same time that the system progresses in its scanning over the fire area, the fire perimeter has to be computed. This new perimeter will be provided to the fire operator in order to be accepted or modified.  Another important analysis functionality is to correlate hot-spots. Given that the image overlaps, the same hot-spot can be detected in many different samples. It is the task of the system to correlate different samples in order to provide the ground operator with the best Hot-Spot sample. This task can be developed in parallel with the rest of the hot-spot and perimeter analysis process.  4. SKY-EYE SOFTWARE ARCHITECTURE  This section describes the software architecture of the Sky-Eye prototype.  In our proposal, we distribute the different constrains different Services. Each Service implements a specific task thus we distribute the system complexity.  This strategy increases the interconnection complexity.  In order to solve this interconnection problem, we have designed and implemented our own communication middleware.  On top of our middleware there is a set of available services that define the UAS System Abstraction Layer (USAL).  The USAL offers an abstraction layer to access the UAS payload devices in a uniform way.  It provides end users with efficient and secure access to all hardware elements.  The USAL provides useful features designed to simplify the complexity of the development of the UAS target application, in this paper, the hot spot mission.  Figure 5 illustrates the main USAL components required for hot spots remote sensing mission.  The top of the figure presents the services that are executing on the AS.  At the left side of the figure and connected with the autopilot, the virtual autopilot service is designed to hide the specificities of the selected autopilot.  This service virtualizes the access to the flight capacities.  It works in conjunction with the flight plan manager, a service that translates dynamically complex flight                                                                                          2 Current fusion camera models do not seem suitable to be boarded on the UAS because they have less resolution.  Also Hyper-spectral cameras or scanners have been discarded in the Sky-Eye prototype because their high cost. On the contrary, we have followed other researcher proposal (Dunagan, 2005) which shows that COTS cameras at low altitudes are able to obtain high quality images.  Figure 3 Sky-Eye ground segment hardware platform overview Figure 4 Hot-Spots remote sensing operation work  flow     profiles into autopilot orders.  These two services, together with the autopilot, form the stack of the flight capabilities of the UAS.  The flight plan manager is supported by the contingency alerts coming from the engine manager and the electrical manager.  An intermediate service, the contingency manager is responsible of monitor data from the engine and the electrical system, detect contingencies, and suggest reactions to the flight plan manager. Also the flight plan manager is supported by the long term planner to develop new and intelligent routes in case of mission updates or large flight deviations.  The counterpart of the mission monitor is the mission manager.  It configures the mission details such as altitude, speed, monitoring strategy, etc.  The long term planner determines the parameters for the best scan to cover the fire area.  The camera controller service manages the camera/s of the UAS issuing commands to the corresponding camera service.  The real time image processing service subscribes to the telemetry and to the images, associates the aircraft position to each image, and published a geo-referenced image.  It also analyses the thermal images looking for hot spots using simple image processing algorithm.  The digital elevation model service helps the image processing in geo-locating the hot spots coordinates.  When the fire operator requests for any specific image, the schedule mission communications service will choose the information to be send.  Finally, the storage service subscribes the most data and stores it for further processing.  On the GCS the flight monitor service is the remote cockpit of the UAS.  The actual information about the flight is mainly coming from the published messages of the virtual autopilot service.  The flight monitor subscribes to these messages, processes their data and transforms it to visual information in a screen in real time. Also on ground a flight plan monitor service acts as a secondary flight display.  It has been designed as a specific tool for dynamically interact with the flight plan as the mission develops.  A third service on the GCS, the mission monitor is in charge of mission data.  In this paper, the mission target is to obtain and show the map of hot spots.  The mission monitor also allows the ground operator to select specific images, to fusion them or to geo-localize visual elements of the images.   5. CONCLUSIONS  UAS provide significant advantages in front of current aerial means in terms of flexibility, precision, cost, and safety.  In a forest fire scenario a UAS can fly at low altitude, taking visual and thermal images of the fire area, while performing real time processing of the pictures to detect hot spots.  Navigation information can be used to geo-reference the hot spots and early notify them to firefighters.  On demand, the thermal and visual pictures are fused and sent to the GCS.    This paper presents an overview of hardware/software architecture of the Sky-Eye UAS prototype being developed for such a scenario of hot spots detection.  The architecture is complemented with the details of the remote sensing workflow concept and the involved services needed to implement the hot spot mission.  The current development of the prototype is on an advanced phase and three flight tests have been already done for fine tuning of the payload related services.  The Virtual Autopilot Service provides position and attitude data to the rest of the services, but the rest of the functionalities of flight services have only been tested on simulation because the autopilot has not been configured yet for this rotorcraft.  This is the next and most important milestone in the Sky-Eye project.  REFERENCES   References from Technical Conferences V.G. Ambrosia, E. Hinkley, "NASA Science Serving Society: Improving Capabilities for Fire Characterization to Effect Reduction in Disaster Losses". IEEE International on Geoscience and Remote Sensing Symposium, 2008. IGARSS 2008. Boston July 2008.  M. Castellnou, “Canvi global i foc forestal a Catalunya. 10 anys de GRAF”, Jornades Tecniques del Cos de Bombers. De Catalunya. Capacitat de gestió dels incedis forestals. Nov 2009, Girona.  S. Dunagan, D. Sullivan, R. Slye, B. Lobitz, E. Hinkley, and S. Herwitz, “A Multi-Sensor Imaging Payload for Mission-Adaptive Remote Sensing Applications” AIAA Infotech@Aerospace, Arlington, VI, Sep 2005.   J. R. Martínez-de-Dios, L. Merino, A. Ollero, L. M. Ribeiro and X. Viegas, “Multi-UAV Experiments: Application to Forest Fires”, Springer Tracts in Advanced Robotics, 2007, Volume 37/2007, 207-228.  G. Xanthopoulos, 2008. “People and the Mass Media during the fire disaster days of 2007 in Greece”. pp. 494-506. Proc. Int. Bushfire Research Conference on “Fire, Environment and Society”, September 1-3, 2008, Adelaide, Australia.  References from Websites AWARE, 2009. “Platform for Autonomous self-deploying and operation of Wireless sensor-actuator networks cooperating with AeRial objEcts.”, AWARE Project IST-2006-33579. http:// grvc.us.es/aware/  WMFS, "Annual Service Plan 2008". http://www.wmfs.net/digitalAssets/101/101059_Annual_Service_Plan_2008_2009_Updated_Document_-_16_June_08.pdf  Figure 5  Services for the Sky-Eye mission 

UAS architecture for forest fire remote sensing

This paper presents the hardware/software

architecture of the Sky-Eye UAS prototype. In particular

it details the hardware of the prototype, its operational

concept and the software avionics architecture. The

software architecture is named UAS Service Abstraction

Layer (USAL) and consists on the set of standard services

required for most UAS missions. The USAL is a

distributed architecture which follows the publish /

subscribe communication paradigm, allowing fast

development of new functionalities. We describe the

USAL services required to properly manage the remote

sensing mission of hot spot detection. This includes the

sensor management, data storage, communications,

image processing, flight plan management and mission

management

UPCommons

https://upcommons.upc.edu/bitstream/2117/16549/1/211104015final00187.pdf

UAS architecture for forest fire remote sensing

Abstract

Similar works

Full text

Available Versions

UPCommons. Portal del coneixement obert de la UPC

UPCommons