1 research outputs found
An embedded deep learning system for augmented reality in firefighting applications
Firefighting is a dynamic activity, in which numerous operations occur
simultaneously. Maintaining situational awareness (i.e., knowledge of current
conditions and activities at the scene) is critical to the accurate
decision-making necessary for the safe and successful navigation of a fire
environment by firefighters. Conversely, the disorientation caused by hazards
such as smoke and extreme heat can lead to injury or even fatality. This
research implements recent advancements in technology such as deep learning,
point cloud and thermal imaging, and augmented reality platforms to improve a
firefighter's situational awareness and scene navigation through improved
interpretation of that scene. We have designed and built a prototype embedded
system that can leverage data streamed from cameras built into a firefighter's
personal protective equipment (PPE) to capture thermal, RGB color, and depth
imagery and then deploy already developed deep learning models to analyze the
input data in real time. The embedded system analyzes and returns the processed
images via wireless streaming, where they can be viewed remotely and relayed
back to the firefighter using an augmented reality platform that visualizes the
results of the analyzed inputs and draws the firefighter's attention to objects
of interest, such as doors and windows otherwise invisible through smoke and
flames.Comment: Accepted to ICMLA Special Session on Deep Learnin