NASA's Autonomous Landing and Hazard Avoidance Technology (ALHAT) project is building a system for detecting hazards and automatically landing controlled vehicles safely anywhere on the Moon. The Flash Light Detection And Ranging (LIDAR) sensor is used to create on-the-fly a 3D map of the unknown terrain for hazard detection. As part of the ALHAT project, a hardware-in-the-loop (HIL) simulation testbed was developed to test the data processing, guidance, and navigation algorithms in real-time to prove their feasibility for flight. Replacing the Flash LIDAR camera with an emulator in the testbed provided a cheaper, safer, more feasible way to test the algorithms in a controlled environment. This emulator must have the same hardware interfaces as the LIDAR camera, have the same performance characteristics, and produce images similar in quality to the camera. This presentation describes the issues involved and the techniques used to create a real-time flash LIDAR emulator to support HIL simulation

Brewster, Paul F.

NASA Technical Reports Server

A t L di d H d A id (ALHAT)u onomous an ng an  azar  vo ance 
Flash LIDAR Emulator for HIL 
Si l imu at on
Paul Brewster
NASA Langley Research Center
ModSim World Conference 
October 14th, 2010
FSSBSlide 1Octorber 14th, 2010
https://ntrs.nasa.gov/search.jsp?R=20110010203 2019-08-30T15:16:29+00:00Z
A t L di d H d A id (ALHAT)
Autonomous Landing and Hazard 
Avoidance Technology
u onomous an ng an  azar  vo ance 
• Introduction
• Problem
• Emulator Development
• Application
• Results
FSSBSlide 2Octorber 14th, 2010
• Future Work
A t L di d H d A id (ALHAT)
Introduction
u onomous an ng an  azar  vo ance 
• Autonomous Landing and Hazard Avoiding 
Technology (ALHAT/ETDPO)
• Goal: Develop and deliver a TRL 6 lunar GNC 
descent and landing subsystem to place humans 
and cargo safely, precisely, repeatedly and 
autonomously anywhere on the lunar surface
under any lighting conditions within 10’s of meters of         
certified landing sites
A h D i th A h h th• pproac : ur ng e pproac  p ase, use ree 
LIDAR systems to automatically scan the landing 
site, detect safe landing areas, and navigate to a 
d t i d f
FSSBSlide 3Octorber 14th, 2010
e erm ne  sa e area
A t L di d H d A id (ALHAT)
Organization
u onomous an ng an  azar  vo ance 
• NASA Johnson Space Center
– Program Management
– Hardware-in-the-Loop Testing (HAST)  
– Avionics (APB)
• NASA Langley Research Center
– LIDAR Sensors 
– 6DOF Simulation (POST2)
• NASA Jet Propulsion Laboratory
– Hazard Detection Algorithms (TSAR)
– System Integration
• Draper Labs
– GNC algorithms
– Navigation Filter
• Applied Physics Laboratory
– Lunar Science
FSSBSlide 4Octorber 14th, 2010
– Lunar Terrain Models
A t L di d H d A id (ALHAT)
System Block Diagram
u onomous an ng an  azar  vo ance 
Op  Cmds 
TSAR
Processor
Flash 
LIDAR
System
Image frames/Maps
Cmds, NAV State
Navigation 
&
Guidance
Safe Sites
System Doppler 
LIDAR 
Velocimeter
Op  Cmds 
Velocity Measurements
LIDAR 
Altimeter
Op  Cmds 
 
FSSBSlide 5Octorber 14th, 2010
Avionics Processing Box Altitude Measurements
A t L di d H d A id (ALHAT)u onomous an ng an  azar  vo ance 
• Flash LIDAR
– Fires a laser pulse, measuring the time for the pulse 
t t b k t th l l ti th di to re urn ac  o e camera, ca cu a ng e s ance
– Uses an array of sensors to create an image of 
distances, rather than a single point
• Doppler LIDAR Velocimeter
– Fires three lasers in orthogonal directions
Determines velocity by measuring the doppler shift of–         
the return beam
• LIDAR Altimeter
– Fires a single laser pulse, measuring the time of 
return
– Calculates the distance using a single point
FSSBSlide 6Octorber 14th, 2010
A t L di d H d A id (ALHAT)
Problem
u onomous an ng an  azar  vo ance 
• Problem: How do we develop, test, and 
evaluate the ALHAT system in a lab 
environment?
– System components are being developed in four 
independent organizations 
– Impractical to use real LIDAR in a closed loop, 
hardware-in-the-loop, real-time lab environment
• Physical constraints
• Schedule constraints
• Cost constraints
• Solution: Use a functionally equivalent 
software emulator to replace the LIDAR 
FSSBSlide 7Octorber 14th, 2010
systems
A t L di d H d A id (ALHAT)
Emulator Requirements
u onomous an ng an  azar  vo ance 
• Complies with Flash LIDAR Interface
– Input
• Command & Control
O t t– u pu
• 256 x 256 Range Image
• 256 x 256 Intensity Image
• 30 Images/Second
Id ti l H d I t f• en ca  ar ware n er aces
– CameraLink (Images)
– RS-232 (Command & Control)
• Similar Image Quality  
– Noise/Signal Ratio
– Dead Pixels
– Precision
I t t i t HAST f k• n egra es n o  ramewor
– Input
• Sensor position & orientation (Ethernet)
• Lunar Terrain Data (Pre-computed) (5000 x 5000 DEM)
FSSBSlide 8Octorber 14th, 2010
A t L di d H d A id (ALHAT)
Emulator Interfaces
u onomous an ng an  azar  vo ance 
Terrain DEM
Pre-computed
Flash Lidar EmulatorSimulator Data
Range/Intensity 
Maps
TCP/IP CameraLink
Command/Control
CameraLink RS-232
FSSBSlide 9Octorber 14th, 2010
A t L di d H d A id (ALHAT)
Emulator Block Diagram
u onomous an ng an  azar  vo ance 
Simulation Environment
TCP/IP
HAST ICD
Channel Link
Interface
Range/Intensity 
Calculation
NASA PDS
Sensor
Model Sensor 
Controller
RS-232
Command
& Control
DemMaker
Terrain
Signal 
Processing 
Interface
Digital 
Signal 
Processing
Future Work
Time
Signal
Flash LIDAR Sensor Emulator
FSSBSlide 10Octorber 14th, 2010
A t L di d H d A id (ALHAT)
Range/Intensity Calculation
u onomous an ng an  azar  vo ance 
• Create a triangle mesh from the DEM data (5000*5000*4 triangles)
• For each pixel on the focal image plane, create a ray from the 
iti th h th i l (256*256 )camera pos on roug  e p xe   rays
• Range is the distance from the camera position to the point where 
the ray intersects the triangulated terrain
• Intensity is reflection*cos(incidence_angle) at that pixel
Focal PlaneCamera Position
Rays
Pixels
FSSBSlide 11Octorber 14th, 2010
Triangulated Terrain
A t L di d H d A id (ALHAT)
Range/Intensity Optimizations
u onomous an ng an  azar  vo ance 
Problem:  The non-real-time implementation of 
the Flash LIDAR takes several seconds per 
f H d I i l t th l t frame.  ow o  mp emen  e emu a or or 
real time?
– Test intersection of 65,536 rays with 100,000,000 
triangles, 30 times a second
Solution:  Use optimization techniques from 
several computer fields:  
– Computational Geometry
– Ray-Tracing 
– Parallel Processing
– Vector CPU processing
– General-Purpose computation on Graphics
FSSBSlide 12Octorber 14th, 2010
    
Processing Units
A t L di d H d A id (ALHAT)
Computational Geometry
u onomous an ng an  azar  vo ance 
Terrain recursively subdivided into 4 
partitions forming a 4-way hierarchical tree
List of triangles
Quad TreeUn-partitioned
Each ray is tested against the parent 
partition
If intersected, the ray is tested against the 
child partitions
Each ray is tested against each triangle
O(log n) per ray , n is number of trianglesO(n) per ray, n is number of triangles
 
Hierarchy Triangulated Terrain
Levels
FSSBSlide 13Octorber 14th, 2010
A t L di d H d A id (ALHAT)
Ray Tracing
u onomous an ng an  azar  vo ance 
1 bounding pyramid around the rays256 x 256 array of rays
Pyramid is recursively tested against eachEach ray is tested for intersection
Bundled RaysUn-Bundled Rays
      
partition
At the leaf partitions, intersect the 256 x 256 
rays against the triangles
     
O(1) partitions, O(n) leaf trianglesO(n), n is number of rays
Bounding Pyramid
FSSBSlide 14Octorber 14th, 2010
Rays
A t L di d H d A id (ALHAT)
Parallel Processing
u onomous an ng an  azar  vo ance 
Divide the bundle into sub-bundles, one for 
each CPU core
All the rays in a single bundle
Parallel BundlesSingle Bundle
Independent tasks for 100% parallelizationNot easy to parallelize
Bounding Pyramids
FSSBSlide 15Octorber 14th, 2010
Rays
A t L di d H d A id (ALHAT)
Vector CPU Processing
u onomous an ng an  azar  vo ance 
• Modern CPUs are scalar processors
– Each instruction operates on one data item at a time
• Streaming SIMD Extensions
– Intel extend the x86 instruction set (SSE)
– One instruction can operate on
• 4 32-bit integers
• 4 32-bit floats  
• 2 64-bit floats
• 2 64-bit integers
• 8 16 bit integers -  
• 16 8-bit characters
– Ideal for Vector/Matrix math
FSSBSlide 16Octorber 14th, 2010
– Additional instructions that must be explicitly used
A t L di d H d A id (ALHAT)
Emulator Design
u onomous an ng an  azar  vo ance 
Ray Tracer
Ethernet
8-Way Parallel
Ray Frustums
Rays
Simulation Data
Quad-Tree
Terrain
DemMaker
FSSBSlide 17Octorber 14th, 2010
A t L di d H d A id (ALHAT)
Sensor Model
u onomous an ng an  azar  vo ance 
• Add Gaussian noise to each pixel in the image
– Signal/Noise Ratio 
– Based on POST2 sensor model or actual hardware 
characteristics
• Pre-calculate random dead pixels
• Use intensity value for pixel cut-out
• Convolve with Gaussian filter for crosstalk or 
bleeding between pixels
FSSBSlide 18Octorber 14th, 2010
A t L di d H d A id (ALHAT)
General-Purpose computation on 
Graphics Processing Units
u onomous an ng an  azar  vo ance 
• A modern GPU is bigger and has more 
computational power than the CPU
• Massively parallel, multi-core processor
– Hundreds of cores per processor
Each core is a vector processor–      
• Ideal for image processing
– Each pixel will execute the exact same program, in 
parallel
• Implemented
– Additive Gaussian Noise  
– Gaussian Convolution
– Pixel Cut-Out
FSSBSlide 19Octorber 14th, 2010
– Image Formatting
A t L di d H d A id (ALHAT)
Emulator Design
u onomous an ng an  azar  vo ance 
Ray Tracer Noise
Convolution
Ethernet
8-Way Parallel
Ray Frustums
Rays
Range/IntensityImage
Signal Processing
Simulation Data
Command/Control
Quad-Tree
Terrain
Camera Link
DemMaker
FSSBSlide 20Octorber 14th, 2010
A t L di d H d A id (ALHAT)
Application
u onomous an ng an  azar  vo ance 
• Original Problem:
– How do we develop, test, and evaluate the ALHAT system 
in a lab environment when we can’t use LIDAR in the lab?           
• Field Test
– All three LIDAR sensors were flown on a helicopter from 
NASA Dryden
– The Avionics Processing Box was also flown, collecting 
data for the GNC and TSAR components
– The Flash LIDAR was connected to the APB
– The first time the Flash LIDAR was connected to the 
Avionics Processing Box
– The first field test for the LIDAR to integrate image
FSSBSlide 21Octorber 14th, 2010
          
processing and active, intelligent camera control
A t L di d H d A id (ALHAT)
Application
u onomous an ng an  azar  vo ance 
Problem: Although an Interface Documents (ICD) exists, the 
Flash LIDAR interface has never been implemented.  How 
can we develop and test the interface for a camera when the 
h ’t b b ilt t?camera asn  een u  ye
Solution: Use the emulator as the testbed to develop the 
interface
– The ICD and interface needed to be modified for FT4
• Image header
• Image resolution
S t ti i• ys em m ng
• Command/control
– The interface was first implemented in the emulator
– The APB was designed, developed, and tested using the emulator 
i fnter ace
– The Flash LIDAR system used the same interface code as the 
emulator
– The emulator was the first to implement the interface, and all other 
FSSBSlide 22Octorber 14th, 2010
implementations were based on it, so the emulator became the de 
facto interface standard
A t L di d H d A id (ALHAT)
Application
u onomous an ng an  azar  vo ance 
Problem: JSC needs a Flash LIDAR to develop 
their avionics software.  Sending a Flash LIDAR 
( d t t it) t JSC ld tan  person o opera e  o  wou  cos  a 
great deal of time, money, and inconvenience
Solution: Send an emulator to JSC for their use 
in software development
– Since JSC didn’t have to wait for the LIDAR to be 
finished and delivered, The APB and the LIDAR could 
be developed in parallel
– The emulator does not require an operator to be with 
it, so no personnel were required to go to JSC
– The emulator can be quickly modified for future field 
FSSBSlide 23Octorber 14th, 2010
tests with very little cost or effort
A t L di d H d A id (ALHAT)
Application
u onomous an ng an  azar  vo ance 
Problem: It is difficult to develop, test, and debug the 
image processing and active camera control 
software using the LIDAR camera    
Solution: Use the emulator as the data source for the 
system software
– The emulator provides a controlled input with known, well-
defined values
– The software and LIDAR camera could be developed in 
parallel
– The emulator can produce data files that can be used to           
help model and simulate the FPGA code for image 
processing
– Based on the emulator using the proprietary Ethernet 
FSSBSlide 24Octorber 14th, 2010
interface, not the ALHAT ICD.
A t L di d H d A id (ALHAT)
Results
u onomous an ng an  azar  vo ance 
FSSBSlide 25Octorber 14th, 2010
A t L di d H d A id (ALHAT)
Results
u onomous an ng an  azar  vo ance 
• An emulator was delivered to JSC in August 2009
• JSC used the emulator to develop their APB 
interface software in preparation for the flight test
• There were no significant interface issues in the 
fli ht t t d it th APB d th LIDARg  es , esp e e  an  e  never 
being physically connected until the field test
• An emulator was used extensively in the       
development process at Langley.  All initial FPGA 
code was developed and tested using the emulator 
first
• An emulator was sent to the test site and used for 
debugging and integration leading up to the test
FSSBSlide 26Octorber 14th, 2010
       
A t L di d H d A id (ALHAT)
Future Work
u onomous an ng an  azar  vo ance 
• Integrate the image processing algorithms into 
the emulator
• Develop an emulator for the velocimeter and 
altimeter LIDAR systems
• Revise for use in future field tests
FSSBSlide 27Octorber 14th, 2010


Flash LIDAR Emulator for HIL Simulation

Abstract

Similar works

Full text

Available Versions

NASA Technical Reports Server