Bitsch Link, Jo Agila

Fritsch, Dieter

Kleusberg, Alfred

Peter, Michael

Schäfer, Bernhardt

Wehrle, Klaus

English

We propose a system enabling users to capture the data needed to construct maps of building interiors, complementedby a smartphone-based pedestrian navigation application utilizing these maps.For this purpose, the user captures photographs of evacuation plans which are processed by the system using a suitable image processing pipeline. As a result, a coarse floor map is available, consisting of polygonal room outlines and identified staircases, where possible. Furthermore, the system derives the user’s initial position (which is marked in the plan) and orientation from the photographed plan.The coarse model can be further refined by geometric or semantic data geo-referenced by a pedestrian positioning solution. In a prototype configuration, two foot-mounted IMUs are used to process position and orientation with an extended Kalman Filter, using the such derived positions of features as door openings, stairs and elevators. In our system, the user downloads these traces from a repository.We make the final floor map available to the user. The coarse area-based model is transformed into walkable paths, annotated with additional semantic annotations, as available on the evacuation plan.Following this, smartphone users can easily use these maps for indoor navigation using only locally available information, without the need for a the building owner to make the maps available to the general public.Our proposed system therefore allows the easy creation and refinement of indoor maps directly applicable for navigation

University of Twente Research Information

2012 INTERNATIONAL CONFERENCE ON INDOOR POSITIONING AND INDOOR NAVIGATION, 13-15TH NOVEMBER 2012Versatile Geo-referenced Mapsfor Indoor Navigation of PedestriansMichael Peter, Dieter FritschInstitute for PhotogrammetryUniversity of StuttgartStuttgart, GermanyMichael.Peter/Dieter.Fritsch@ifp.uni-stuttgart.deBernhardt Schäfer, Alfred KleusbergInstitute of NavigationUniversity of StuttgartStuttgart, GermanySchaefer/Kleusberg@nav.uni-stuttgart.deJó Ágila Bitsch Link, Klaus WehrleCommunication and Distributed SystemsRWTH Aachen UniversityAachen GermanyJo.Bitsch/Klaus.Wehrle@rwth-aachen.deAbstract—We propose a system enabling users to capture thedata needed to construct maps of building interiors, comple-mented by a smartphone-based pedestrian navigation applicationutilizing these maps.For this purpose, the user captures photographs of evacuationplans which are processed by the system using a suitable imageprocessing pipeline. As a result, a coarse floor map is available,consisting of polygonal room outlines and identified staircases,where possible. Furthermore, the system derives the user’s initialposition (which is marked in the plan) and orientation from thephotographed plan.The coarse model can be further refined by geometric orsemantic data geo-referenced by a pedestrian positioning solution.In a prototype configuration, two foot-mounted IMUs are used toprocess position and orientation with an extended Kalman Filter,using the such derived positions of features as door openings,stairs and elevators. In our system, the user downloads thesetraces from a repository.We make the final floor map available to the user. Thecoarse area-based model is transformed into walkable paths,annotated with additional semantic annotations, as available onthe evacuation plan.Following this, smartphone users can easily use these mapsfor indoor navigation using only locally available information,without the need for a the building owner to make the mapsavailable to the general public.Our proposed system therefore allows the easy creation andrefinement of indoor maps directly applicable for navigation.Keywords—foot-mounted IMU; indoor navigation; floor maps;map refinementI. INTRODUCTIONPedestrian navigation in indoor environments has receivedincreasing interest in recent years. However, in comparisonto outdoor environments, indoor navigation is complicatedby multiple factors: Firstly, location based services and ap-plications in indoor environments demand for much higheraccuracies than typical outdoor applications. Secondly, globalpositioning systems are unavailable or do not provide thenecessary accuracy. These restrictions result in either the ne-cessity for expensive infrastructure or the use of less accurateinfrastructure independent positioning approaches. Further-more, map matching as used to support outdoor positioningis hindered by the fact that pedestrian movement is generallyless constraint than vehicle movement. Finally, the availabilityof indoor models is heavily dependent on the owner of therespective buildings.Therefore, we propose a system for the reconstructionof indoor models from photographed evacuation plans. Thegenerated maps can be further refined using data tracesrecorded with an IMU system. The result is directly usablefor smartphone-based infrastructure-less positioning.The paper consists of three parts: Section II describes thedata collector’s derivation of a coarse indoor model by ana-lyzing the photographed evacuation plan. With initial valuesfrom this analysis, the MEMS IMU positioning is initialized.Additional measurements for the positioning are obtained fromthe map and activity analysis, which is outlined in sectionIII. The end user scenario is presented in section IV, wherea map is created on the fly using the evacuation plan andcorrected using the high precision IMU data, which canbe immediately used for pedestrian indoor navigation in asmartphone application.ContributionsThe main contributions of our work are:• Creating navigable maps from evacuation plans onthe fly: We derive a coarse indoor model by analysis ofphotographed evacuation plans and set initial values forMEMS IMU positioning.• IMU based map refinement: We propose to furtherrefine automatically the created indoor model. An ex-periment with two foot-mounted MEMS IMUs was con-ducted to improve the positioning accuracy and to inves-tigate additional benefits from dual the IMU approach.• Infrastucture independent smartphone navigation:The end user can directly use the refined maps for smart-phone based indoor navigation at no additional overheadfor the building owner.II. ANALYSIS OF PHOTOGRAPHED EVACUATION PLANSThe general feasibility of indoor modeling from pho-tographed evacuation plans in a single well-known layout hasalready been shown in [1]. Here, we will present the main idea978-1-4673-1954-6/12$31.00 c© 2012 IEEE2012 INTERNATIONAL CONFERENCE ON INDOOR POSITIONING AND INDOOR NAVIGATION, 13-15TH NOVEMBER 2012Fig. 1. From left to right: original image; white balanced and Wallis filteredimage; wall structures (white) and symbol areas (orange); edgesof the approach and improvements enabling its generalizationto arbitrary plans.A. Reconstruction of Coarse Indoor ModelsFirstly, we enhance the input image correcting the colorbalance and brightness differences.Secondly, we divide the image into foreground and back-ground using adaptive thresholding with a big block size. Theresulting binary image contains both the sought-after groundplan and evacuation symbols which have to be identified andremoved. Even though most currently available plans do notfollow the ISO standard [2] completely, colors are generallyused to distinguish symbols from other plan elements. We usethis fact and the Color Structure Code [3] combined withthresholds for the colors to be expected to identify symbolareas (see figure 1).In order to reconstruct the wall structures, contours areextracted from the resulting cleaned binary image. Thesecontours can be simplified and closed whenever a symbol areais hit. The outer boundary of all edges will then be matchedto an available model of the building’s outer shell, deliveringthe transformation parameters from image coordinates to realworld coordinates.During the following reconstruction of the facets of the 2Dmodel, apart from rooms also stairs may be detected by theiraspect ratio and the length of their shortest edge. The repeatedoccurrence of neighboring stairs will lead to the identificationof staircases. The number of detected stairs together with acommon stair height results in an approximate room heightwhich can be used to extrude the edges of the 2D model towalls in the final 3D model and explicitly reconstruct stairs.B. Initial Values for MEMS IMU PositioningApart from the possibility to reconstruct a coarse indoormodel, photographed evacuation plans may furthermore beused to derive the initial values needed for the foot-mountedMEMS IMU positioning approach. These initial values are theposition of the user photographing the plan and his orientation.The initial position is marked in the evacuation plan bya symbol which may be detected by template matching (ifthe symbol is known) or by its uniqueness (concerning color,shape or size) among the other symbols. These image coordi-nates can then be transformed to world coordinates using thetransformation parameters found as stated in section II-A.Fig. 2. Trajectories (right = red, left = green) of two foot-mounted IMUs onlyusing ZUPTs. Initial heading and position from from the emergency map.Fig. 3. Trajectories of two foot-mounted IMUs. Coordinates and headingfrom the emergency map are used as measurement for the EKF.The initial orientation is a combination of the viewingdirection inside the plan and the angle between the user’s lineof sight and the evacuation plan. As stated in [2], the planshould be oriented according to the direction of view of theuser in front of it, resulting in the ”up” direction in imagecoordinates. To compute the angle between the user’s line ofsight and the evacuation plan, we use the approach presentedby Zhang & He [4].III. POSITIONING AND ORIENTATION WITH TWOFOOT-MOUNTED IMUSIn the prototype configuration, two foot-mounted IMUs areused to obtain positions and orientation individually for bothfeet using a strapdown calculation and a 15 element errorstate EKF. In contrast to [5], [6], no magnetometer sensors areused additionally. The initial positions and the heading fromsection II-B are used together with roll and pitch values froma static alignment [7] for the initialization of the strapdowncalculation.With activity recognition algorithms, the stance phases andthe type of locomotion (e. g. walking, running, upstairs anddownstairs) are detectable using the inertial observations [8].Stance phases are used to feed the EKF with zero velocity up-dates (ZUPTs) [5], [6]. With only the (pseudo)measurementsof the velocity, the complete error state vector is not ob-servable, therefore additional measurements are needed: Somefeatures from the model reconstructed before (section II-A)can be used as coordinate updates (CUPTs), e.g. elevators,beginning and ending of stairs, because they can also beidentified by the activity recognition.Due to gyroscope errors and the absence of a (magnetome-ter) heading measurement, the trajectories are without a stableheading, still they offer stride lengths which are accurate to afew centimeters [8]. A heading measurement can be formedusing the stride length differences from the right and the lefttrajectory [9]. Also the geometric constrains of the footfallsfrom both feet can be implemented and used.2012 INTERNATIONAL CONFERENCE ON INDOOR POSITIONING AND INDOOR NAVIGATION, 13-15TH NOVEMBER 201211111112.52.52.522441,5111,5,532211111112.52.52.522441,5111,5,5322242412S 43S6S711F73S8S4 61232'453'Fig. 4. Flow of information during navigation using FootPath [10]. (1) Theapplication obtains map material, e.g., from the photograph of an evacuationplan. (2) Using the inbuilt accelerometer, a mobile phone detects steps.(3) The detected steps are combined with heading information using theinbuilt magnetometer. Alternatively, (2’) the smartphone camera records andcompresses a video of the floor. Our system reads out the motion vectors.(3’) The detected motion vectors are transformed into virtual steps togetherwith heading information from the magnetometer. (4) The detected steps aremapped onto a map. (5) The base map provides the available orientations andcandidate paths. (6) The user gets feedback about the current most probableposition, based on the detected scores.An experiment was performed using two IMUs, one at eachfeet, walking a short path at an university building. The startingposition was equal to the ending and was manually marked atthe map with ’A’, see figures 2 and 3.The result of the strapdown calculation with EKF canbe extended by various additions, as the results will showone of them exemplary. The trajectories are placed on theafore mentioned map with an initial position and heading,performing a two-dimensional similarity transformation, butwithout any additional heading measurement, (see figure 2).As expected, the trajectories heading accuracy worsen veryfast.Figure 3 shows the trajectories using initial heading andposition from the emergency plan and position and headingof the beginning and ending of the stairs. The coordinatesare derived by a simulated feature extraction of the map. Thecorresponding stance phases are extracted from the initial rawdata using the activity recognition. Now this information isused as a measurement input for the EKF for the epochs ofthe stance phases. The trajectories fit better and the headingaccuracy of the right IMU (red trajectory) has improvedsignificantly. After some position and heading updates at theleft stair, both trajectories suffer from the inaccurate headingagain, which will be enhanced by the implementation of thegeometric constrains of the footfalls.IV. USING MAPS FOR SMARTPHONE NAVIGATIONAs we showed in our previous work [11], smartphones arewell equipped to serve as navigational aids to users not onlyin outside environments, but also inside buildings. However,the general availability of indoor maps is still problematic:Many building owners do not want to make exact buildingplans available to the general public, but only to authorizedpeople or visitors. In addition, they may not have or want tospend the resources to create or provide electronic versions ofthe building maps.To address these limitations our approach is to make thecreation of navigable maps as fast and easy as possible, whileat the same time not interfering with the rights of the buildingowners.We envision the process in the following way:1) A user enters a building and takes a picture of anevacuation plan. Either using on-board resources orusing a webservice, the photograph is transformed intoa basic coarse grained map.2) Our system downloads high accuracy tracks, createdusing an IMU system recorded previously inside thebuilding. This data is used to correct inaccuracies andincomplete parts in the evacuation plan based mapcreation.3) We immediately make the derived map available to theuser for navigation.Keep in mind, that we do not need to share the createdmap with other users or the general public. Through thismechanism, we do not infringe on the buildings owners rights,while at the same time providing the user with a better indoornavigation experience.The navigation process follows our previous work on Foot-Path [10], [11] as shown in Figure 4. After obtaining mapmaterial either from a public source or via a photographedevacuation plan, we present the user with a selection ofdestinations, to which she can navigate. The smartphone thendetects the steps of the user using the inbuilt sensors. If this isnot possible, we use the rear-facing camera instead to measurethe optical flow and derive pseudo-steps. On the basis ofthis speed and heading detection, we map the resulting stepsunto the available map. As we show in [10], this system isaccurate enough to provide the user with accurate turn by turnnavigation instructions.V. DISCUSSIONIn this work, we addressed the problem of missing indoormaps and how to solve it, especially when the building owneris not willing to release building plans to the general public.However, there are also other possibilities, especially in thecontext of public buildings or hospitals:• Specially prepared Wifi-Access Points at the buildingentrance. These access points might have a specific SSID,identifying them as a local indoor map provider.• QR-Codes or RFID Tags with links to a usable buildingplan, which are locally accessible. As these might includepassphrases, only people actually visiting the buildinghave access to building plan.• RFID Tags, that contain the complete map necessaryto find a specific location from the current location. Atan information poster, there might be tags embedded inbehind the name, including the complete path informationto a destination.But all this possibilities are limited by the usage of additionalinfrastructure, which is not available everywhere. Using datawhich is already available and visible for the user has a clear2012 INTERNATIONAL CONFERENCE ON INDOOR POSITIONING AND INDOOR NAVIGATION, 13-15TH NOVEMBER 2012Fig. 5. Screenshot during navigation of our FootPath application.advantage. When the map is available on the mobile device,the next question to solve is the pedestrian navigation. Withthe dual IMU approach, we try to use as much information aspossible from the human walking pattern. Using FootPath, wecan then make use of this information for efficient smartphone-based indoor navigation. We expect the different approachesto complement each other well.VI. CONCLUSIONS AND OUTLOOKWe presented a work-in-progress system for deriving andusing geo-referenced maps for indoor navigation of pedestri-ans. The extracted features of photographed evacuation mapsare used for indoor navigation with foot-mounted IMUs. Onthe other hand, the trajectories are used to specify walkableareas in the maps. Finally, geo-referenced maps for indoornavigation can be created with the shared IMU trajectoriesvia a smartphone application.The algorithms of the dual IMU trajectory calculationshould also be implemented for online calculation on a mo-bile platform. With possible positions updates from the mapextraction and activity recognition, a backward filtering of thetrajectories could improve their accuracies.ACKNOWLEDGMENTThis research was funded in part by the Federal State ofBaden-Württemberg, Germany and the DFG Cluster of Excel-lence on Ultra-high Speed Information and Communication(UMIC), German Research Foundation grant DFG EXC 89.REFERENCES[1] M. Peter, N. Haala, and D. Fritsch, “Using photographed evacuationplans to support MEMS IMU navigation,” in Proceedings of the 2011International Conference on Indoor Positioning and Indoor Navigation(IPIN2011), Guimarães, Portugal, 2011.[2] “ISO 23601: Safety identification - escape and evacuation plansigns,” 2009. [Online]. Available: http://www.iso.org/iso/iso catalogue/catalogue tc/catalogue detail.htm?csnumber=41685[3] L. Priese and P. Sturm, “Introduction to the color structurecode and its implementation,” 2003. [Online]. Available: http://www.uni-koblenz.de/∼lb/lb downloads/download/csc.pdf[4] Z. Zhang and L.-W. He, “Whiteboard scanning and image enhancement,”Digital Signal Processing, vol. 17, no. 2, pp. 414–432, Mar. 2007.[5] E. Foxlin, “Pedestrian tracking with shoe-mounted inertial sensors,”Computer Graphics and Applications, IEEE, vol. 25, no. 6, pp. 38–46,2005.[6] A. R. Jiménez, F. Seco, J. C. Prieto, and J. Guevara, “Indoor PedestrianNavigation using an INS/EKF framework for Yaw Drift Reduction anda Foot-mounted IMU,” in WPNC 2010: 7th Workshop on Positioning,Navigation and Communication 2010., 2010.[7] P. Aggarwal, Z. Syed, A. Noureldin, and N. El-Sheimy, MEMS-basedintegrated navigation, ser. GNSS technology and applications series,P. Aggarwal, Ed. Boston: Artech House, 2010.[8] B. Schäfer, F. Wild-Pfeiffer, W. Xue, C. Becker, and U. Lindemann,“Applications of body-mounted MEMS inertial sensors,” Proceedingsof the 2011 IAG General Assembly, Melbourne, Australia, 2011, vol.13x, 2012, under review.[9] B. Schäfer, “Determination of heading information from gait cyclepattern using stride length estimation with reduced IMUs on right andleft foot,” in Proceedings of the 2011 International Conference on IndoorPositioning and Indoor Navigation (IPIN2011), Guimarães, Portugal,2011.[10] J. A. Bitsch Link, F. Gerdsmeier, P. Smith, and K. Wehrle, “IndoorNavigation on Wheels (and on Foot) using Smartphones,” 2012, underreview.[11] J. A. Bitsch Link, P. Smith, N. Viol, and K. Wehrle, “FootPath: AccurateMap-based Indoor Navigation Using Smartphones,” in Proceedings ofthe 2011 International Conference on Indoor Positioning and IndoorNavigation (IPIN2011), Guimarães, Portugal, 2011.

Versatile geo-referenced maps for indoor navigation of pedestrians

https://ris.utwente.nl/ws/files/30050792/Peter2017versatile.pdf

Versatile geo-referenced maps for indoor navigation of pedestrians

Abstract

Similar works

Full text

Available Versions

University of Twente Research Information