Robust monitoring of bridges is necessary to ensure its serviceability and traffic safety. A reliable sensing system to measure bridge responses is the key to such monitoring approaches. This research discusses the resilience of vision-based monitoring (VBM) for accurate and reliable bridge monitoring. VBM deploys cameras to capture the movement of bridges and uses suitable image processing algorithms to derive information on bridge health. VBM as a low cost and user-friendly monitoring system for accurate response measurement, simultaneous multiple target tracking and hardware adaptability is assessed based on literature. Measures to make VBM resilient in adverse field conditions are also discussed. Overall findings emphasise the accuracy and reliability of VBM for holistic and cost-efficient monitoring of bridges

Al-Habaibeh, A

Borah, S

Kaveh, B

Kromanis, R

Nottingham Trent Institutional Repository (IRep)

Borah, et al.  The Resilience of Vision-based Technology for Bridge Monitoring  Measuring and analysing resilience of interdependent STE systems Resilient sensing systems for infrastructure monitoring  Borah, S. et al., (Eds). Proceedings of the Joint International Resilience Conference 2020  Interconnected: Resilience innovations for Sustainable Development Goals 4TU Centre for Resilience Engineering / Future Resilience Systems - SEC 1   The Resilience of Vision-based Technology for Bridge Monitoring  Sushmita Boraha, Amin Al-habaibehb, Rolands Kromanisc, Bahareh Kavehd a Nottingham Trent University, sushmita.borah@ntu.ac.uk  b Nottingham Trent University, amin.al-habaibeh@ntu.ac.uk  c University of Twente, r.kromanis@utwente.nl d Nottingham Trent University, bahareh.kaveh@ntu.ac.uk    Summary Robust monitoring of bridges is necessary to ensure its serviceability and traffic safety. A reliable sensing system to measure bridge responses is the key to such monitoring approaches. This research discusses the resilience of vision-based monitoring (VBM) for accurate and reliable bridge monitoring. VBM deploys cameras to capture the movement of bridges and uses suitable image processing algorithms to derive information on bridge health. VBM as a low cost and user-friendly monitoring system for accurate response measurement, simultaneous multiple target tracking and hardware adaptability is assessed based on literature. Measures to make VBM resilient in adverse field conditions are also discussed. Overall findings emphasise the accuracy and reliability of VBM for holistic and cost-efficient monitoring of bridges. Keywords Bridge monitoring; Computer vision; Structural health monitoring (SHM); Resilient sensing system; Vision-based monitoring (VBM). 1. Introduction Bridges are important assets of civil infrastructure that enables mobility and has both economic and social significance. However, most of the existing bridges are ageing. For example, the majority of the currently operating bridges in Europe were built in the 1950s and are reaching the end of their design life (Gkoumas et al., 2019). With continuously increasing traffic loads, it is becoming increasingly important to monitor and maintain these ageing infrastructures. Recent tragic incidents like the Genoa Bridge Collapse (O’Reilly et al., 2018) further highlights the need for efficient bridge monitoring. Traditionally brides are monitored periodically through a visual survey of its accessible parts. Such visual inspections are subjective and susceptible to human error.  Structural health monitoring (SHM) systems are more frequently introduced in bridge condition assessment. Robust and continuous measurement of bridge responses such as displacement, strain, etc. under applied loads (e.g., traffic load, temperature and wind) provide information on bridge performance. A general bridge SHM framework is illustrated in Figure 1. Bridge responses are measured with contact sensors such as a linear variable differential transducer, Fibre Bragg Grating sensors, etc. or non-contact sensors such as cameras. Response time-histories are analysed for anomaly such as damage in the bridge. Thereafter, bridge management authorities intervene in maintenance work, if required, based on the outcome of analysed data. This research discusses the resilience of vision-based monitoring for accurate and reliable bridge monitoring. VBM is a non-contact, non-destructive system that measures structural responses using advances in camera technology and computer vision. These systems pose competitive advantages over contact sensors. For example, cameras can be found at low-cost; VBM is non-destructive, offers measurement collection of multiple targets, and has simple instrumentation and installation (Xu and Brownjohn, 2018). This abstract aims to analyse the performance of VBM for bridge response measurement in terms of accuracy, distributed monitoring, hardware adaptability and identifies means to make VBM resilient by addressing its limitations based on literature.   Borah, et al.  The Resilience of Vision-based Technology for Bridge Monitoring  Measuring and analysing resilience of interdependent STE systems Resilient sensing systems for infrastructure monitoring  Borah, S. et al., (Eds). Proceedings of the Joint International Resilience Conference 2020  Interconnected: Resilience innovations for Sustainable Development Goals 4TU Centre for Resilience Engineering / Future Resilience Systems - SEC 2   Figure 1: A general framework of Bridge health monitoring (Borah et al., 2020) 2. Vision-based monitoring of bridges Vision-based monitoring (VBM) systems consist of hardware components such as cameras, lens, tripods, etc for image collection and image processing algorithms for measurement of structural responses. These are non-contact sensors and measure structural responses such as deflections and strain which are analysed for structural performance. Figure 2 shows a schematic arrangement of VBM used for monitoring the mid-span structural response of a truss bridge. VBM measures displacements of targets (structural features) under loading using suitable image processing algorithms (Feng and Feng, 2018). The targets can be natural targets such as bolts and nuts or artificial targets attached to the bridge. In Figure 2, sequential image frames, In (n = 0,1,2,3,…,n) of the mid-span joint are captured corresponding to time tn (n = 0,1,2,3,…,n). t0 represents the time of no loading. Position of targets in the sequential images is tracked over time to generate its displacement-time histories. VBM commonly uses the concept of digital image correlation to track the position of targets. Digital image correlation compares images of the structure in reference (un-deformed) and current (deformed) frames to provide full-field displacements and strains (Bing et al., 2009). The algorithms are a combination of suitable sub-pixel registration, template matching and camera calibration techniques. Template matching is the principal component of the algorithm. Commonly used template matching approaches are area-based matching, feature-based matching, optical flow estimation and shape-based matching. Structural responses such as strains and natural frequency are then derived from the measured target displacements (Brownjohn et al., 2017).   Figure 2: Schematic of a vision-based monitoring system Borah, et al.  The Resilience of Vision-based Technology for Bridge Monitoring  Measuring and analysing resilience of interdependent STE systems Resilient sensing systems for infrastructure monitoring  Borah, S. et al., (Eds). Proceedings of the Joint International Resilience Conference 2020  Interconnected: Resilience innovations for Sustainable Development Goals 4TU Centre for Resilience Engineering / Future Resilience Systems - SEC 3  3. Resiliency of VBM A resilient monitoring system provides accurate and reliable structural safety information with simpler instrumentation, minimal health and safety hazard and at optimal cost. It should also be able to sustain errors induced by environmental factors and mechanical malfunction in long-term monitoring. This section discussed VBM as a resilient monitoring system for bridge monitoring.   3.1. Accuracy The measurement accuracy of vision-based technology is dependent on parameters such as camera-to-target distance, target pattern features, lighting conditions, camera mounting stability and video-processing methods (Xu and Brownjohn, 2018). Khuc et al. (2017) suggested a measurement accuracy of 0.04 mm based on a vision-based laboratory experiment in a short-range distance (<14 m). A sub-pixel accuracy of 0.5 to 0.01 pixel had been reported in algorithms used for vision-based monitoring (Bing et al., 2006). VBM system has shown promising evidence in providing reliable information on bridge performance such as its deflection profile, load carrying capacity, damage detection, vibration serviceability assessment, etc. (Dong and Catbas, 2020).  3.2. Distributed monitoring Most contact sensor-based SHM systems require numerous sensors for distributed monitoring of bridges. However, multiple targets in the bridge can be tracked using a single camera or a combination of a few cameras in VBM. Lydon et al. (2018) developed a vision-based monitoring system using two modified GoPro cameras for distributed displacement measurement in Verner bridge, Northern Ireland. Kromanis et al. (2019a) created super-resolution images of the entire laboratory beam by stitching images captured by a robotic camera which can provide high-resolution data points for distributed monitoring of the structure. Xu et al. (2018) achieved measurement accuracy of ±0.037 mm for camera-to-target distance of 5.70 m using a GoPro camera fitted with a wide-angle lens to measure the displacement of several targets in a laboratory beam. Thus, VBM can enable holistic monitoring of bridges at a low overall cost and with simpler instrumentation.  3.3. Hardware adaptability VBM system is adaptable with multiple camera types thereby facilitating a flexible and cost-effective monitoring system as per user’s requirement. Lydon et al (2018) achieved high accuracy in measuring the displacement of a laboratory beam using a Nikon D180 (approximate cost £2500) as well as a modified GoPro (approximate cost £500) camera. Unmanned aerial vehicles are emerging hardware used in VBM to collect visual information of damages in structures (Perry et al., 2020). Moreover, promising accuracy of response measurement has been recorded by researchers using readily available advances in the in-built camera of smartphones (Kromanis et al., 2019b; Orak and Ozturk, 2018). 3.4. Making VBM resilient VBM has easy instrumentation that can enable accurate distributed monitoring of structures at a low cost. The resiliency of VBM can be further improved by mitigating its common errors sources such as camera-movement, environmental factors, etc. Camera unsteadiness can induce an error in measurement, especially in long term field applications due to motion induced by wind, traffic vibration, etc. Camera-motion induced errors are usually compensated in VBM by using stationary objects in the background as a reference. Lee et al. (2020) proposed a dual-camera system where the main camera tracks the target and the sub-camera, tightly attached to the main camera, measured the 6-DOF motion of the main camera. Motion induced error calculated from the 6-DOF motion of the main camera compensated the error from 44.1 mm to 1.1 mm. The camera assembly is often sheltered in tents to avoid error due to wind in field implementations (Ribeiro et al., 2014). Environmental factors such as illumination level, fog, background disturbance can interfere with field implementation of VBM. Ye et al. (2016) encountered difficulty in pattern matching when the illumination level was below 75 lux and the vapour level was high. Such error due to bad weather can be enhanced by applying suitable image processing algorithms. Artificial lights are also used to illumination the targets (Ribeiro et al., 2014). Vasileva et al. (2018) successfully adapted VBM to sustain the extreme cold climate of Russia by incorporating an auto-heating unit and a drying cartridge in the camera unit.  Borah, et al.  The Resilience of Vision-based Technology for Bridge Monitoring  Measuring and analysing resilience of interdependent STE systems Resilient sensing systems for infrastructure monitoring  Borah, S. et al., (Eds). Proceedings of the Joint International Resilience Conference 2020  Interconnected: Resilience innovations for Sustainable Development Goals 4TU Centre for Resilience Engineering / Future Resilience Systems - SEC 4  4. Conclusions Frequent monitoring and timely maintenance are crucial to ensure the serviceability of bridges and avoid catastrophic failures such as Genoa Bridge Collapse. Cameras and computer vision technology comprises the vision-based monitoring (VBM) system to provide accurate non-contact monitoring of bridges. This abstract evaluates the performance of VBM as a resilient monitoring system based on literature. Following conclusions can be drawn: (a) Measurement accuracy of about 0.04 mm or sub-pixel accuracy of 0.5 to 0.01 pixel has been reported in the literature. (b) The required accuracy of VBM has been achieved for simultaneous monitoring of multiple targets using one or a combination of a few cameras. (c) The monitoring system is adaptable with a range of low-cost and expensive cameras, thereby enabling a flexible monitoring system as per users’ need. (d) The resiliency of VBM can be improved by incorporating innovative measures to eliminate common errors sources such as camera-movement, environmental factors, etc. References Bing, P., Qian, K., Xie, H., Asundi, A., 2009, Two-dimensional digital image correlation for in-plane displacement and strain measurement: A review. Measurement Science and Technology, 20. Bing, P., Xie, H.M., Xu, B.Q., Dai, F.L., 2006, Performance of sub-pixel registration algorithms in digital image correlation. Measurement Science and Technology, 17, 1615–1621. Borah, S., Al-habaibeh, A., Kromanis, R., 2020, The Effect of Temperature Variation on Bridges - A literature Review. In: 2nd International Conference on Energy and Sustainable Futures (ICESF 2020). Hatfield, Hertfordshire, UK, pp. 1–5. Brownjohn, J.M.W., Xu, Y., Hester, D., 2017, Vision-based bridge deformation monitoring. Frontiers in Built Environment, 3, 23. Dong, C.Z., Catbas, F.N., 2020, A review of computer vision-based structural health monitoring at local and global levels. Structural Health Monitoring, In press. Feng, D., Feng, M.Q., 2018, Computer vision for SHM of civil infrastructure: From dynamic response measurement to damage detection – A review. Engineering Structures, 156, 105–117. Gkoumas, K., Marques Dos Santos, F.L., Van Balen, F.L., Tsakalidis, M., Ortega Hortelano, A., Grosso, M., Haq, G., Pekár, F., 2019, Research and innovation in bridge maintenance, inspection and monitoring. Publications Office of the European Union. Khuc, T., Catbas, F.N., 2017, Completely contactless structural health monitoring of real-life structures using cameras and computer vision. Structural Control and Health Monitoring, 24, 1–17. Kromanis, R., Forbes, C., Borah, S., 2019a, Super-resolution images for measuring structural response. In: SMAR 2019 - Fifth Conference on Smart Monitoring, Assessment and Rehabilitation of Civil Structures. Potsdam, Germany, pp. 1–8. Kromanis, R., Xu, Y., Lydon, D., Martinez del Rincon, J., Al-Habaibeh, A., 2019b, Measuring structural deformations in the laboratory environment using smartphones. Frontiers in Built Environment, 5. Lee, J., Lee, K.C., Jeong, S., Lee, Y.J., Sim, S.H., 2020, Long-term displacement measurement of full-scale bridges using camera ego-motion compensation. Mechanical Systems and Signal Processing, 140, 106651. Lydon, D., Lydon, M., Del Rincon, J.M., Taylor, S.E., Robinson, D., O’Brien, E., Catbas, F.N., 2018, Development and field testing of a time-synchronized system for multi-point displacement calculation using low-cost wireless vision-based sensors. IEEE Sensors Journal, 18, 9744–9754. O’Reilly, G.J., Monteiro, R., Scattarreggia, N., Martino Calvi, P., Pinho, R., Calvi, G.M., Malomo, D., Moratti, M., 2018, Once upon a Time in Italy: The Tale of the Morandi Bridge. Structural Engineering International, 0, 1–20. Borah, et al.  The Resilience of Vision-based Technology for Bridge Monitoring  Measuring and analysing resilience of interdependent STE systems Resilient sensing systems for infrastructure monitoring  Borah, S. et al., (Eds). Proceedings of the Joint International Resilience Conference 2020  Interconnected: Resilience innovations for Sustainable Development Goals 4TU Centre for Resilience Engineering / Future Resilience Systems - SEC 5  Orak, M.S., Ozturk, T., 2018, Monitoring cantilever beam with a vision-based algorithm and smartphone. Vibroengineering PROCEDIA, 17, 107–111. Perry, B.J., Guo, Y., Atadero, R., van de Lindt, J.W., 2020, Streamlined bridge inspection system utilizing unmanned aerial vehicles (UAVs) and machine learning. Measurement: Journal of the International Measurement Confederation, 164, 108048. Ribeiro, D., Calçada, R., Ferreira, J., Martins, T., 2014, Non-contact measurement of the dynamic displacement of railway bridges using an advanced video-based system. Engineering Structures, 75, 164–180. Vasileva, A. V., Vasilev, A.S., Konyakhin, I.A., 2018, Vision-based system for long-term remote monitoring of large civil engineering structures: Design, testing, evaluation. Measurement Science and Technology, 29. Xu, Y., Brownjohn, J., Kong, D., 2018, A non-contact vision-based system for multipoint displacement monitoring in a cable-stayed footbridge. Structural Control and Health Monitoring, 25, 1–23. Xu, Y., Brownjohn, J.M.W., 2018, Review of machine-vision based methodologies for displacement measurement in civil structures. Journal of Civil Structural Health Monitoring, 8, 91–110. Ye, X.W., Yi, T.H., Dong, C.Z., Liu, T., 2016, Vision-based structural displacement measurement: System performance evaluation and influence factor analysis. Measurement, 88, 372–384.   

The resilience of vision-based technology for bridge monitoring: measuring and analysing resilience of interdependent STE systems resilient sensing systems for infrastructure monitoring

http://irep.ntu.ac.uk/id/eprint/42273/1/1406002_Al-Habaibeh.pdf

The resilience of vision-based technology for bridge monitoring: measuring and analysing resilience of interdependent STE systems resilient sensing systems for infrastructure monitoring

Abstract

Similar works

Full text

Available Versions

Nottingham Trent Institutional Repository (IRep)