Standard network metrics, such as throughput, latency and reception probability, are the most popular performance indicators used in the literature to describe and compare communication protocol variations. However, these “traditional” network-centric PI are not adapted to the distributed, information-centric nature of the beaconing communication pattern, nor do they cover application level reliability or freshness of information.

In this paper, we introduce a more suitable metric called Update Delay, represented as a Complementary Cumulative Distribution Function (CCDF). We will show how this single Update Delay performance indicator can be an optimal representation of the freshness and reliability of the information about a certain transmitter, i.e. awareness about vehicles and their current state in the vicinity. This paper extends on the methodological aspects of the approach, as well as introduces several concrete examples

Härri, Jérôme

Kloiber, Bernhard

Rico-Garcia, Cristina

Strang, Thomas

English

Institute of Transport Research:Publications

19th ITS World Congress, Vienna, Austria, 22/26 October 2012 EU-00364 Update Delay: A new information-centric metric for a combined communication and application level reliability evaluation of CAM based Safety Applications   Bernhard Kloiber1*, Cristina Rico-Garcia1, Jérôme Härri2, Thomas Strang1 1. German Aerospace Center (DLR), Germany, Münchner Straße 20, 82234 Weßling, Tel.: +49 8153 28-3208, bernhard.kloiber@dlr.de 2. EURECOM, France   Abstract Although Intelligent Transport System (ITS) communications is based on the WLAN standard IEEE 802.11, the requirements of safety related ITS applications have been changed completely. Basically, they require up-to-date information about the status of other vehicles in the vicinity, which is currently implemented by each vehicle broadcasting periodically this kind of information. Consequently, the usual performance metrics like throughput, latency, reception probability, etc. are no longer suitable for analysis and evaluation of ITS applications. In this paper we first introduce Awareness Quality, which is the application level metric for CAM based safety applications. Then we define a new information-centric network level metric called Update Delay, which is highly correlated with the Awareness Quality. Finally we show how to obtain the Awareness Quality from a communications perspective by using Update Delay measurements, to evaluate the reliability of safety related applications.  Keywords: Update Delay, information-centric metric, ITS, application reliability   Introduction The Intelligent Transport System (ITS) communications in Europe is called ITS-G5 and is based on the common Wireless Local Area Network (WLAN) standard 802.11 [1]. Although the ITS communication technology kept almost the same, the applications and its Update Delay: A new information-centric metric for a combined communication and application level reliability evaluation of CAM based Safety Applications 2 requirements have changed completely, compared to common WLAN applications. Safety related ITS applications basically require information about the status of other vehicles in the surrounding, usually referred to as (cooperative) Awareness. In ITS networks, the cooperative awareness is mainly provided by each vehicle broadcasting periodically so called Cooperative Awareness Messages (CAMs). They contain information about their current position, speed, heading, etc. [2]. Being aware of surrounding vehicles, one is able for example, to estimate their trajectories and also the probability of an imminent collision, a functionality required by each CAM based collision avoidance application. Reliable trajectory estimation requires a highly up-to-date knowledge about neighbouring vehicles. The more frequently CAMs are received, the more up to date is the information about the appropriate vehicle. The fineness and up-to-dateness of that knowledge, we define as Awareness Quality, which is the application level metric for CAM based safety applications from an information theory point of view. Default network performance metrics, known from common WLAN applications, such as throughput, latency and reception probability, are no longer suitable to evaluate the reliability of CAM based safety applications, because they don’t consider the awareness quality. So, we lack a clear network performance metric, which is able to make a statement about the reliability of the safety application, regarding the awareness quality. Additionally, we lack an appropriate representation of the metric, too. Common performance analysis publications show averaged values of the used metric. Averaging means the loss of information, especially with respect to occasional but safety critical events.  In this paper, we introduce a new information-centric network metric called Update Delay, represented as a Complementary Cumulative Distribution Function (CCDF) [3, 4]. As the network metric Update Delay is closely linked with the application metric Awareness Quality, it allows an application level reliability evaluation from a communication perspective. That requires a detailed introduction of the Awareness Quality and a clear definition of the Update Delay metric. We further give a detailed description of the correlation between Update Delay and Awareness Quality according to CAM broadcasts, and we explain how to use Update Delay measurements for reliability evaluation of CAM based applications.    Related Work Up to now, several papers have been published, which analyze vehicular networks, communicating with ITS-G5 or 802.11p [5, 6, 7, 8, 9, 10, 11, 12, 13]. Most of them use the well-known default network metrics like throughput, latency, packet delivery/loss ratio, etc., but do not consider its applicability from an application perspective, for instance, evaluation of application reliability. ElBatt et al. additionally measured the packet inter-reception time, which basically Update Delay: A new information-centric metric for a combined communication and application level reliability evaluation of CAM based Safety Applications 3 corresponds to our introduced Update Delay metric, and plotted it against simulation time. In contrast to that representation, we further propose an appropriate statistical processing of the measured values, by presenting it as CCDF. The advantages of the CCDF representation will be shown later in this paper. Only a few publications have been found, which try to define Awareness in vehicular networks. Mittag et al. [14], for instance, introduced the Neighborhood Awareness, as the probability, that node i is aware of its neighboring nodes, while being aware means, having received at least one beacon message within the last second. Unfortunately, this definition is mainly focusing on network level but not application level, including information theoretic aspects. Schmidt et al. defined a binary awareness metric by forming the Quotient between the number of detected vehicles and the number of all vehicles within a certain distance and comparing it with a desired threshold value [15]. A disadvantage of this awareness definition is the binary codomain, which is not able to incorporate information theoretic aspects, like a continuously decreasing up-to-dateness of state information from other vehicles with time.   Application level metric: Awareness Quality The Awareness Quality metric is time as well as space dependent and typically varies from application to application. As the Awareness Quality describes the fineness and up-to-dateness of information, we compare its timely behaviour with the behaviour of an Information Fusion Filter. Their functionality can be described by a two-step approach. Without loss of generality, the two steps are as follows, assuming the vehicles’ state only contains a 1D position, as depicted in Figure 1 a): • Prediction Step: In absence of CAMs, e.g. due to collisions, the vehicles state is estimated by using a prediction model. Because each prediction brings additional uncertainty into the state estimation, prediction reduces the Awareness Quality. • Update Step: When a CAM is received, the state estimation is updated by using the contained status information. Assuming adequate accuracy of the received data, the Awareness Quality is increased with each update. Consequently, the most important influence on the timely behaviour of the Awareness Quality is the elapsed time between two consecutive CAMs from the same transmitter. The space dependency of the Awareness Quality for safety related applications is basically associated with the distance to the transmitting vehicle. Considering a CAM based collision avoidance application, the probability for a collision is the higher, the closer the distance between the vehicles. Likewise, the Awareness Quality of each other is required to be the higher, the closer the distance between the vehicles is. Figure 1 b) shows an example behaviour of the Awareness Quality with distance. Update Delay: A new information-centric metric for a combined communication and application level reliability evaluation of CAM based Safety Applications 4  Figure 1: Time and space dependent behavior of Awareness Quality.  Network level metric: Update Delay  Definition As default network metrics are not able to represent cooperative Awareness, we propose a new information-centric metric, the afore-mentioned Update Delay [3, 4]. Without loss of generality, we define the Update Delay as follows: The Update Delay UDi,j is defined as the elapsed time, while expected CAMs from vehicle j are not received by vehicle i. If time t passes between two consecutive CAM receptions of vehicle i from vehicle j, we write: UDi,j = t  Awareness Quality Correlation The link between the network level metric Update Delay and the application level metric Awareness Quality is indirect and depicted in Figure 2. It shows the correlation between Update Delay and Awareness Quality within the layers of the ITS communication stack (left). The Awareness Quality as an application level metric is only controlled indirectly by the Update Delay as a network level metric. On Network layer the Update Delay measures the time between two consecutive CAM receptions. The information contained in the CAM is then processed by the Facility layer. It is the way of processing the information, which directly influences the behaviour of the Awareness Quality, e.g. by using an Information Fusion Filter for state estimation. An application again only defines requirements on the Awareness Quality.  Update Delay: A new information-centric metric for a combined communication and application level reliability evaluation of CAM based Safety Applications 5  Figure 2: Correlation between Update Delay and Awareness Quality from a communications perspective. As a consequence, the reliability evaluation of CAM based safety applications is completely decoupled from the Update Delay evaluation on network level. But if the Information Fusion Filter on the Facility layer is known, the link between Update Delay and Awareness Quality is established. Without loss of generality, we assume an Information Fusion Filter for processing the information contained in received CAMs. Figure 2 shows exemplarily the dependency between Update Delay and Awareness Quality (right). It shows the time domain on the x-axis and the Awareness Quality on the y-axis. By having a closer look on the behaviour of the curve, the afore-mentioned prediction and update steps can be clearly identified. The time elapsed between two CAM receptions, corresponds to the Update Delay. The higher the Update Delay values, the more the Awareness Quality decreases, due to the increasing uncertainty by prediction. How much it decreases with time, heavily depends on the used prediction model and its accuracy in describing the real vehicles behaviour.  Representation As the Update Delay itself is similar to the packet inter arrival time metric in [7], we additionally propose an appropriate representation as Complementary Cumulative Distribution Function (CCDF). Examples of this representation are depicted in Figure 3.  To obtain the Update Delay CCDF, we first collect measured Update Delay values and build a histogram out of this data. The histogram can be easily transformed into a Cumulative Distribution Function (CDF). The CCDF is just 1 – CDF. The stairs behaviour of the Update Delay CCDF is because of the discrete update interval in steps of 0.5 s by using a 2 Hz CAM transmission rate.  The interpretation of the Update Delay CCDF chart is as follows: The x-axis shows certain Update Delay values in seconds, the y-axis depicts the probability for exceeding a certain Update Delay: A new information-centric metric for a combined communication and application level reliability evaluation of CAM based Safety Applications 6 Update Delay value UD p(ud > UD). An example will be described in the next subsection.   Figure 3: Example Update Delay CCDFs for different considered Awareness Ranges. As the Update Delay is a pure time based network metric, it can only represent the timely behaviour of the Awareness Quality. But in limiting the Update Delay evaluation and CCDF representation to various Awareness Ranges, we can easily add the distance dependency, too. Figure 3 shows the Update Delay CCDF for three different considered Awareness Ranges. That means, for each curve only vehicles located within the considered Awareness Range have been taken into account to measure the Update Delay.  Update Delay from an Application Perspective As a pure network level metric, the Update Delay is easy to measure, either in simulations or in real measurement campaigns. In analysing the Update Delay including its representation as CCDF, the reliability of CAM based safety applications can be evaluated, too. In doing so, the following steps are necessary: 1. Application Requirements: The safety application requirements on the Awareness Quality requirements have to be determined. Typically, it’s a minimum required Awareness Quality AQmin, if fallen below, the application is not working correctly any more.  2. Requirements Mapping: If the information processing method on Facility layer is known, the application requirement on the Awareness Quality has to be mapped onto the corresponding network requirement regarding the Update Delay. Typically, it is a maximum allowed Update Delay value UDmax, if exceeded, the Awareness Quality falls below AQmin because of missing updates. Update Delay: A new information-centric metric for a combined communication and application level reliability evaluation of CAM based Safety Applications 7 3. Update Delay CCDF look up: Knowing the maximum allowed Update Delay UDmax, the probability for exceeding UDmax can be looked up by using the Update Delay CCDFs. This exceedance probability is also the probability for falling below AQmin and likewise the probability for the safety application not working properly (with respect to its requirements). The following example should illustrate the procedure above more clearly: Consider a CAM based collision avoidance application, which requires a minimum Awareness Quality AQmin of 99 % within an Awareness Range of 50 m. Let’s assume, that the mapping by the Information Fusion Filter leads to a maximum allowed Update Delay UDmax of 1 s. To get the application failure probability, one can search for UDmax on the x-axis, by taking care of the appropriate curve, and look up its corresponding exceedance probability of 10-3 on the y-axis, as depicted in Figure 3. The other way round is also possible, i.e. starting with a minimum required application failure probability and looking for the appropriate UDmax, which again can be mapped to the achievable AQmin.   Conclusion This paper describes a methodology for evaluating the reliability of CAM based (safety) applications: First we introduced the application level metric Awareness Quality, which is the metric for CAM based (safety) applications. We further defined a new information-centric network level metric called Update Delay and its representation as Complementary Cumulative Distribution Function (CCDF). As a network level metric, the Update Delay is easy to measure, so we highlighted its correlation with the Awareness Quality, and how it can be used for evaluating the application reliability from a communications perspective.   Ackknowledgments EURECOM acknowledges the support of its industrial members: BMW Group, Cisco, Monaco Telecom, Orange, SAP, SFR, STEricsson, Swisscom, Symantec, Thales.   References 1. IEEE, “802.11-2012 - IEEE Standard for Information technology – Telecommunications and information exchange between systems Local and metropolitan area networks–Specific requirements Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY) Specifications,” IEEE Computer Society, IEEE Standard 802.11-2012, 2012. Update Delay: A new information-centric metric for a combined communication and application level reliability evaluation of CAM based Safety Applications 8 2. ETSI, “Intelligent Transport Systems (ITS); Vehicular Communications; Basic Set of Applications; Part 2: Specification of Cooperative Awareness Basic Service,” ETSI TC ITS, Technical Specification 102 637-2, March 2011, version 1.2.1. 3. C. R. Garcia, A. Lehner, P. Robertson, T. Strang, ”Performance of MAC protocols in beaconing Mobile Ad-hoc Multibroadcast Networks”, 3rd International Workshop on Multiple Access Communications (MACOM), September 2010. 4. B. Kloiber, T. Strang, F. de Ponte-Müller, C. R. Garcia, M. Röckl, ”An Approach for Performance Analysis of ETSI ITS-G5A MAC for Safety Applications”, ITST, November 2010. 5. Stephan Eichler, ”Performance Evaluation of the IEEE 802.11p WAVE Communication Standard”, WiVeC, September 2007. 6. Jijun Yin, Tamer ElBatt, Gavin Yeung, Bo Ryu, Stephen Habermas, Hariharan Krishnan and Timothy Talty, ”Performance Evaluation of Safety Applications over DSRC Vehicular Ad Hoc Networks”, VANET’04, ACM, October 2004. 7. T. ElBatt, S. K. Goel, G. Holland, H. Krishnan, J. Parikh, ”Cooperative Collision Warning Using Dedicated Short Range Wireless Communications”, VANET’06, ACM, September 2006. 8. K. Bilstrup, E. Uhlemann, E. G. Ström, and U. Bilstrup, “On the ability of the 802.11p mac method and stdma to support realtime vehicle-to-vehicle communication,” EURASIP J. Wirel. Commun. Netw., vol. 2009, pp. 5:1–5:13, Jan. 2009. 9. Achim Brakemeier, ”White Paper on Network Design Limits and VANET Performance”, Version 0.5, Car2Car Communication Consortium, November 2008. 10. R. K. Schmidt, T. Kollmer, T. Leinmüller, B. Böddeker and G. Schäfer, ”Degradation of Transmission Range in VANETs caused by Interference”, PIK - Praxis der Informationsverarbeitung und Kommunikation (Special Issue on Mobile Ad-hoc Networks), 2009. 11. F. Schmidt-Eisenlohr, M. Torrent-Moreno, J. Mittag and H. Hartenstein, ”Simulation Platform for Inter-Vehicle Communications and Analysis of Periodic Information Exchange”, IEEE/IFIP WONS, January 2007. 12. F. Bai, H. Krishnan, ”Reliability Analysis of DSRC Wireless Communication for Vehicular Safety Applications”, ITSC 2006, September 2006. 13. Marc Torrent-Moreno, ”Inter-Vehicle Communications: Assessing Information Dissemination under Safety Constraints”, IEEE/IFIP WONS, January 2007. Update Delay: A new information-centric metric for a combined communication and application level reliability evaluation of CAM based Safety Applications 9 14. J. Mittag, F. Thomas, J. Härri, H. Hartenstein, “A Comparison of Single- and Multi-hop Beaconing in VANETs”, VANET ’09, ACM, Beijing, China, 2009. 15. R. K. Schmidt, R. Lasowski, T. Leinmüller, C. Linnhoff-Popien, G. Schäfer, “An Approach for Selective Beacon Forwarding to Improve Cooperative Awareness”, 2nd IEEE Vehicular Networking Conference (VNC 2010), New Jersey, USA, 2010. 

An Approach for Performance Analysis of ETSI ITS-G5A MAC for Safety Applications”,

An Approach for Selective Beacon Forwarding to Improve Cooperative Awareness”,

Cooperative Collision Warning Using Dedicated Short Range Wireless Communications”,

Degradation of Transmission Range in VANETs caused by Interference”, PIK - Praxis der Informationsverarbeitung und Kommunikation (Special Issue on Mobile Ad-hoc Networks),

Hariharan Krishnan and Timothy Talty,

IEEE Standard for Information technology – Telecommunications and information exchange between systems Local and metropolitan area networks–Specific requirements Part 11: Wireless LAN Medium Access Control (MAC) and Physical Layer (PHY)

Inter-Vehicle Communications: Assessing Information Dissemination under Safety Constraints”,

On the ability of the 802.11p mac method and stdma to support realtime vehicle-to-vehicle communication,”

Performance of MAC protocols in beaconing

Reliability Analysis of DSRC Wireless Communication for Vehicular Safety Applications”,

Simulation Platform for Inter-Vehicle Communications and Analysis of Periodic Information Exchange”,

Transport Systems (ITS); Vehicular Communications; Basic Set of Applications; Part 2: Specification of Cooperative

Update Delay: A new information-centric metric for a combined communication and application level reliability evaluation of CAM based Safety Applications

White Paper on Network Design Limits and VANET Performance”, Version 0.5, Car2Car Communication Consortium,

Update Delay: A new Information-Centric Metric for a Combined Communication and Application Level Reliability Evaluation of CAM based Safety Applications

Update Delay: A new Information-Centric Metric for a Combined Communication and Application Level Reliability Evaluation of CAM based Safety Applications

Abstract

Similar works

Full text

Available Versions

Institute of Transport Research:Publications