Amateur Volunteered Geographic Information (VGI) has been used together with Professional Geographic Information (PGI) since its inception during the mid 00's alongside neogeography. While the geographic accuracy and quality of VGI has been demonstrated to be more than good enough, no previous research has been published on the influence VGI has on the user perceptions of the mashup. This paper presents a quantitative investigation into how including VGI in mashups, and telling users their mashup contains VGI influences user perceptions of quality and authority- which ultimately decide if the user wishes to utilise the mashup or not

Andrew May (1250316)

Christopher J. Parker (4862320)

Val Mitchell (1250562)

English

Loughborough University Institutional Repository

   This item was submitted to Loughborough’s Institutional Repository (https://dspace.lboro.ac.uk/) by the author and is made available under the following Creative Commons Licence conditions.      For the full text of this licence, please go to: http://creativecommons.org/licenses/by-nc-nd/2.5/  Using VGI in Online Design 1  Using VGI To Enhance User Judgements Of Quality And Authority Christopher J. Parker1, Andrew May2, Val Mitchell3 Loughborough Design School Loughborough University, Loughborough, LE11 3TU +44 (0)1509 22 6900 1[C.Parker] 2[A.J.May] 3[V.A.Mitchell] @lboro.ac.uk  ABSTRACT: Amateur Volunteered Geographic Information (VGI) has been used together with Professional Geographic Information (PGI) since its inception during the mid 00’s alongside neogeography. While the geographic accuracy and quality of VGI has been demonstrated to be more than good enough, no previous research has been published on the influence VGI has on the user perceptions of the mashup. This paper presents a quantitative investigation into how including VGI in mashups, and telling users their mashup contains VGI influences user perceptions of quality and authority- which ultimately decide if the user wishes to utilise the mashup or not. KEYWORDS: INFORMATION, ACCESS, QUALITY, HUMAN FACTORS; VGI   1 Introduction  Previous research has investigated the decision of consumers to utilise Volunteered Geographic Information (VGI: Goodchild 2007) within personal, professional and social realms, and the way VGI is perceived and utilised alongside Professional Geographic Information (PGI) within a real world use context (Parker et al. 2012). Here, Rieh (2002) demonstrated that it is more useful to consider the attributes of VGI and PGI (e.g. its currency) rather than the professionalism of the contributor. Importantly, it was shown that the consumer places similar critical analysis and judgements on both VGI and PGI. However, a need arises to understand how these perceptions influence the design of online mashups that present geographic-based information. The aim of this research was to understand in detail the impact of including VGI alongside PGI on user judgements from a human factors perspective. The motivation for undertaking such a study was to relate demonstrable accuracy with perceived quality related to a consumer-user. Ultimately, this knowledge can be used to identify unique opportunities for VGI to increase confidence and utility in the end user’s information use experience. In order to understand the perceptions of VGI and PGI in sufficient detail, a singular Special Interest Group (SIG) was required. Previous research (Parker et al. 2010) has suggested that SIGs understand, produce and integrate VGI more than other user groups. For this investigation wheelchair users were selected as the target consumer group as they 1) exhibit the same personal needs as the general population outside of their user group, 2) are required to critically evaluate information relative to their end goals to overcome their disability, and 3) inhabit a relatively information poor environment providing scope for research and investigation that has some real world impact. Their activity also involves a certain level of risk, which promotes an increased amount of critical reflection during their information search (Carlson and Gieseke 1983). Importantly, wheelchair access is not the focus of this paper, but instead the way a singular user group uses VGI (from a homogenous contributor base) alongside and in relation to PGI. 2 Supporting Literature Accessibility, as defined by Handy and Niemeir (1997), is determined by the spatial distribution of potential destinations, the ease of reaching each destination, and the magnitude, quality, and character of the activities found there. Ray and Ryder (2003) demonstrated in an investigation into the information sources used by disabled tourists how “word of mouth, the internet and travel guides” Using VGI in Online Design 2  formed the most important information sources when planning for travel. Despite such studies and a variety of official guidelines on wheelchair access (Evans 2009, HMSO 2010, Ray and Ryder 2003) no published research exists linking the human factors of access information to VGI contained within the data. This is in spite of projects such as WalksWithWheelchairs (walkswithwheelchairs.com 2008) which have been in existence for as long as VGI has been named. Rieh (2002) has developed a framework to describe how users perceive information delivered online. In reflection on her findings, Rieh commented that relevance criteria may be used to explore and assess the users’ perception of information with characteristics of information objects, characteristics of services, user knowledge, tasks and other factors influencing the users perception of information. This was describe in terms of quality (good, accurate, current, useful and important) and authority (trustworthy, credible, reliable, scholarly, official and authoritative). Applying this theoretical framework, the influencing factors may be controlled or varied within an experimental website to explore the influence of these changes on the user within measureable subcategories.  3 Methodology VGI Data as presented within the experiment was collected through five participant observation sessions, working with wheelchair users along a specific public transport route in London from the position of participant as observer. PGI data about accessibility related issues along the specified route was collected through sources such as Transport For London (TFL 2009), South Eastern Rail (Southeastern 2011) and Network Rail (Network Rail 2011).  VGI and PGI data was compiled within an interactive experimental website (www.freetraveller.co.uk) which assigned visitors (participants) to one of four groups. They were then presented with three maps of travel access information and asked to move around the interface considering how the information may be of use to them when planning a trip to London; see Figure 1.  Using VGI in Online Design 3   Figure 1. An Example Mashup As Used In The Experiment Following this they were asked to complete a 32 question Likert scale survey to assess their judgements on quality and authority. The conditions under which the groups were assigned is presented in Table 1.  Table 1. Variance of Information Presented to Participants and Group Conditions  Information Presented on Map PGI PGI + VGI What Participants are Told The Map Contains PGI Group 1 Group 2 PGI + VGI Group 3 Group 4  A third option of presenting participants with only VGI data was not included due to time and budget constraints, difficulties in achieving a large enough sample size and the focus of this paper on use of VGI alongside PGI, rather than VGI instead of PGI. MANOVA analysis was run on the results of the survey using SPSS 19 (IBM 2011) to attain 1) the influence of the inclusion of VGI alongside PGI within a mashup, 2) the influence of believing the information presented within the mashup includes VGI alongside PGI and 3) how may these influences be understood in terms of quality and authority; including their subcategories.  3.1 Descriptive Statistics Analysis included 101 participants, all being permanent wheelchair users between 18 and 65 without cognitive or sensory disabilities; see Table 2. None of those involved with the collection of the VGI data set took part in the online experiment to prevent confounding of results through previous experience. Additionally, none of the participants were members of any Special Interest Groups (SIG) associated with VGI projects (e.g. OpenStreetMap, WikiMapia, etc.). Therefore the study participants were considered unbiased towards VGI, as would have been expected from SIG members who hold a consistent bias towards VGI (Parker et al. 2010). Using VGI in Online Design 4  Table 2. Breakdown Of Experiment Participants  Sex  Male Female TOTAL Group 1 11 12 23 Group 2 16 17 33 Group 3 6 16 22 Group 4 7 16 23 TOTAL 40 60 101  The confidence and familiarity of the user with online mashups was a potential limiting factor of the analysis. However, as Figure 2 demonstrates, the vast majority of participants were very confident using online maps prior to engagement with the Free Traveller experiment. Consequently, the influence of participants being uncomfortable using mashups similar to those included in the experiment can be considered negligible.  Figure 2. Participant Confidence Using Online Maps (rating scale where 1 represents very low confidence and 5 represents very high confidence)  4 Results and Analysis A two-way between-group multivariate analysis was performed to investigate the inclusion of VGI alongside PGI within a mashup, and the influence of being told a mashup contains VGI alongside PGI on the user judgement of a mashups quality and authority. Nine dependent variables were used: goodness, accuracy, currency, usefulness, importance, credibility, reliability, official and authority. The independent variables were: 1) belief that their map contained VGI alongside PGI and 2) whether their mashup actually contained VGI; irrespective of belief. Preliminary assumption testing was conducted to check for normality, linearity, univariate and multivariate outliers, homogeneity or variance-covariance matrices, and multicollinearity, with no serious violations noted.  No significant differences were observed between those who were told that their mashups contained PGI + VGI and those who were told that their mashups contained only PGI, (F (9, 89) = .79, p = .625; Using VGI in Online Design 5  Wilks’ Lambda = .93; ηp2 = .074). There were statistically significant differences between those who were presented with mashups containing PGI + VGI and those who were presented with mashups containing only PGI on the combined dependant variables, (F (9, 89) = 2.99, p = .004; Wilks’ Lambda = .77; ηp2 = .232). No statistically significant interaction was observed between the fixed variables. When the results for the dependent variables related to the information as presented to the participants were considered separately, the only user judgement to reach statistical significance, using a Bonferroni adjusted alpha level of .006, was currency: F (1, 97) = 10.81, p = .001, ηp2 = .10. The ηp2 of .10 represents 10 per cent of the variance in perceived currency scores explained by belief that the mashup in use contains VGI. Under the generally accepted criteria of Cohen (1988) this constitutes a medium effect size. An inspection of the mean scores indicated that those who believed that their mashup contained PGI + VGI reported slightly higher levels of perceived currency in the map date (  = 13.43, SD = 2.44) than those who had believed that their mashup contained only PGI (  = 12.95, SD = 2.30).  5 Discussion 5.1 Quality and Authority Overall, VGI was shown to increase perceptions of the mashups quality and authority by a significant amount, as well as producing an enhanced acceptance of the system; use experience. However, it was the aspect of quality which showed the greatest enhancement. In a study examining user perceptions of Wikipedia, Yaari et al. (2011) highlighted how increasing the amount of information available to the user increased perceptions of quality and authority, agreeing with Blumenstock (2008) and Tillotson (2002). However, it should be noted that Rieh (2002) did not find quality to be as much of a major criterion in terms of users judgement the appropriateness of online information. An explanation of why this may be occurring was offered by Yaari et al. (2011) as users assessing the content of information via the rule of thumb of length equating to quality.  Within the experimental sample telling participants that their data contained VGI and PGI did not achieve statistical significance over informing them that their data contained just PGI, it is important not to disregard such outcome as meaningless. The non-significance of this outcome within the sample demonstrates that the commonly held perception of amateur volunteered information may not have a strong influence on the way information is judged online in terms of fitness for purpose.  5.2 Currency Presenting users with VGI alongside PGI produced a significant and positive influence on judgements of currency with a medium effect size. This outcome is supported by Goodchild (2008), who commented that “perhaps the most significant area of geospatial data qualities for VGI is currency, or the degree to which the database is up-to-date”.   This finding may be explained by the work of Nolan (1976), Gitelson and Crompton (1983) and Schuett (1993) who demonstrated that information received from informal sources is the most informative due to its ability to reflect changes in the environment. A final perspective on this may be gained by considering the way information from volunteers was presented as different to that of professionals. VGI covered more general and subjective/ temporal issues (e.g. lifts hard to find, need better signs) while PGI covered more objective features (e.g. station is step free for easy wheelchair access). While in combination they may increase the users perception of the website due to the added scale of information (Idris et al. 2011) another explanation could be that the inclusion of objective temporal information increases perceptions of the information being current. 6 Conclusion This study has demonstrated that by including VGI within a mashup alongside PGI, the overall user Using VGI in Online Design 6  perception of quality were noticeably increased; noticeably influenced by perceived currency of the information. Consequently the influence of VGI within neogeography (mashups) has an impact on enhancing the usability of the GIS system, rather than simply being a faster or more cost effective way of sourcing GIS.  This work suggests that it would be valuable to further research the influence of VGI alongside PGI upon the information perceptions of end users.  This could in turn lead to mashups of higher usability in terms of the satisfaction and confidence that end users feel in using the information provided; for example to manage the risk associated with their information search related activity. Further research should investigate the outcomes of this study in relation to user groups with similar needs to the wheelchair users (in terms of constraints on independent travel), for example parents with push chairs along the same public transport route in London. This will develop the understanding of how VGI has a general impact on usability, and which factors applicable to general information search, or are unique to wheelchair users. This may lead to further generation of usability design related theory for inclusion of VGI and PGI within the neogeographic domain. 7 References  Blumenstock J E (2008). Size Matters: Word Count as a Measure of Quality on Wikipedia. In Proceeding of the 17th international conference on World Wide Web, Beijing, China.  Carlson J A and Gieseke R J (1983). Price Search in a Product Market. Journal of Consumer Research, 9(4), 357-365.  Cohen J (1988). Statistical Power Analysis for the Behavioral Sciences (Second edition). Hillsdale, New Jersey, USA, Lawrence Erlbaum Associates.  Evans G (2009). Accessibility, Urban Design and the Whole Journey Environment. Built Environment, 35(3), 366-385.  Gitelson R J and Crompton J L (1983). The Planning Horizons and Sources of Information used by Pleasure Vacationers. Journal of Travel Research, 21(3), 2-7.  Goodchild M F (2008). Spatial Accuracy 2.0. In Spatial Uncertainty: Proceedings of the Eighth International Symposium on Spatial Accuracy Assessment in Natural Resources and Environmental Sciences, Liverpool, UK.  Goodchild M F (2007). Citizens as Sensors: The World of Volunteered Geography. GeoJournal, 69(4), 211-221.  Handy S L and Niemeir D A (1997). Measuring Accessibility, an Exploration of Issues and Alternatives. Environment and Planning A, 291175-1194.  HMSO (2010). Equality Act 2010. Civil Rights edn. Parliament of the United Kingdom, London, UK.  IBM (2011). SPSS. USA: IBM.  Idris N H, Jackson M J and Abrahart R J (2011). Colour Coded Traffic Light Labeling: A Visual Quality Indicator to Communicate Credibility in Map Mash-Up Applications. In The International Conference on Humanities, Social Sciences, Science & Technology (ICHSST), Manchester, UK.  Network Rail (2011). London Bridge. WWW document, http://www.nationalrail.co.uk/stations/lbg/details.html  Nolan S D (1976). Tourists' use and Evaluation of Travel Information Sources: Summary and Conclusions. Journal of Travel Research, 14(3), 6-8.  Using VGI in Online Design 7  Parker C J, May A J and Mitchell V (2012). The Role of VGI and PGI in Supporting Outdoor Activities. Applied Ergonomics,  Parker C J, May A J and Mitchell V (2010). An Exploration of Volunteered Geographic Information Stakeholders. In Proceedings of the GIS Research UK 18th Annual Conference, London, UK.  Ray N M and Ryder M E (2003). ‘‘Ebilities’’ Tourism: An Exploratory Discussion of the Travel Needs and Motivation of the Mobility-Disabled. Tourism Management, 2457-72.  Rieh S Y (2002). Judgment of Information Quality and Cognitive Authority in the Web. Journal of the American Society for Information Science and Technology, 53(2), 145-161.  Schuett M A (1993). Information Sources and Risk Recreation: The Case of Whitewater Kayakers. Journal of Park and Recreation Administration, 11(1), 67-77.  Southeastern (2011). Greenwich. WWW document, http://www.nationalrail.co.uk/stations/gnw/details.html  TFL (2009). Step-Free Tube Guide. London Underground edn. Transport for London, London, UK.  Tillotson J (2002). Web Site Evaluation: A Survey of Undergraduates. Online Information Review, 26(6), 392-403.  walkswithwheelchairs.com (2008). Thornton Reservoir. WWW document, http://www.walkswithwheelchairs.com/Leicestershire/587  Yaari E, Baruchson-Arbib S and Bar-Ilan J (2011). Information Quality Assessment of Community Generated Content: A User Study of Wikipedia. Journal of Information Science, 37(5), 487-498.   8 Biography Christopher J. Parker Research Associate at Loughborough University Design School, focusing on the Usability of Volunteered Geographic Information from a Human Factors perspective. Andrew May and Val Mitchell are Research Fellows specialising in User-Centred Design of new technologies. This study was undertaken within the Ideas in Transit project (www.ideasintransit.org), funded by the UK government (via the Engineering and Physical Sciences Research Council, the Technology Strategy Board and the Department for Transport).  

Using VGI to enhance user judgements of quality and authority

https://repository.lboro.ac.uk/articles/Using_VGI_to_enhance_user_judgements_of_quality_and_authority/9338456/files/16947089.pdf

Using VGI to enhance user judgements of quality and authority

Abstract

Similar works

Full text

Available Versions

Loughborough University Institutional Repository