The research described here was supported by the award made by the RCUK Digital Economy programme to the dot.rural Digital Economy Hub, award reference: EP/G066051/1; and by the Innovate UK award reference: 102615.Peer reviewedPostprin

Corsar, David

Edwards, Peter

Journal of Data and Information Quality

English

Public sector organisations worldwide are implementing Open Data initiatives, which, it is hoped, will stimulate economic growth, increase transparency and accountability, and improve engagement between data consumers (typically citizens) and data holders/publishers (Open Knowledge 2012). Open Data is defined as data that "anyone can freely access, use, modify, and share for any purpose." As developers have started using open data within their applications, they are reporting quality issues with such datasets that have subsequently been addressed by the data publishers. These include OpenStreetMap developers correcting the location of 18,000 UK bus stops and users identifying errors and omissions in data relating to UK registered charities. There is a growing recognition among the open data community that, to maximize the impact of such initiatives in terms of economic growth and increased accountability, focus must shift from publication of data to issues such as coverage, openness, and quality. Definitions of quality in the open data context vary considerably; for example, the European Data Portal considers data to be of high quality if "humans can understand it and machines can manipulate it" and point to the 5 Star Open Data rating system as a data marque. Others, such as the G8 Open Data Charter and the Open Data Institute Certification Badges focus on the provision of metadata, data schema descriptions, use of shared data dictionaries, license used, file format, and publisher support for interacting with data users

Open Access Institutional Repository at Robert Gordon University

CORSAR, D. and EDWARDS, P. 2017. Challenges of open data quality: more than just license, format, an customer support. Journal of data and information quality [online], 9(1), pages 1-4. Available from: https://doi.org/10.1145/3139489      © ACM, 2017. This is the author's version of the work. It is posted here by permission of ACM for your personal use. Not for redistribution. The definitive version was published in Journal of Data and Information Quality Volume 9 Issue 1, September 2017, http://doi.acm.org/10.1145/3110291  This document was downloaded from https://openair.rgu.ac.uk Challenges of open data quality: more than just license, format, an customer support. CORSAR, D. and EDWARDS, P. 2017 AChallenges of Open Data Quality: More Than Just License, Format,and Customer SupportDAVID CORSAR, University of AberdeenPETER EDWARDS, University of AberdeenCCS Concepts: rSocial and professional topics → Quality assurance; rApplied computing → E-government;Additional Key Words and Phrases: Open DataACM Reference Format:David Corsar, and Peter Edwards, XXXX. Challenges of Open Data Quality More Than Just License, For-mat, and Customer Support. JDIQ V, N, Article A (January YYYY), 3 pages.DOI: http://dx.doi.org/10.1145/0000000.00000001. INTRODUCTIONPublic sector organisations world-wide are implementing Open Data initiatives,which, it is hoped, will stimulate economic growth, increase transparency and account-ability, and improve engagement between data consumers (typically citizens) and dataholders/publishers [Open Knowledge 2012]. Open Data is defined as data that “any-one can freely access, use, modify, and share for any purpose.”1 As developers havestarted using open data within their applications they are reporting quality issueswith such data sets that have subsequently been addressed by the data publishers2.These include OpenStreetMap3 developers correcting the location of 18,000 UK busstops, and users identifying errors and omissions in data relating to UK registeredcharities. There is a growing recognition among the open data community that, inorder to maximize the impact of such initiatives in terms of economic growth andincreased accountability, focus must shift from publication of data to issues such ascoverage, openness, and quality4. Definitions of quality in the open data context varyconsiderably; for example, the European Data Portal considers data to be of high qual-ity if “humans can understand it and machines can manipulate it”5 and point to the 5Star Open Data rating system6 as a data marque. Others, such as the G8 Open Data1http://opendefinition.org/2For further examples see, http://bit.ly/opendata-betterdata3http://www.openstreetmap.org4https://opendatawatch-public.sharepoint.com/Pages/MR-Indices.aspx5http://www.europeandataportal.eu/elearning/en/module5/#/id/co-016http://5stardata.info/The research described here was supported by the award made by the RCUK Digital Economy programmeto the dot.rural Digital Economy Hub, award reference: EP/G066051/1; and by the Innovate UK awardreference: 102615.Author’s addresses: D. Corsar and P. Edwards, Computing Science, University of Aberdeen, Aberdeen, UK,AB24 5UA.Permission to make digital or hard copies of all or part of this work for personal or classroom use is grantedwithout fee provided that copies are not made or distributed for profit or commercial advantage and thatcopies bear this notice and the full citation on the first page. Copyrights for components of this work ownedby others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, orrepublish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Requestpermissions from permissions@acm.org.c© YYYY Copyright held by the owner/author(s). Publication rights licensed to ACM. 1539-9087/YYYY/01-ARTA $15.00DOI: http://dx.doi.org/10.1145/0000000.0000000ACM Journal of Data and Information Quality, Vol. V, No. N, Article A, Publication date: January YYYY.A:2 D. Corsar and P. Edwards.Table I. Sample Open Air Quality Data. Extract from Comma-Separated Values file downloaded from the UKDEFRA data archive, providing data measured at two monitoring sites in Aberdeen, UK.Hourly measurement data supplied by UK-air on 25/10/2016All Data GMT hour endingStatus: V=Verified P=Provisionaly Verified N=Not Verified S=SuspectAberdeen AberdeenUnion StreetRoadsideDate Time VolatilePM2.5(Hourlymeasured)Status SulphurdioxideStatus Nitrogenoxides asnitrogendioxideStatus01/01/2016 01:00:00 -3 V ugm-3(TEOM FDMS)Nodata92.80215 V ugm-301/01/2016 02:00:00 -4.2 V ugm-3(TEOM FDMS)Nodata52.66069 V ugm-301/01/2016 03:00:00 -2.8 V ugm-3(TEOM FDMS)Nodata60.26527 V ugm-3Charter7 and the Open Data Institute Certification Badges8 focus on the provision ofmetadata, data schema descriptions, use of shared data dictionaries, license used, fileformat, and publisher support for interacting with data users.2. AN EXAMPLE: POLLUTANT EMISSION DATAPollutant emissions are a key dataset included in the Global Open Data Index assess-ment9, and are currently published by 61 of the 121 countries included in the latestsurvey. In the UK this data is recorded at 300 monitoring sites across the country, andpublished as CSV files by the DEFRA Data Archive10; Table I provides an extract ofdata obtained from this service for two monitoring sites in the city of Aberdeen, UK11.While small, this extract illustrates the kind of quality issues frequently encounteredin open data. The file does not comply with the RFC 4180 CSV specification12, as thefirst three rows provide meta-information about the data, rather than a header lineand data records. Consistency issues are also evident: the “Status” column providestwo pieces of information: the datum’s status (indicating if it has been subjected toDEFRA’s data verification process13) and the measurement units; and missing dataappears to be handled inconsistently, either with the text “No data” or an empty cell.Accuracy issues are also evident, as shown by the negative values for Volatile PM2.5,which typically indicate an error with the calibration of the device, despite these valueshaving been verified, indicating that the PM2.5 values cannot be trusted.In Scotland, emissions data is also available from the Scottish Air Quality (SAQ)website14. Comparison of data from the SAQ and DEFRA sources identifies a furtheraccuracy issue: DEFRA provide data to (several) decimal places, while the SAQ serviceappears to round the data to zero decimal places. There are also inconsistencies in theschemas, as the SAQ dataset includes a “units” column, instead of overloading the“Status” column as shown in Table I. Such consistency issues are also evident when7https://www.gov.uk/government/publications/open-data-charter/g8-open-data-charter-and-technical-annex8https://certificates.theodi.org/en/about/badgelevels9http://index.okfn.org/dataset/10https://uk-air.defra.gov.uk/data/data selector11The complete dataset can be download at https://uk-air.defra.gov.uk/data/data selector?q=695574#mid12https://www.ietf.org/rfc/rfc4180.txt13https://uk-air.defra.gov.uk/assets/documents/The Data Verification and Ratification Process.pdf14http://www.scottishairquality.co.uk/data/data-selectorACM Journal of Data and Information Quality, Vol. V, No. N, Article A, Publication date: January YYYY.Challenges of Open Data Quality: More Than Just License, Format, and Customer Support A:3comparing emissions data published by different countries: for example, the EPA inthe USA15 uses the label “ug/m3 SC” for Volatile PM2.5, while the UK uses “ugm-3.”3. IMPROVING THE QUALITY OF OPEN DATA: CHALLENGES AND OPPORTUNITIESWe must consider the extent to which the quality of data in open data portals can beimproved given the variety of such data (in terms of domains covered and intendeduse by “anyone ... for any purpose”) and the typically subjective, use oriented view ofquality [Wang and Strong 1996]. We believe this highlights an opportunity for dataquality researchers to consult with the open data community to document the qualityissues they experience, with the aim of identifying two categories of quality metrics:those generally relevant to all open datasets; and those relevant to the various typesof data that are routinely published. Examples of the former include completeness ofthe dataset, representational consistency (including how data and missing data aredescribed), accessibility, conformance to file formats, and metrics developed by [Neu-maier et al. 2016] to assess the quality of metadata in repositories as part of improvingdiscoverability; an example data type specific metric would be Volatile PM2.5 shouldbe greater than 0, which is potentially relevant to multiple publishers and datasets.While both types of metrics could be used by data publishers to perform qual-ity assurance, the prominent challenge then becomes how to support such individu-als/organisations, who may not necessarily have a technical or data-specialist back-ground, with understanding how the metrics may apply to their data, when and howany assessment processes should be utilised, and how to use the results of those as-sessments to improve the quality of the data they publish. This requires working withopen data publishers to identify suitable intervention points within the publicationprocess, an activity which may benefit from alignment with guides, such as the opendata handbook [Open Knowledge 2012], and integration of quality tools into softwareframeworks, such as the W3C Digital Data Toolkit16. A related challenge here is howto effectively utilise the crowd (both data users, and the wider network of workersavailable via crowdsourcing platforms) in the quality improvement process. As dis-cussed above, there have been instances of ad-hoc, informal feedback from data usersto publishers, and more generally, [Acosta et al. 2013] have demonstrated the use ofthe crowd to identify and resolve quality issues that are beyond the current capabili-ties of machines. For example, improving the semantic accuracy of bus stop locations.However, the use of the crowd is non-trivial and is not guaranteed to produce benefi-cial outcomes [Markovic and Edwards], and so raises new research opportunities intothe appropriate workflows and incentive/reward models for using the crowd to identifyand resolve quality issues with open data.REFERENCESM. Acosta, A. Zaveri, E. Simperl, D. Kontokostas, S. Auer, and J. Lehmann. 2013. Crowdsourcing LinkedData Quality Assessment. In The Semantic Web – ISWC 2013. Springer Berlin Heidelberg, 260–276.M. Markovic and P. Edwards. In press. The Challenge of Quality in Social Computation. Journal of Dataand Information Quality (In press).S. Neumaier, J. Umbrich, and A. Polleres. 2016. Automated Quality Assessment of Metadata Across OpenData Portals. Journal of Data and Information Quality 8, 1, Article 2 (Oct. 2016), 29 pages.Open Knowledge. 2012. The Open Data Handbook. Retrieved Feb 1, 2016 fromhttp://opendatahandbook.org/.R.Y. Wang and D.M. Strong. 1996. Beyond Accuracy: What Data Quality Means to Data Consumers. Journalof Management Information Systems 12, 4 (1996), 5–33.15https://www.epa.gov/outdoor-air-quality-data16http://www.w3cdigitaldatatoolkit.comACM Journal of Data and Information Quality, Vol. V, No. N, Article A, Publication date: January YYYY.

Challenges of open data quality: more than just license, format, and customer support.

David Corsar

Peter Edwards

Crossref

Challenges of Open Data Quality

The research described here was supported by the award made by the RCUK Digital Economy programme to the dot.rural Digital Economy Hub, award reference: EP/G066051/1; and by the Innovate UK award reference: 102615.Peer reviewe

The Open University

AChallenges of Open Data Quality: More Than Just License, Format,and Customer SupportDAVID CORSAR, University of AberdeenPETER EDWARDS, University of AberdeenCCS Concepts: rSocial and professional topics → Quality assurance; rApplied computing → E-government;Additional Key Words and Phrases: Open DataACM Reference Format:David Corsar, and Peter Edwards, XXXX. Challenges of Open Data Quality More Than Just License, For-mat, and Customer Support. JDIQ V, N, Article A (January YYYY), 3 pages.DOI: http://dx.doi.org/10.1145/0000000.00000001. INTRODUCTIONPublic sector organisations world-wide are implementing Open Data initiatives,which, it is hoped, will stimulate economic growth, increase transparency and account-ability, and improve engagement between data consumers (typically citizens) and dataholders/publishers [Open Knowledge 2012]. Open Data is defined as data that “any-one can freely access, use, modify, and share for any purpose.”1 As developers havestarted using open data within their applications they are reporting quality issueswith such data sets that have subsequently been addressed by the data publishers2.These include OpenStreetMap3 developers correcting the location of 18,000 UK busstops, and users identifying errors and omissions in data relating to UK registeredcharities. There is a growing recognition among the open data community that, inorder to maximize the impact of such initiatives in terms of economic growth andincreased accountability, focus must shift from publication of data to issues such ascoverage, openness, and quality4. Definitions of quality in the open data context varyconsiderably; for example, the European Data Portal considers data to be of high qual-ity if “humans can understand it and machines can manipulate it”5 and point to the 5Star Open Data rating system6 as a data marque. Others, such as the G8 Open Data1http://opendefinition.org/2For further examples see, http://bit.ly/opendata-betterdata3http://www.openstreetmap.org4https://opendatawatch-public.sharepoint.com/Pages/MR-Indices.aspx5http://www.europeandataportal.eu/elearning/en/module5/#/id/co-016http://5stardata.info/The research described here was supported by the award made by the RCUK Digital Economy programmeto the dot.rural Digital Economy Hub, award reference: EP/G066051/1; and by the Innovate UK awardreference: 102615.Author’s addresses: D. Corsar and P. Edwards, Computing Science, University of Aberdeen, Aberdeen, UK,AB24 5UA.Permission to make digital or hard copies of all or part of this work for personal or classroom use is grantedwithout fee provided that copies are not made or distributed for profit or commercial advantage and thatcopies bear this notice and the full citation on the first page. Copyrights for components of this work ownedby others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, orrepublish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Requestpermissions from permissions@acm.org.c© YYYY Copyright held by the owner/author(s). Publication rights licensed to ACM. 1539-9087/YYYY/01-ARTA $15.00DOI: http://dx.doi.org/10.1145/0000000.0000000ACM Journal of Data and Information Quality, Vol. V, No. N, Article A, Publication date: January YYYY.A:2 D. Corsar and P. Edwards.Table I. Sample Open Air Quality Data. Extract from Comma-Separated Values file downloaded from the UKDEFRA data archive, providing data measured at two monitoring sites in Aberdeen, UK.Hourly measurement data supplied by UK-air on 25/10/2016All Data GMT hour endingStatus: V=Verified P=Provisionaly Verified N=Not Verified S=SuspectAberdeen AberdeenUnion StreetRoadsideDate Time VolatilePM2.5(Hourlymeasured)Status SulphurdioxideStatus Nitrogenoxides asnitrogendioxideStatus01/01/2016 01:00:00 -3 V ugm-3(TEOM FDMS)Nodata92.80215 V ugm-301/01/2016 02:00:00 -4.2 V ugm-3(TEOM FDMS)Nodata52.66069 V ugm-301/01/2016 03:00:00 -2.8 V ugm-3(TEOM FDMS)Nodata60.26527 V ugm-3Charter7 and the Open Data Institute Certification Badges8 focus on the provision ofmetadata, data schema descriptions, use of shared data dictionaries, license used, fileformat, and publisher support for interacting with data users.2. AN EXAMPLE: POLLUTANT EMISSION DATAPollutant emissions are a key dataset included in the Global Open Data Index assess-ment9, and are currently published by 61 of the 121 countries included in the latestsurvey. In the UK this data is recorded at 300 monitoring sites across the country, andpublished as CSV files by the DEFRA Data Archive10; Table I provides an extract ofdata obtained from this service for two monitoring sites in the city of Aberdeen, UK11.While small, this extract illustrates the kind of quality issues frequently encounteredin open data. The file does not comply with the RFC 4180 CSV specification12, as thefirst three rows provide meta-information about the data, rather than a header lineand data records. Consistency issues are also evident: the “Status” column providestwo pieces of information: the datum’s status (indicating if it has been subjected toDEFRA’s data verification process13) and the measurement units; and missing dataappears to be handled inconsistently, either with the text “No data” or an empty cell.Accuracy issues are also evident, as shown by the negative values for Volatile PM2.5,which typically indicate an error with the calibration of the device, despite these valueshaving been verified, indicating that the PM2.5 values cannot be trusted.In Scotland, emissions data is also available from the Scottish Air Quality (SAQ)website14. Comparison of data from the SAQ and DEFRA sources identifies a furtheraccuracy issue: DEFRA provide data to (several) decimal places, while the SAQ serviceappears to round the data to zero decimal places. There are also inconsistencies in theschemas, as the SAQ dataset includes a “units” column, instead of overloading the“Status” column as shown in Table I. Such consistency issues are also evident when7https://www.gov.uk/government/publications/open-data-charter/g8-open-data-charter-and-technical-annex8https://certificates.theodi.org/en/about/badgelevels9http://index.okfn.org/dataset/10https://uk-air.defra.gov.uk/data/data selector11The complete dataset can be download at https://uk-air.defra.gov.uk/data/data selector?q=695574#mid12https://www.ietf.org/rfc/rfc4180.txt13https://uk-air.defra.gov.uk/assets/documents/The Data Verification and Ratification Process.pdf14http://www.scottishairquality.co.uk/data/data-selectorACM Journal of Data and Information Quality, Vol. V, No. N, Article A, Publication date: January YYYY.Challenges of Open Data Quality: More Than Just License, Format, and Customer Support A:3comparing emissions data published by different countries: for example, the EPA inthe USA15 uses the label “ug/m3 SC” for Volatile PM2.5, while the UK uses “ugm-3.”3. IMPROVING THE QUALITY OF OPEN DATA: CHALLENGES AND OPPORTUNITIESWe must consider the extent to which the quality of data in open data portals can beimproved given the variety of such data (in terms of domains covered and intendeduse by “anyone ... for any purpose”) and the typically subjective, use oriented view ofquality [Wang and Strong 1996]. We believe this highlights an opportunity for dataquality researchers to consult with the open data community to document the qualityissues they experience, with the aim of identifying two categories of quality metrics:those generally relevant to all open datasets; and those relevant to the various typesof data that are routinely published. Examples of the former include completeness ofthe dataset, representational consistency (including how data and missing data aredescribed), accessibility, conformance to file formats, and metrics developed by [Neu-maier et al. 2016] to assess the quality of metadata in repositories as part of improvingdiscoverability; an example data type specific metric would be Volatile PM2.5 shouldbe greater than 0, which is potentially relevant to multiple publishers and datasets.While both types of metrics could be used by data publishers to perform qual-ity assurance, the prominent challenge then becomes how to support such individu-als/organisations, who may not necessarily have a technical or data-specialist back-ground, with understanding how the metrics may apply to their data, when and howany assessment processes should be utilised, and how to use the results of those as-sessments to improve the quality of the data they publish. This requires working withopen data publishers to identify suitable intervention points within the publicationprocess, an activity which may benefit from alignment with guides, such as the opendata handbook [Open Knowledge 2012], and integration of quality tools into softwareframeworks, such as the W3C Digital Data Toolkit16. A related challenge here is howto effectively utilise the crowd (both data users, and the wider network of workersavailable via crowdsourcing platforms) in the quality improvement process. As dis-cussed above, there have been instances of ad-hoc, informal feedback from data usersto publishers, and more generally, [Acosta et al. 2013] have demonstrated the use ofthe crowd to identify and resolve quality issues that are beyond the current capabili-ties of machines. For example, improving the semantic accuracy of bus stop locations.However, the use of the crowd is non-trivial and is not guaranteed to produce benefi-cial outcomes [Markovic and Edwards], and so raises new research opportunities intothe appropriate workflows and incentive/reward models for using the crowd to identifyand resolve quality issues with open data.REFERENCESM. Acosta, A. Zaveri, E. Simperl, D. Kontokostas, S. Auer, and J. Lehmann. 2013. Crowdsourcing LinkedData Quality Assessment. In The Semantic Web – ISWC 2013. Springer Berlin Heidelberg, 260–276.M. Markovic and P. Edwards. In press. The Challenge of Quality in Social Computation. Journal of Dataand Information Quality (In press).S. Neumaier, J. Umbrich, and A. Polleres. 2016. Automated Quality Assessment of Metadata Across OpenData Portals. Journal of Data and Information Quality 8, 1, Article 2 (Oct. 2016), 29 pages.Open Knowledge. 2012. The Open Data Handbook. Retrieved Feb 1, 2016 fromhttp://opendatahandbook.org/.R.Y. Wang and D.M. Strong. 1996. Beyond Accuracy: What Data Quality Means to Data Consumers. Journalof Management Information Systems 12, 4 (1996), 5–33.15https://www.epa.gov/outdoor-air-quality-data16http://www.w3cdigitaldatatoolkit.comACM Journal of Data and Information Quality, Vol. V, No. N, Article A, Publication date: January YYYY.

Challenges of Open Data Quality : More Than Just License, Format, and Customer Support

https://aura.abdn.ac.uk/bitstream/2164/9324/1/challenges_open_data.pdf

Challenges of Open Data Quality : More Than Just License, Format, and Customer Support

Abstract

Similar works

Full text

Available Versions

Open Access Institutional Repository at Robert Gordon University

Crossref

The Open University