The Development of an Evaluation Framework for eGovernment Systems

This paper is a positioning paper which outlines a proposal for engaging in the evaluation of eGovernment systems. The primary purpose of our proposed research is to develop, apply, test, and disseminate an evaluation framework which can support continuous, adaptable, and reflective evaluation of eGovernment systems. The theoretical bases for the methodology will be the Information Systems (IS), Soft Systems Methodology, SSM (Checkland and Scholes, 1990) which provides the platform for the analyses of the ‘soft’ aspects (e.g. human, political, cultural and organisational factors) and the Hard Systems Methodology (HSM) which provides methods and tools for quantitative measures and analyses of the system. A further three interrelated bases are: Reflective Practice, Organisational Learning (OL), and Information and Knowledge Management (IKM). Some of the key underlying principles to a successful evaluation framework are good data collection and analyses methods, an evaluative reflective practice approach whichentails the complete process of identification and analysis of strengths and problems, followed by rigorous testing, implementation, and revision of solutions. Such a cycle encourages organisational learning and promotes continuous improvement to both the evaluation framework and system. Additionally, it aims to cultivate an organisational culture that supports evaluation through reflection, continuous learning, and knowledge management which facilitates knowledge creation, capture, sharing, application and dissemination

The elicitation of key performance indicators of e-government providers: A bottom-up approach

Copyright @ 2013 EMCIS.Delivering an adequate e-Government service (e-service) is becoming more of a necessity in today's digital world. In order to improve e-services and increase the engagement of both users' and providers' side, studies on the performance evaluation of such provided e-services are taking places. However a clear identification of the key performance indicators from the e-Government providers’ side is not well explored. This shortcoming hampers the conduct of a holistic evaluation of an e-service provision from the perspective of its stakeholders in order to improve e-services as well as to increase e-services take-ups. In this paper, a systematic process to identify indicators is implemented based on a bottom-up approach. The process used three focus-group meetings with providers, users, and academics in Qatar, Lebanon and UK to collect, identify and validate key indicators from the perspective of e-services’ providers. The approach resulted in the identification of five factors levels (service, technology, employees, policy and management and social responsibilities) with fifteen sub-categories of SMART variables. Hence, leading to the development of a new model, STEPS, that can fully explain and predict e-government success from the providers’ point of view. It will work as a strategic management tool to align various stakeholders on common goal and values based on evidence based evaluation of e-services using smart measurable indicators for the improvement of an e-service at the engagement level in the field of e-government. In addition, other fields can benefit from the outcome of this work, such as logistics service providers, who make their services available across new and existing relationships between the Internet commerce firms, their customers, and their vendors

A Tale of Evaluation and Reporting in UK Smart Cities

Global trends towards urbanisation are associated with wide-ranging challenges and opportunities for cities. Smart technologies create new opportunities for a range of smart city development and regeneration programmes designed to address the environmental, economic and social challenges concentrated in cities. Whilst smart city programmes have received much publicity, there has been much less discussion about evaluation of smart city programmes and the measurement of their outcomes for cities. Existing evaluation approaches have been criticised as non-standard and inadequate, focusing more on implementation processes and investment metrics than on the impacts of smart city programmes on strategic city outcomes and progress. To examine this, the SmartDframe project conducted research on city approaches to the evaluation of smart city projects and programmes, and reporting of impacts on city outcomes. This included the ‘smarter’ UK cities of Birmingham, Bristol, Manchester, Milton Keynes and Peterborough. City reports and interviews with representative local government authorities informed the case study analysis. The report provides a series of smart city case studies that exemplify contemporary city practices, offering a timely, insightful contribution to city discourse about best practice approaches to evaluation and reporting of complex smart city projects and programmes

Towards evaluation design for smart city development

Smart city developments integrate digital, human, and physical systems in the built environment. With growing urbanization and widespread developments, identifying suitable evaluation methodologies is important. Case-study research across five UK cities - Birmingham, Bristol, Manchester, Milton Keynes and Peterborough - revealed that city evaluation approaches were principally project-focused with city-level evaluation plans at early stages. Key challenges centred on selecting suitable evaluation methodologies to evidence urban value and outcomes, addressing city authority requirements. Recommendations for evaluation design draw on urban studies and measurement frameworks, capitalizing on big data opportunities and developing appropriate, valid, credible integrative approaches across projects, programmes and city-level developments

City indicators : now to Nanjing

This paper provides the key elements to develop an integrated approach for measuring and monitoring city performance globally. The paper reviews the role of cities and why indicators are important. Then it discusses past approaches to city indicators and the systems developed to date, including the World Bank's initiatives. After identifying the strengths and weaknesses of past experiences, it discusses the characteristics of optimal indicators. The paper concludes with a proposed plan to develop standardized indicators that emphasize the importance of indicators that are measurable, replicable, potentially predictive, and most important, consistent and comparable over time and across cities. As an innovative characteristic, the paper includes subjective measures in city indicators, such as well-being, happy citizens, and trust.Cultural Policy,City Development Strategies,Cultural Heritage&Preservation,ICT Policy and Strategies,Housing&Human Habitats

A business model perspective for ICTs in public engagement

This is the post-print version of the Article. The official published article can be accessed from the link below - Copyright @ 2012 ElsevierPublic institutions, in their efforts to promote meaningful citizen engagement, are increasingly looking at the democratic potential of Information and Communication Technologies (ICTs). Previous studies suggest that such initiatives seem to be impeded by socio-technical integration barriers such as low sustainability, poor citizen acceptance, coordination difficulties, lack of understanding and failure to assess their impact. Motivated by these shortcomings, the paper develops and applies a business model perspective as an interceding framework for analysis and evaluation. The underlying principle behind this approach is that it is not technology per se which determines success, but rather the way in which the businessmodel of the technological artifact is configured and employed to achieve the strategic goals. The business model perspective is empirically demonstrated with the case of an online petitioning system implemented by a UK local authority. The case illustrates the importance of considering ICTs in public engagement from a holistic view to make them more manageable and assessable

‘Modernisation’ and the role of policy levers in the Learning and Skills Sector

This paper examines the changing use of policy levers in the English post-compulsory education and training system, often referred to as the Learning and Skills Sector. Policy steering by governments has increased significantly in recent years, bringing with it the development of new forms of arms-length regulation. In the English context these changes were expressed during the 1980s and 1990s through neo-liberal New Public Management and, since 1997, have been extended through the New Labour government’s project to further ‘modernise’ public services. We look here at the changing use of policy levers (focussing in particular on the role of targets, funding, inspection, planning and initiatives) over three historical phases, paying particular attention to developments since the formation of the Learning and Skills Council (LSC) in 2001. We conclude by considering the range of responses adopted by education professionals in this era of ‘modernisation’

COBRA framework to evaluate e-government services: A citizen-centric perspective

E-government services involve many stakeholders who have different objectives that can have an impact on success. Among these stakeholders, citizens are the primary stakeholders of government activities. Accordingly, their satisfaction plays an important role in e-government success. Although several models have been proposed to assess the success of e-government services through measuring users' satisfaction levels, they fail to provide a comprehensive evaluation model. This study provides an insight and critical analysis of the extant literature to identify the most critical factors and their manifested variables for user satisfaction in the provision of e-government services. The various manifested variables are then grouped into a new quantitative analysis framework consisting of four main constructs: cost; benefit; risk and opportunity (COBRA) by analogy to the well-known SWOT qualitative analysis framework. The COBRA measurement scale is developed, tested, refined and validated on a sample group of e-government service users in Turkey. A structured equation model is used to establish relationships among the identified constructs, associated variables and users' satisfaction. The results confirm that COBRA framework is a useful approach for evaluating the success of e-government services from citizens' perspective and it can be generalised to other perspectives and measurement contexts. Crown Copyright © 2014.PIAP-GA-2008-230658) from the European Union Framework Program and another grant (NPRP 09-1023-5-158) from the Qatar National Research Fund (amember of Qatar Foundation

Wakefield Area Working Evaluation Framework - Literature Review

The Wakefield Metropolitan District Council (MDC) Area Working programme offers a radical and innovative approach to service improvement and redesign with the aim of better meeting community needs and addressing inequalities. Community participation is a core component of the Wakefield model, which is underpinned by the wider goals of encouraging active citizenship and community empowerment. There is an acknowledged need to evidence whether the Area Working approach leads to improved services, to what extent there is meaningful community engagement and ultimately to assess whether it makes a difference to people in their neighbourhoods. This literature review addresses a range of key questions that will usefully inform the development of an evaluation framework in relation to the deployment of Area Working within Wakefield. The aims of the literature review were to scope existing models of evaluation used to assess the deployment and impact of Area Working and to identify potential evaluation frameworks and benchmark indicators. A systematic literature search was undertaken to identify published and grey literature on Area Working and similar programmes and relevant literature was reviewed. This search was supplemented by key literature identified through previous research. This brief report presents a summary of findings and makes some recommendations for the development of an evaluation framework for Wakefield Area Working Programme