Holistic Framework for Monitoring Quality in Higher Education Institutions in the Kingdom of Saudi Arabia using Business Intelligence Dashboards

Quality Assurance (QA) in Higher Education Institutions (HEIs) is a challenging issue as it requires continuous improvement to cope with operational and financial difficulties. Governments impose QA standards to guarantee a minimum level of service quality that HEI should provide. HEIs in the Kingdom of Saudi Arabia (KSA) face challenges in monitoring QA as they are required to report on their compliance with the National Centre for Academic Accreditation and Evaluation (NCAAA) QA standards by measuring 23 Key Performance Indicators (KPIs). HEIs typically use interactive spreadsheets and surveys to measure the degree of compliance through annual audit. This practice is time consuming, requires significant personnel, and the results are not reported in a timely manner. If an institution fails to comply with a certain standard at any period in the academic year, top management may not be notified until the next audit takes place. This affects the accreditation status of the institution. HEIs also tend to develop their own performance metrics and institutional specific KPIs to keep track of strategic planning processes. HEIs in KSA are making increasing use of Social Media. Public opinions expressed in Social Media can result in negative impact on the HEI’s reputation, and consequently, affect their financial standing. The literature suggests that HEIs should monitor public opinion on Social Media to be able to obtain valuable feedback on user satisfaction and quality of services. Frameworks for monitoring QA in HEIs, as described in the current literature, were not developed for the KSA environment, and so do not meet the needs of KSA HEIs. Current frameworks do not take into consideration the mandatory requirements of NCAAA 2018, monitoring of institutional specific KPIs, and sentiment analysis of Social Media data. There was also a need to provide guidance to HEIs as to which factors should be included in QA systems to provide a systematic approach to QA monitoring in KSA. This research provides a novel solution to these challenges by developing a Holistic Framework for monitoring Higher Education Quality using Business Intelligence dashboards (HF-HEQ-BI). The HF-HEQ-BI provides a mechanism for determining the requirements for QA monitoring in HEIs in KSA and enables the factors to be mapped to support visualisation through Business Intelligence dashboards. A Framework Utilisation Tool was also developed to show how HF-HEQ-BI factors can be visualised for the purpose of developing BI dashboards. The HF-HEQ-BI framework was developed from a literature review and Gap Analysis. The developed framework addresses the factors that HEIs should consider when designing QA systems and takes into consideration the role of Social Media in monitoring service quality. To validate the selection of the factors on which the framework was based, the factors were validated with a panel of domain experts and a survey of QA practitioners was also carried out. The use of qualitative and quantitative approaches provided triangulation. A prototype dashboard was developed from the HF-HEQ-BI framework for the purpose of monitoring QA in HEIs in KSA using a case study based on HEI in Saudi Arabia. An evaluation tool for BI systems in HE was developed, and the prototype dashboard was evaluated by practitioners. The proposed dashboard was based on the underpinning framework utilisation tool which provides visualisation reporting utilising the 8 NCAAA standards and 23 KPIs, institutional specific KPIs, and sentiment analysis of Social Media data

A user experience framework for business intelligence dashboards : an appreciative inquiry study within an agile software development environment

In South Africa and globally, companies focusing on business optimisation and continuous improvement have embraced business intelligence for the purpose of decision-making as data are becoming more readily available. Companies recognise the need for, and importance of, user experience when interacting with the dashboards to assist users with decision-making and to enable users to take timely evidence-based actions. It is no longer sufficient only to support users in achieving their goals by employing basic usability principles; digital products need to provide an overall positive user experience to be considered successful. Numerous frameworks have been developed for the user experience of digital systems. However, with the literature emphasizing the importance of context, those user experience frameworks cannot be transferred to business intelligence. This dearth of evidence-based business intelligence specific user experience frameworks comprises the rationale for this study. The research methodology is comprised of mixed methods as this methodological paradigm aligns best with the pragmatist and appreciative inquiry research approach. The research design framework draws on the Affordance Theory, Logic Model and Agile software development approaches. The context of the study is that of software development in an agile environment in South Africa. The theoretical research contribution comprises of the validated conceptual literature-based framework for the user experience of business intelligence dashboards and the compilation and use of an original research design framework that was utilised to guide the research. This research study also has value for industry in that it has produced a novel validated practitioners’ framework that can be used in practice to identify user experience shortcomings and highlight opportunities for improvement.School of ComputingPh. D. (Information Systems