Framework for evaluating and selecting mobile-learning applications using multi criteria decision making techniques

The use of mobile learning (m-learning) applications in education has increased dramatically in recent years. M-learning applications are installed by users through a variety of mobile device distribution platforms. For a wide audience to accept them, these applications must be stable and of high quality. The decision to purchase m-learning applications needs systematic guidelines so that the appropriate one can be selected to provide a viable and effective solution to educational organizations. Usability in m-learning applications has been studied as a non-functional problem in several previous studies. In reality, Saudi tertiary institutions still lack a systematic, efficient, and well-defined framework for evaluating and selecting m-learning applications due to the lack of reliable m-learning application selection methods. Therefore, this study addresses this gap by proposing a framework to support and improve m-learning applications evaluation and selection process named as Mobile-Learning Application Evaluation and Selection Framework (MLA-ESF). MLA-ESF supports evaluation and selection of m-learning applications and integration of functional and non-functional requirements as well as addresses mismatch problems. In addition, the MLA-ESF is developed to assist and guide developers and educational organizations in selecting the required m-learning application in a more systematic and repeatable manner. Moreover, the MLA-ESF framework provides a guideline for future theoretical research, as well as being a practical and usable tool in real contexts. The study is conducted in four main phases: a survey and interview of decision-makers and users to identify the evaluation criteria, development of the framework based on the Evaluation Theory, development of a new decision-making technique by integrating Fuzzy Analytical Hierarchy Process (FAHP), Technique for Order of Preference by Similarity to Ideal Solution (TOPSIS), and GAP Analysis (GA) to handle user requirements mismatches, and validation of the applicability and reliability of MLAESF using experts review, case study and yardstick validation. The study shows that the evaluated aspects of MLA-ESF namely, inputs, actions, outcomes, are feasible and demonstrate their potential and applicability to be applied in the real environment as 75% of the experts found it as useful, 66.7% find it easy to implement, and 75% find the techniques are adequate and sufficient

A framework for cots software evaluation and selection for COTS mismatches handling and non-functional requirements

The decision to purchase Commercial Off-The-Shelf (COTS) software needs systematic guidelines so that the appropriate COTS software can be selected in order to provide a viable and effective solution to the organizations. However, the existing COTS software evaluation and selection frameworks focus more on functional aspects and do not give adequate attention to accommodate the mismatch between user requirements and COTS software specification, and also integration with non functional requirements of COTS software. Studies have identified that these two criteria are important in COTS software evaluation and selection. Therefore, this study aims to develop a new framework of COTS software evaluation and selection that focuses on handling COTS software mismatches and integrating the nonfunctional requirements. The study is conducted using mixed-mode methodology which involves survey and interview. The study is conducted in four main phases: a survey and interview of 63 organizations to identify COTS software evaluation criteria, development of COTS software evaluation and selection framework using Evaluation Theory, development of a new decision making technique by integrating Analytical Hierarchy Process and Gap Analysis to handle COTS software mismatches, and validation of the practicality and reliability of the proposed COTS software Evaluation and Selection Framework (COTS-ESF) using experts’ review, case studies and yardstick validation. This study has developed the COTS-ESF which consists of five categories of evaluation criteria: Quality, Domain, Architecture, Operational Environment and Vendor Reputation. It also provides a decision making technique and a complete process for performing the evaluation and selection of COTS software. The result of this study shows that the evaluated aspects of the framework are feasible and demonstrate their potential and practicality to be applied in the real environment. The contribution of this study straddles both the research and practical perspectives of software evaluation by improving decision making and providing a systematic guidelines for handling issue in purchasing viable COTS software