On Evaluating Commercial Cloud Services: A Systematic Review

Background: Cloud Computing is increasingly booming in industry with many competing providers and services. Accordingly, evaluation of commercial Cloud services is necessary. However, the existing evaluation studies are relatively chaotic. There exists tremendous confusion and gap between practices and theory about Cloud services evaluation. Aim: To facilitate relieving the aforementioned chaos, this work aims to synthesize the existing evaluation implementations to outline the state-of-the-practice and also identify research opportunities in Cloud services evaluation. Method: Based on a conceptual evaluation model comprising six steps, the Systematic Literature Review (SLR) method was employed to collect relevant evidence to investigate the Cloud services evaluation step by step. Results: This SLR identified 82 relevant evaluation studies. The overall data collected from these studies essentially represent the current practical landscape of implementing Cloud services evaluation, and in turn can be reused to facilitate future evaluation work. Conclusions: Evaluation of commercial Cloud services has become a world-wide research topic. Some of the findings of this SLR identify several research gaps in the area of Cloud services evaluation (e.g., the Elasticity and Security evaluation of commercial Cloud services could be a long-term challenge), while some other findings suggest the trend of applying commercial Cloud services (e.g., compared with PaaS, IaaS seems more suitable for customers and is particularly important in industry). This SLR study itself also confirms some previous experiences and reveals new Evidence-Based Software Engineering (EBSE) lessons

Evidence in Requirements Engineering: A Systematic Literature Review Protocol

Requirements Engineering (RE) is recognized as one of the critical phases in software development. RE has its own journals and conferences where lots of work has been published. As the area is maturing, increasingly large numbers of empirically supported studies have been reported in RE. There is a need to synthesize evidence based RE literature. We plan to systematically investigate evidence based RE studies to see and report state of the art in evidence based RE reported research. This paper aims at providing a systematic literature review (SLR) protocol to describe a process for synthesizing the empirically supported work in the area of RE that will eventually present a state of the art of the field. This SLR intends to not only summarize the empirical data regarding RE but will also be helpful for various practitioners in this field to find out areas of RE rich in terms of tools, techniques, frameworks, models and guidelines to aid in their work. It will also facilitate RE researchers to identify knowledge gaps to recognize needs and chances for future research directions in this field

Understanding user experience of mobile video: Framework, measurement, and optimization

Since users have become the focus of product/service design in last decade, the term User eXperience (UX) has been frequently used in the field of Human-Computer-Interaction (HCI). Research on UX facilitates a better understanding of the various aspects of the user’s interaction with the product or service. Mobile video, as a new and promising service and research field, has attracted great attention. Due to the significance of UX in the success of mobile video (Jordan, 2002), many researchers have centered on this area, examining users’ expectations, motivations, requirements, and usage context. As a result, many influencing factors have been explored (Buchinger, Kriglstein, Brandt & Hlavacs, 2011; Buchinger, Kriglstein & Hlavacs, 2009). However, a general framework for specific mobile video service is lacking for structuring such a great number of factors. To measure user experience of multimedia services such as mobile video, quality of experience (QoE) has recently become a prominent concept. In contrast to the traditionally used concept quality of service (QoS), QoE not only involves objectively measuring the delivered service but also takes into account user’s needs and desires when using the service, emphasizing the user’s overall acceptability on the service. Many QoE metrics are able to estimate the user perceived quality or acceptability of mobile video, but may be not enough accurate for the overall UX prediction due to the complexity of UX. Only a few frameworks of QoE have addressed more aspects of UX for mobile multimedia applications but need be transformed into practical measures. The challenge of optimizing UX remains adaptations to the resource constrains (e.g., network conditions, mobile device capabilities, and heterogeneous usage contexts) as well as meeting complicated user requirements (e.g., usage purposes and personal preferences). In this chapter, we investigate the existing important UX frameworks, compare their similarities and discuss some important features that fit in the mobile video service. Based on the previous research, we propose a simple UX framework for mobile video application by mapping a variety of influencing factors of UX upon a typical mobile video delivery system. Each component and its factors are explored with comprehensive literature reviews. The proposed framework may benefit in user-centred design of mobile video through taking a complete consideration of UX influences and in improvement of mobile videoservice quality by adjusting the values of certain factors to produce a positive user experience. It may also facilitate relative research in the way of locating important issues to study, clarifying research scopes, and setting up proper study procedures. We then review a great deal of research on UX measurement, including QoE metrics and QoE frameworks of mobile multimedia. Finally, we discuss how to achieve an optimal quality of user experience by focusing on the issues of various aspects of UX of mobile video. In the conclusion, we suggest some open issues for future study

Real-time human ambulation, activity, and physiological monitoring:taxonomy of issues, techniques, applications, challenges and limitations

Automated methods of real-time, unobtrusive, human ambulation, activity, and wellness monitoring and data analysis using various algorithmic techniques have been subjects of intense research. The general aim is to devise effective means of addressing the demands of assisted living, rehabilitation, and clinical observation and assessment through sensor-based monitoring. The research studies have resulted in a large amount of literature. This paper presents a holistic articulation of the research studies and offers comprehensive insights along four main axes: distribution of existing studies; monitoring device framework and sensor types; data collection, processing and analysis; and applications, limitations and challenges. The aim is to present a systematic and most complete study of literature in the area in order to identify research gaps and prioritize future research directions

Openness Underpins Evidence Based Practice in Information Systems

This paper summarises research in progress on developing an evidence based approach in information systems (EBIS). EBIS offers a solution to information systems failures. Openness lies at the heart of EBIS, with open-access publishing and a culture of openness to new ways of working between IS academics and practitioners in the production of the evidence base. The authors have published an EBIS agenda and are developing a roadmap to implement this agenda. Their research activities are aimed at contributing to this roadmap and comprise: (i) setting up the open-access, peer-reviewed ‘Evidence Based Information Systems Journal’ which provides a vehicle to explore open access publishing and a research impact case study; (ii) creating an EBP culture, through establishing research networks; (iii) investigating IS practitioners’ use of resources; (iv) developing a model-driven approach for the analysis and synthesis of qualitative research within systematic literature reviews (SLRs) and using this to conduct an SLR on telehealthcare innovations. The ‘Evidence Based Information Systems journal’ lies at the heart of this research, providing the evidence repository with clear findings to take away and use, a forum to discuss and reflect on the evidence, and thus contributing to creating the paradigm shift to EBIS

EVIDENCE-BASED INFORMATION SYSTEMS: A DECADE LATER

The “evidence-based practice” paradigm was proposed to IS researchers a decade ago. Since then evidence-based practice has become established across a range of disciplines, but it has received relatively little attention in IS. This paper explains the idea of evidence-based practice and reviews the related work found in the IS research literature. Some possible reasons for the lack of widespread adoption in IS are suggested. Systematic literature reviews (SLRs), a key research method in evidence-based practice, are explained. Recent developments in SLRs are discussed, which enable a richer and more nuanced approach to understanding information systems than found in conventional SLRs. It is proposed that these developments now make SLRs more suitable for synthesising empirical studies in IS. Greater use of SLRs by IS researchers would enable us to develop a cumulative knowledgebase of use to both researchers and practitioners

Software product line testing - a systematic mapping study

Context: Software product lines (SPL) are used in industry to achieve more efficient software development. However, the testing side of SPL is underdeveloped. Objective: This study aims at surveying existing research on SPL testing in order to identify useful approaches and needs for future research. Method: A systematic mapping study is launched to find as much literature as possible, and the 64 papers found are classified with respect to focus, research type and contribution type. Results: A majority of the papers are of proposal research types (64 %). System testing is the largest group with respect to research focus (40%), followed by management (23%). Method contributions are in majority. Conclusions: More validation and evaluation research is needed to provide a better foundation for SPL testing

A systematic mapping study on integration proposals of the personas technique in agile methodologies

Agile development processes are increasing their consideration of usability by integrating various user‐centered design techniques throughout development. One such technique is Personas, which proposes the creation of fictitious users with real preferences to drive application design. Since applying this technique conflicts with the time constraints of agile development, Personas has been adapted over the years. Our objective is to determine the adoption level and type of integration, as well as to propose improvements to the Personas technique for agile development. A systematic mapping study was performed, retrieving 28 articles grouped by agile methodology type. We found some common integration strategies regardless of the specific agile approach, along with some frequent problems, mainly related to Persona modelling and context representation. Based on these limitations, we propose an adaptation to the technique in order to reduce the creation time for a preliminary persona. The number of publications dealing with Personas and agile development is increasing, which reveals a growing interest in the application of this technique to develop usable agile softwareThis research was funded by the Spanish Ministry of Science, Innovation and Universities research grant PGC2018-097265-B-I00, MASSIVE project (RTI2018-095255-B-I00) and by EIT-Health, grant number 19091 (POSITIVE project). This research was also supported by the Madrid Region R&D programme (project FORTE, P2018/TCS-4314