The Development and Evaluation of E-Management Platform for Patient-reported Outcomes Research Based on Business Process Improvement Theory

Background Patient-reported outcomes (PROs) refer to the health-related information coming directly from patients without any explanation by the doctors or others. It is of great significance for patients to participate in medical decision-making, have a higher quality of care，and have a better medical outcome. Therefore, patient-reported outcomes measurements (PROMs) have important clinical value. Patient-reported Outcomes Measurement Information System (PROMIS) is one of the most accurate PROMs led by NIH and focuses on health-related big data's scientific characteristics. PROMIS advocates cross-population, cross-region, and cross-disease studies and is widely recognized among the research community. PROMIS Health Organization (PHO) now promotes the global implementation of PROMIS by setting up PROMIS National Center (PNC) in various countries, and PNCs organize the research cooperation with many research groups to promote PROMIS. PROMIS China Center is one of the PNCs that takes charge of the PROs cooperative research with many research teams to promote PROMIS in China. Due to the diverse tasks, complex procedures, and low efficiency of the current cooperative management mode of PROMIS China Center, it cannot meet the efficient research management requirements. Therefore, it is necessary to analyze the needs, improve the process, and build an E-management platform under the guidance of business improvement theory to improve the management efficiency of the patient-reported outcome research and eventually promote PROMIS in China. Objectives The general purpose of this research is to improve the management process of patient-reported outcomes research based on the business process improvement theory and develop an E-Management platform as well as evaluate the usability of the E-management platform. Purpose one: The need analysis of the E-management platform for the patient-reported outcomes research; Purpose two: The framework construction of the E-management platform for patient-reported outcomes research; Purpose three: The development of the E-management platform for patient-reported outcomes research; Purpose four: The usability evaluation of the E-Management Platform for Patient-reported outcomes research. Methods The study was comprised of 4 parts： Part1: The need analysis of the E-management platform for the patient-reported outcomes research Based on business improvement theory (ESIA theory), the researcher conducted the semi-structured qualitative interviews, including five rounds of focus group interviews with 5 management staffs of the PROMIS China Center and one-on-one interviews with 11 cooperative team members to know the current management process of patient-reported outcomes research, the problems of current management process, and the need for the E-management platform. Part2: The framework construction of the E-management platform for patient-reported outcomes research The researcher held a brainstorming discussion with 13 management staffs to improve the current management process based on business improvement theory; Meanwhile, the researcher accomplished the literature study about the patient-reported outcomes research E-management platforms of other countries and analyzed the framework; Then, the researcher combined the results of need analysis, brainstorming, and literature study to draw the preliminary framework of the patient-reported outcomes research E-management platform; Finally, the researcher held an expert group meeting with 7 experts from the field of both nursing research and software engineering to revise the preliminary framework and make a final framework. Part3: The development of the E-management platform for patient-reported outcomes research The researchers organized and coordinated close cooperation among experts in different fields. Based on the final framework of the E-management platform formed in the early stage, the researchers adopted the human-centered concept and the agile development method to develop the E-management platform with 3 software engineers. Part4: The usability evaluation of the E-Management Platform for Patient-reported outcomes research The researcher implemented the formative usability evaluation during the process of development to find the problems about the interface, like font and module; Then the researcher implemented the summarized usability evaluation by task analysis and semi-structured interviews with 15 users，including 5 management staffs of the PROMIS China Center and 10 cooperative team members. The evaluation outcomes were reported after data analysis. Results Part1: The need analysis of the E-management platform for the patient-reported outcomes research The qualitative research results show the: ① PROMIS translation, validation, clinical application management processes, and the researcher presented them in text and flow chart. ② The current process's problems, like the current management process is tedious; the current management process involves too much paper materials; it is difficult for management staffs to supervise the process. ③ the need for the E-management: all the participants said they need the E-management platform and expressed their specific needs. Part2: The framework construction of the E-management platform for patient-reported outcomes research The brainstorming efficiently improved the management process, and the improved process was shown with text and flowchart; The literature study presented the framework of the Netherlands PROMIS E-management platform; the researcher drew a preliminary framework of the E-management platform under the guidance of need analysis, improved management process, and Netherlands PROMIS E-management platform, which contains 7 first-level modules, and 5 second-level modules. After the expert group meeting, the final framework was constructed, which includes 6 first-level modules: "Home", " Introduction ", "PROMIS Measurement", "News Center", "Resources", and " Research Cooperation", 16 second-level modules, and the content covered by the second-level modules. Part3: The development of the E-management platform for patient-reported outcomes research The researchers demonstrated the preliminary framework of the E-management platform to the software engineers, and we worked jointly to complete the development of the E-management platform after the steps of requirement clarification, design, development, testing, and release. Part4: The usability evaluation of the E-Management Platform for Patient-reported outcomes research During the formative usability evaluation, the researcher found many problems, and they were all solved by software engineers; During the summarized usability evaluation, we demonstrated the platform in terms of effectiveness, efficiency, and satisfaction. Based on the scores of the usability evaluation questionnaire after the test: system usefulness is 5.2 (from management staffs) and 5.8 (from cooperative team members), information quality is 6.8 (from management staffs) and 6.0 (from cooperative team members), interface quality is 5.4 (from management staffs) and 5.9 (from cooperative team members), and overall evaluation is 6.0 (from management staffs) and 6.2 (from cooperative team members). Through qualitative interviews, we understand the user experience of the platform. The interview data shows two topics related to usability. By combining quantitative and qualitative outcomes, the researcher thinks that the E-management platform's usability is generally good, but there are still some shortcomings. Conclusions Both the management staffs and cooperative team members of the PROMIS China Center have a high level of need for the E-management platform. The final framework based on need analysis and process improvement is good guidance for developing the E-management platform. The usability evaluation shows that the E-management platform has a good usability and can help manage the patient-reported outcomes cooperative research. It will be able to enhance the management efficiency of patients-reported outcomes research extensively and accelerate the rapid promotion of PROMIS in China

Development and Usability Evaluation of a Nursing Graduate Information Management System (GSMIS) Based on User Experience

[Research Background] Recent years, the rapid development of information technology in colleges and universities and the continuous expansion of graduate students have not only increased the pressure of graduate student management in the School of Nursing of Fudan University, but also accelerated the pace of information technology construction in the college, and the School of Nursing urgently needs to use the existing good platform to realize the information technology of nursing graduate student management in combination with the actual situation of the college, so as to realize the communication between graduate student supervisors, graduate students and management, and also to build an objective and scientific clinical competence assessment system in line with It is also important to provide material for the construction of an objective and scientific clinical competence assessment and evaluation system that is in line with the postgraduate nursing education, which is important to achieve the cultivation goals and improve the quality of nursing graduate training. [Research Purpose] Based on the theory of user experience and the concept of "User-Centered Design", this project develops the information management system for graduate students in the School of Nursing by studying the actual needs of teachers and graduate students in the School of Nursing, which can be divided into the following contents: (1) Construction of the framework content of a Nursing GSMIS (2) Development and testing of a Nursing GSMIS (3) Usability evaluation of the a Nursing GSMIS [Research Methods] 1. Demand analysis of the Nursing GSMIS (1) Preliminary framework: Using target sampling method and maximum variation method, semi- structured interviews were conducted with both faculty and student users of the School of Nursing to build the preliminary framework of the system. (2) Final framework: Expert focus group interviews were used to revise and adjust the required content such as interface design, business functions and performance, and to determine the final framework of the system. 2. Development and testing of Nursing GSMIS The “Rapid Prototype Interaction” model was used to develop the system. After the prototype was developed, the Shanghai Institute of Comprehensive Application of Network Technology was contacted for professional system testing, and then the system was modified and adjusted until it met the research requirements. 3. Usability evaluation of Nursing GSMIS A mixed evaluation method of usability test, questionnaire and qualitative interview was used to conduct a more comprehensive and objective usability evaluation of the GSMIS successfully developed in the early stage. It was used to understand the users' feelings and experiences of the system, analyze the advantages and shortcomings of the system, and explore its actual usability.[Research Results] 1. Demand analysis of Nursing GSMIS (1) Preliminary framework of the system: The faculty and student users interviewed expressed that information management is an important step in the progress of the college and the nursing discipline, and expressed strong support and need for the construction of our information system. Based on the literature review and the needs of faculty and students, the framework was initially formulated as four modules: basic information of admission, cultivation process, research results, and clinical cases. (2)The final framework of Nursing GSMIS: It is built from modules and functions, performance, interface, text, color, etc.; Modules include: basic information module for admission, training process module, scientific research achievement module, clinical case library module, employment information module, graduation tracking module; functions include password setting and retrieval, user log-in, user management, data maintenance, custom query, audit and message notification; Performance can be summarized as follows: data structure is clear, complete, with real-time, expansion, operation and stability; Interface Design is friendly and beautiful, simple and easy to use; Layout should highlight the key points; text design is easy to recognize and read, color design reflects the connotation and characteristics of the college. The performance can be summarized as: clear data structure, complete, real-time, scalability, operability and stability; friendly and beautiful interface design, simple and easy to use, layout to highlight the key points; text design is easy to identify, easy to read, color design reflects the connotation and characteristics of the college. 2. Development and testing of Nursing GSMIS Based on the requirements, the engineers used the “Rapid Prototype Interaction Model” to develop the system. After the prototype was developed, the Shanghai Institute of Comprehensive Application of Network Technology was contacted for professional system testing, and then after many rounds of software discussions and corrections between the software engineers and researchers, the GSMIS finally had the registration and log-in page, the informed consent notification page, the information notification page and six main modules. 3. Usability evaluation of Nursing GSMIS (1) Usability Test: Based on the usability test theory, the target sampling method was used, and five graduate students users were selected to participate in the test is sufficient, and the results are as follows. 1) Quantitative results: The effectiveness was mostly at 100%, and the efficiency was 56.2min/s, 72.2min/s, 53min/s, 225.2min/s, 75.8min/s, 33min/s, and 5.5min/s; The overall score of the post-scene questionnaire (ASQ) was 1.24. The evaluation indexes all indicated that the users rated the system highly. 2) Qualitative results: The test subjects all easily completed and highly praised the system during the test, indicating that the operation was not difficult and satisfied the interaction needs of users. (2) SUS Questionnaire: A total of 40 users completed the SUS using the convenience sampling method and the maximum variance method, and the Mean±SD was 70.23±7.7. Among them, the mean score of the Usability sub-scale was 71.59±8.93 , and the mean score of the ease of Learning sub-scale was 64.77±7.53, with scores ranging from 60 to 70. (3) Qualitative interviews: Using target sampling method and maximum variation method, a total of 10 people were interviewed, including 5 students and 5 staff users, and the number of interviews was 3, with no repeated interviews or secondary interviews. The maximum interview time was 46 minutes, the minimum was 25 minutes, and the average was 35.5 minutes. The results can be summarized into three aspects: general evaluation, content and function evaluation, and shortcomings and improvement. This study combines qualitative and quantitative results, reflecting the good usability of the system, but there are also shortcomings and improvements. [Conclusion] This study used various research methods such as literature review, qualitative interview, and focus group interview to construct a system framework based on user experience, completed the development of the graduate student information management system in the School of Nursing, and used mixed methods such as out-of-sound thinking method, usability testing method, questionnaire survey method, and qualitative interview to evaluate the usability of the system, and the results showed that the system has high acceptance, ease of use, and usability

PENGEMBANGAN EXCITED MATH GAMES UNTUK MENINGKATKAN PEMAHAMAN KONSEP HIMPUNAN SISWA SMP

The aim of the research is to develop interactive learning media Excited Math Games on set material for class VII junior high school students. The research method used is development research referring to the ADDIE model. The research instruments were media development sheets and student learning outcomes tests. The results of the research are based on the design carried out (1) analysis: needs analysis which shows that the use of learning media is still limited, and students have not been actively involved in the learning process, technology analysis, a combination of Power Point and iSping Suite 10 used as the main software, and curriculum analysis according to with the 2013 curriculum; (2) design: collecting references, designing material content according to aspects of understanding concepts, preparing stoyboards and flowcharts to make it easier to create learning media; (3) development; creating learning media, expert validation to determine the validity of learning media, and revising phase I products based on validator input; (4) implementation: trial to obtain data on students' level of understanding of concepts after applying learning media; (5) evaluation: analyzing student learning test results to determine the level of understanding of concepts. The quality of the learning media developed meets the very valid criteria based on the assessment of media experts and material experts with scores of 3.8125 and 3.65 respectively. Meanwhile, the level of understanding of students' concepts after applying the Excited Math Games learning media obtained very high criteria based on student learning outcomes tests with a completeness percentage of 86.67% and an average score of 88 or above the students' completeness score, namely > 70

Development and evaluation of a software-mediated process assessment approach in IT service management

To operate in a highly competitive business environment, organisations require the support of continually improving IT services. The dominant academic literature on ITService Management (ITSM) focuses on the measurement of the outcome of ITSM implementation. Consequently, there is limited research on the measurement of ITSM processes. The ITSM industry has defined a number of processes as best practices in the IT Infrastructure Library (ITIL®) framework and the international standard forITSM, ISO/IEC 20000. However, there is a lack of a transparent and efficient process assessment method to improve ITSM processes. This research aims to address the dual problems of the lack of transparency and the need for efficiency in ITSM process assessment. Using the design science research methodology, an iterative design process was followed to develop a research artefact in the form of a method: the Software-Mediated Process Assessment (SMPA) approach that enables researchers and practitioners to assess the ITSM processes in a transparent and efficient way. The four phases in theSMPA approach include preparation for the assessment; online survey to collect assessment data; measurement of process capability; and reporting of process improvement recommendations. The international standard for process assessment ISO/IEC 15504 and associated assessment models provided support for a transparent method. A Decision Support System (DSS) was implemented to demonstrate efficient use of the SMPA approach. Using a theoretically-grounded fit profile based on the Task-Technology Fit theory, the international standards and DSS technology were implemented in the SMPA approach to address the research problem. The DSS platform was provided by an industry partner Assessment Portal Pty Ltd. that specialises in online assessment services. Two case study organisations provided test sites for the evaluation of the SMPA approach. The two organisations are the Queensland Government’s primary IT service provider, CITEC and the IT service department of an Australian local government authority, Toowoomba Regional Council. Using the quality models from the international standard for software quality evaluation ISO/IEC 25010, the usability and ii outcomes of the SMPA approach were evaluated. Evidence from the case study evaluations indicated that the SMPA approach is usable for ITSM process assessment in order to support decision-making on process improvements. Further discussions of the research findings provided design knowledge that included the emergence of the concept of virtualisability in ITSM process assessments and a proposal of a hybrid ITSM process assessment method. Moreover, iterations ofself-assessments of ITSM processes using the SMPA approach may facilitate continual service improvement. Based on the design knowledge obtained, the contributions of this research to theory and practice were articulated. The SMPA approach extends prior guidelines on ITSM process assessment by providing a fine-grained method to assess ITSM processes. The SMPA approach clarifies the impact of software mediation to support transparency and efficiency in the way process assessments are conducted. This research also demonstrates how the SMPA approach is applied in practice by enabling IT organisations to self-assess the capability of their ITSM processes. Upon reflection, the design science research method was found to be highly suitable to develop an artefact to solve a research problem and to evaluate the practical utility of the artefact. The SMPA approach is a research artefact that is implemented as a DSS; hence it is readily accessible to practitioners. The focus on practical utility provides researchers with results that are more readily endorsed, thus maximising the impact of the research findings in practice

A systematic review of quality attributes and measures for software product lines

[EN] It is widely accepted that software measures provide an appropriate mechanism for understanding, monitoring, controlling, and predicting the quality of software development projects. In software product lines (SPL), quality is even more important than in a single software product since, owing to systematic reuse, a fault or an inadequate design decision could be propagated to several products in the family. Over the last few years, a great number of quality attributes and measures for assessing the quality of SPL have been reported in literature. However, no studies summarizing the current knowledge about them exist. This paper presents a systematic literature review with the objective of identifying and interpreting all the available studies from 1996 to 2010 that present quality attributes and/or measures for SPL. These attributes and measures have been classified using a set of criteria that includes the life cycle phase in which the measures are applied; the corresponding quality characteristics; their support for specific SPL characteristics (e. g., variability, compositionality); the procedure used to validate the measures, etc. We found 165 measures related to 97 different quality attributes. The results of the review indicated that 92% of the measures evaluate attributes that are related to maintainability. In addition, 67% of the measures are used during the design phase of Domain Engineering, and 56% are applied to evaluate the product line architecture. However, only 25% of them have been empirically validated. In conclusion, the results provide a global vision of the state of the research within this area in order to help researchers in detecting weaknesses, directing research efforts, and identifying new research lines. In particular, there is a need for new measures with which to evaluate both the quality of the artifacts produced during the entire SPL life cycle and other quality characteristics. There is also a need for more validation (both theoretical and empirical) of existing measures. In addition, our results may be useful as a reference guide for practitioners to assist them in the selection or the adaptation of existing measures for evaluating their software product lines. © 2011 Springer Science+Business Media, LLC.This research has been funded by the Spanish Ministry of Science and Innovation under the MULTIPLE (Multimodeling Approach For Quality-Aware Software Product Lines) project with ref. TIN2009-13838.Montagud Gregori, S.; Abrahao Gonzales, SM.; Insfrán Pelozo, CE. (2012). A systematic review of quality attributes and measures for software product lines. Software Quality Journal. 20(3-4):425-486. https://doi.org/10.1007/s11219-011-9146-7S425486203-4Abdelmoez, W., Nassar, D. M., Shereschevsky, M., Gradetsky, N., Gunnalan, R., Ammar, H. H., et al. (2004). Error propagation in software architectures. In 10th international symposium on software metrics (METRICS), Chicago, Illinois, USA.Ajila, S. A., & Dumitrescu, R. T. (2007). Experimental use of code delta, code churn, and rate of change to understand software product line evolution. Journal of Systems and Software, 80, 74–91.Aldekoa, G., Trujillo, S., Sagardui, G., & Díaz, O. (2006). Experience measuring maintainability in software product lines. In XV Jornadas de Ingeniería del Software y Bases de Datos (JISBD). Barcelona.Aldekoa, G., Trujillo, S., Sagardui, G., & Díaz, O. (2008). Quantifying maintanibility in feature oriented product lines, Athens, Greece, pp. 243–247.Alves de Oliveira Junior, E., Gimenes, I. M. S., & Maldonado, J. C. (2008). A metric suite to support software product line architecture evaluation. In XXXIV Conferencia Latinamericana de Informática (CLEI), Santa Fé, Argentina, pp. 489–498.Alves, V., Niu, N., Alves, C., & Valença, G. (2010). Requirements engineering for software product lines: A systematic literature review. Information & Software Technology, 52(8), 806–820.Bosch, J. (2000). Design and use of software architectures: Adopting and evolving a product line approach. USA: ACM Press/Addison-Wesley Publishing Co.Briand, L. C., Differing, C. M., & Rombach, D. (1996a). Practical guidelines for measurement-based process improvement. Software Process-Improvement and Practice, 2, 253–280.Briand, L. C., Morasca, S., & Basili, V. R. (1996b). Property based software engineering measurement. IEEE Transactions on Software Eng., 22(1), 68–86.Calero, C., Ruiz, J., & Piattini, M. (2005). Classifying web metrics using the web quality model. Online Information Review, 29(3): 227–248.Chen, L., Ali Babar, M., & Ali, N. (2009). Variability management in software product lines: A systematic review. In 13th international software product lines conferences (SPLC), San Francisco, USA.Clements, P., & Northrop, L. (2002). Software product lines. 2003. Software product lines practices and patterns. Boston, MA: Addison-Wesley.Crnkovic, I., & Larsson, M. (2004). Classification of quality attributes for predictability in component-based systems. Journal of Econometrics, pp. 231–250.Conference Rankings of Computing Research and Education Association of Australasia (CORE). (2010). Available in http://core.edu.au/index.php/categories/conference%20rankings/1 .Davis, A., Dieste, Ó., Hickey, A., Juristo, N., & Moreno, A. M. (2006). Effectiveness of requirements elicitation techniques: Empirical results derived from a systematic review. In 14th IEEE international conference requirements engineering, pp. 179–188.de Souza Filho, E. D., de Oliveira Cavalcanti, R., Neiva, D. F. S., Oliveira, T. H. B., Barachisio Lisboa, L., de Almeida E. S., & de Lemos Meira, S. R. (2008). Evaluating domain design approaches using systematic review. In 2nd European conference on software architecture, Cyprus, pp. 50–65.Ejiogu, L. (1991). Software engineering with formal metrics. QED Publishing.Engström, E., & Runeson, P. (2011). Software product line testing—A systematic mapping study. Information & Software Technology, 53(1), 2–13.Etxeberria, L., Sagarui, G., & Belategi, L. (2008). Quality aware software product line engineering. Journal of the Brazilian Computer Society, 14(1), Campinas Mar.Ganesan, D., Knodel, J., Kolb, R., Haury, U., & Meier, G. (2007). Comparing costs and benefits of different test strategies for a software product line: A study from Testo AG. In 11th international software product line conference, Kyoto, Japan, pp. 74–83, September 2007.Gómez, O., Oktaba, H., Piattini, M., & García, F. (2006). A systematic review measurement in software engineering: State-of-the-art in measures. In First international conference on software and data technologies (ICSOFT), Setúbal, Portugal, pp. 11–14.IEEE standard for a software quality metrics methodology, IEEE Std 1061-1998, 1998.Inoki, M., & Fukazawa, Y. (2007). Software product line evolution method based on Kaizen approach. In 22nd annual ACM symposium on applied computing, Korea.Insfran, E., & Fernandez, A. (2008). A systematic review of usability evaluation in Web development. 2nd international workshop on web usability and accessibility (IWWUA’08), New Zealand, LNCS 5176, Springer, pp. 81–91.ISO/IEC 25010. (2008). Systems and software engineering. Systems and software Quality Requirements and Evaluation (SQuaRE). System and software quality models.ISO/IEC 9126. (2000). Software engineering. Product Quality.Johansson, E., & Höst, R. (2002). Tracking degradation in software product lines through measurement of design rule violations. In 14th International conference on software engineering and knowledge engineering, Ischia, Italy, pp. 249–254.Journal Citation Reports of Thomson Reuters. (2010). Available in http://thomsonreuters.com/products_services/science/science_products/a-z/journal_citation_reports/ .Khurum, M., & Gorschek, T. (2009). A systematic review of domain analysis solutions for product lines. The Journal of Systems and Software.Kim, T., Ko, I. Y., Kang, S. W., & Lee, D. H. (2008). Extending ATAM to assess product line architecture. In 8th IEEE international conference on computer and information technology, pp. 790–797.Kitchenham, B. (2007). Guidelines for performing systematic literature reviews in software engineering. Version 2.3, EBSE Technical Report, Keele University, UK.Kitchenham, B., Pfleeger, S., & Fenton, N. (1995). Towards a framework for software measurement validation. IEEE Transactions on Software Engineering, 21(12).Landis, J. R., & Koch, G. G. (1977). The measurement of observer agreement for categorical data. Biometrics, 33, 159–174.Mendes, E. (2005). A systematic review of Web engineering research. International symposium on empirical software engineering. Noosa Heads, Australia.Meyer, M. H., & Dalal, D. (2002). Managing platform architectures and manufacturing processes for non assembled products. Journal of Product Innovation Management, 19(4), 277–293.Montagud, S., & Abrahão, S. (2009). Gathering Current knowledge about quality evaluation in software product lines. In 13th international software product lines conferences (SPLC), San Francisco, USA.Montagud, S., & Abrahão, S. (2009). A SQuaRE-bassed quality evaluation method for software product lines. Master’s thesis, December 2009 (in Spanish).Needham, D., & Jones, S. (2006). A software fault tree metric. In 22nd international conference on software maintenance (ICSM), Philadelphia, Pennsylvania, USA.Niemelä, E., & Immonen, A. (2007). Capturing quality requirements of product family architecture. Information and Software Technology, 49(11–12), 1107–1120.Odia, O. E. (2007). Testing in software product lines. Master Thesis Software Engineering of School of Engineering, Bleking Institute of Technology. Thesis no. MSE-2007:16, Sweden.Olumofin, F. G., & Mišić, V. B. (2007). A holistic architecture assessment method for software product lines. Information and Software Technology, 49, 309–323.Pérez Lamancha, B., Polo Usaola, M., & Piattini Velthius, M. (2009). Software product line testing—a systematic review. ICSOFT, (1), 23–30.Poels, G., & Dedene, G. (2000). Distance-based software measurement: necessary and sufficient properties for software measures. Information and Software Technology, 42(I), 35–46.Prehofer, C., van Gurp, J., & Bosch, J. (2008). Compositionality in software platforms. In Emerging methods, technologies and process management in software engineering. Wiley.Rahman, A. (2004). Metrics for the structural assessment of product line architecture. Master Thesis on Software Engineering, Thesis no. MSE-2004:24. School of Engineering, Blekinge Institute of Technology, Sweden.Sethi, K., Cai, Y., Wong, S., Garcia, A., & Sant’Anna, C. (2009). From retrospect to prospect: Assessing modularity and stability from software architecture. Joint working IEEE/IFIP conference on software architecture, 2009 & European conference on software architecture. WICSA/ECSA.Shaik, I., Abdelmoez, W,. Gunnalan, R., Shereshevsky, M., Zeid, A., Ammar, H. H., et al. (2005). Change propagation for assessing design quality of software architectures. 5th working IEEE/IFIP conference on software architecture (WICSA’05).Siegmund, N., Rosenmüller, M., Kuhlemann, M., Kästner, C., & Saake, G. (2008). Measuring non-functional properties in software product lines for product derivation. In 15th Asia-Pacific software engineering conference, Beijing, China.Sun Her, J., Hyeok Kim, J., Hun Oh, S., Yul Rhew, S., & Dong Kim, S. (2007). A framework for evaluating reusability of core asset in product line engineering. Information and Software Technology, 49, 740–760.Svahnberg, M., & Bosch, J. (2000). Evolution in software product lines. In 3rd international workshop on software architectures for products families (IWSAPF-3). Las Palmas de Gran Canaria.Van der Hoek, A., Dincel, E., & Medidović, N. (2003). Using services utilization metrics to assess the structure of product line architectures. In 9th international software metrics symposium (METRICS), Sydney, Australia.Van der Linden, F., Schmid, K., & Rommes, E. (2007). Software product lines in action. Springer.Whitmire, S. (1997). Object oriented design measurement. John Wiley & Sons.Wnuk, K., Regnell, B., & Karlsson, L. (2009). What happened to our features? Visualization and understanding of scope change dynamics in a large-scale industrial setting. In 17th IEEE international requirements engineering conference.Yoshimura, K., Ganesan, D., & Muthig, D. (2006). Assessing merge potential of existing engine control systems into a product line. In International workshop on software engineering for automative systems, Shangai, China, pp. 61–67.Zhang, T., Deng, L., Wu, J., Zhou, Q., & Ma, C. (2008). Some metrics for accessing quality of product line architecture. In International conference on computer science and software engineering (CSSE), Wuhan, China, pp. 500–503

ANALISIS BUTIR TES KOMPETENSI SISTEM REM DAN SISTEM KEMUDI MATA PELAJARAN CHASSIS PADA SMK KEAHLIAN TEKNIK KENDARAAN RINGAN

Penelitian ini dilatar belakangi belum sesuainya penilaian hasil belajar dengan instrumen tes yang dibuat oleh guru dalam mengevaluasi pembelajaran, maka tujuan penelitian ini yaitu untuk mengetahui kualitas tes pada mata pelajaran chassis (kompetensi sistem rem dan sistem kemudi) kelas XI yang dianalisis dilihat dari validitas, daya pembeda, tingkat kesukaran, dan reliabilitas. Melalui tes yang memenuhi kualitas baik diharapkan dapat mengevaluasi pembelajaran sesuai penilaian hasil belajar yang sebenarnya. Metode penelitian yang digunakan adalah metode deskriptif pendekatan kuantitatif. Penelitian ini dilakukan di SMK Negeri di Kota Bandung dengan subyek penelitian adalah siswa kelas XI TKR berjumlah 25 siswa. Instrumen penelitian yang digunakan yaitu butir tes soal pilihan ganda. Berdasarkan hasil validitas isi dari 70 soal yang dibuat, 16 soal dinyatakan tidak valid dan 54 soal dinyatakan valid. Soal yang valid dilakukan pengujian dan pengolahan data menggunakan program komputer software ANATES V4, hasil analisis diperoleh menggunakan beberapa pertimbangan yaitu dilihat dari validitas butir soal menggunakan korelasi butir soal, daya pembeda, dan tingkat kesukaran sehingga dari beberapa pertimbangan mendapatkan simpulan 24 soal yang memenuhi kualitas baik dan koefisien reliabilitas soal memiliki nilai sebesar 0,80 berada pada kategori tinggi. Hasil penelitian ini dapat dijadikan referensi dan bermanfaat dalam pembuatan instrumen tes di bidang evaluasi pembelajaran serta menjadi bahan masukkan dan pertimbangan untuk menentukan kualitas tes bagi pendidik atau guru dalam pembelajaran di sekolah dan lembaga pendidikan. Kata Kunci: Analisis, kualitas tes, soal, chassis (kompetensi sistem rem dan sistem kemudi). This present study was based on the incomplete and inappropriateness of learning outcomes using test instruments made by the teacher in the process of learning evaluation. The purpose of this study is to find out test quality on the subject of chassis (brake system and rudder system competencies) of the XI grade. This analysis focused on looking at the validity, distinguishing power, level of difficulty, and reliability. Through the test with the good quality, it was expected that it could evaluate the learning process based on the exact learning outcomes. In this carried out study, the applied method was descriptive method using quantitative approach. In addition, the research instruments used this study were test items of multiple choices. Based on the validity results of the contents out of 70 questions, it could be concluded that 16 questions were considered invalid and 54 questions were valid. The valid questions would be then tested and put into data analysis using software computer program ANATES V4. The results of analysis used several considerations viewed from the validity of question items using the correlation of the question items; namely 25 significant questions and 29 insignificant questions. Distinguishing power covered 18 questions with very invalid criteria, 11 questions with invalid criteria, 5 questions with a bit good criteria, 7 questions with good criteria, and 13 questions with very good criteria. The levels of difficulty covered; 4 questions with very difficult criteria, 7 questions with difficult criteria, 34 questions with average criteria, 4 question with easy criteria, and 5 questions with very easy criteria. Based on the aforementioned considerations, it can be concluded that 24 questions met good qualities and the question reliability coefficient reached 0,80 on the high category. The results of this present study can be used as a reference and are beneficial in the process of test instrument formulation in the field of learning evaluation and are as suggestions and considerations to determine test quality for the teachers in learning processes in schools and education institutions. Keywords: Analysis, test quality, questions, chassis (brake system and rudder system competencies

Iterative criteria-based approach to engineering the requirements of software development methodologies

Software engineering endeavours are typically based on and governed by the requirements of the target software; requirements identification is therefore an integral part of software development methodologies. Similarly, engineering a software development methodology (SDM) involves the identification of the requirements of the target methodology. Methodology engineering approaches pay special attention to this issue; however, they make little use of existing methodologies as sources of insight into methodology requirements. The authors propose an iterative method for eliciting and specifying the requirements of a SDM using existing methodologies as supplementary resources. The method is performed as the analysis phase of a methodology engineering process aimed at the ultimate design and implementation of a target methodology. An initial set of requirements is first identified through analysing the characteristics of the development situation at hand and/or via delineating the general features desirable in the target methodology. These initial requirements are used as evaluation criteria; refined through iterative application to a select set of relevant methodologies. The finalised criteria highlight the qualities that the target methodology is expected to possess, and are therefore used as a basis for de. ning the final set of requirements. In an example, the authors demonstrate how the proposed elicitation process can be used for identifying the requirements of a general object-oriented SDM. Owing to its basis in knowledge gained from existing methodologies and practices, the proposed method can help methodology engineers produce a set of requirements that is not only more complete in span, but also more concrete and rigorous

Integrating automated support for a software management cycle into the TAME system

Software managers are interested in the quantitative management of software quality, cost and progress. An integrated software management methodology, which can be applied throughout the software life cycle for any number purposes, is required. The TAME (Tailoring A Measurement Environment) methodology is based on the improvement paradigm and the goal/question/metric (GQM) paradigm. This methodology helps generate a software engineering process and measurement environment based on the project characteristics. The SQMAR (software quality measurement and assurance technology) is a software quality metric system and methodology applied to the development processes. It is based on the feed forward control principle. Quality target setting is carried out before the plan-do-check-action activities are performed. These methodologies are integrated to realize goal oriented measurement, process control and visual management. A metric setting procedure based on the GQM paradigm, a management system called the software management cycle (SMC), and its application to a case study based on NASA/SEL data are discussed. The expected effects of SMC are quality improvement, managerial cost reduction, accumulation and reuse of experience, and a highly visual management reporting system

Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration

Testing in Continuous Integration (CI) involves test case prioritization, selection, and execution at each cycle. Selecting the most promising test cases to detect bugs is hard if there are uncertainties on the impact of committed code changes or, if traceability links between code and tests are not available. This paper introduces Retecs, a new method for automatically learning test case selection and prioritization in CI with the goal to minimize the round-trip time between code commits and developer feedback on failed test cases. The Retecs method uses reinforcement learning to select and prioritize test cases according to their duration, previous last execution and failure history. In a constantly changing environment, where new test cases are created and obsolete test cases are deleted, the Retecs method learns to prioritize error-prone test cases higher under guidance of a reward function and by observing previous CI cycles. By applying Retecs on data extracted from three industrial case studies, we show for the first time that reinforcement learning enables fruitful automatic adaptive test case selection and prioritization in CI and regression testing.Comment: Spieker, H., Gotlieb, A., Marijan, D., & Mossige, M. (2017). Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration. In Proceedings of 26th International Symposium on Software Testing and Analysis (ISSTA'17) (pp. 12--22). AC