Uncertainty explicit assessment of off-the-shelf software: Selection of an optimal diverse pair

Assessment of software COTS components is an essential part of component-based software development. Sub-optimal selection of components may lead to solutions with low quality. The assessment is based on incomplete knowledge about the COTS components themselves and other aspects, which may affect the choice such as the vendor's credentials, etc. We argue in favor of assessment methods in which uncertainty is explicitly represented (`uncertainty explicit' methods) using probability distributions. We have adapted a model (developed elsewhere by Littlewood, B. et al. (2000)) for assessment of a pair of COTS components to take account of the fault (bug) logs that might be available for the COTS components being assessed. We also provide empirical data from a study we have conducted with off-the-shelf database servers, which illustrate the use of the method

Uncertainty explicit assessment of off-the-shelf software: A Bayesian approach

Assessment of software COTS components is an essential part of component-based software development. Poorly chosen components may lead to solutions of low quality and that are difficult to maintain. The assessment may be based on incomplete knowledge about the COTS component itself and other aspects (e.g. vendor’s credentials, etc.), which may affect the decision of selecting COTS component(s). We argue in favor of assessment methods in which uncertainty is explicitly represented (‘uncertainty explicit’ methods) using probability distributions. We provide details of a Bayesian model, which can be used to capture the uncertainties in the simultaneous assessment of two attributes, thus, also capturing the dependencies that might exist between them. We also provide empirical data from the use of this method for the assessment of off-the-shelf database servers which illustrate the advantages of ‘uncertainty explicit’ methods over conventional methods of COTS component assessment which assume that at the end of the assessment the values of the attributes become known with certainty

Software Test Management Tool Evaluation Framework

Tarkvara testimine on korduvalt tõestanud oma olulisust tarkvara arenduse juures viimase kümnendi jooksul. Tarkvara testimise tunnustuse kasvuga on esile kerkinud paljud elektroonilised testide haldamissüsteemid (THS). Kuigi nende hindamiseks on mitmeid võimalusi, pole me siiski leidnud selleks ühtselt aktsepteeritud meetodit. Me usume, et see on probleem, mida tuleks uurida, sest THS hindamine on sageli subjektiivne, sõltudes pigem hindaja arvamusest kui objektiivsest lähenemisest. Sama mure on ka kvaliteedikontrolli meeskondade juhtidel, kui neil palutakse hinnata, kas THS, mida neil kasutatakse, vastab ettevõtte vajadustele. Mõistmaks THS hindamise olulisust, uurisime me testimisprotsesside alast kirjandust ning analüüsisime hetkel olemasolevaid rakendusi. Seejärel kaardistasime tuvastatud testimisprotsessid ning nende väljundid. Läbi viidud analüüsi tulemusena saadud andmete põhjal koostasime veebiküsitluse ning saatsime Eesti IT-firmadele. Uuringu tulemuste põhjal koostasime me THS hindamisraamistiku, mis aitab ettevõtetel mõõta, kas ostetav THS on joondatud firma eesmärkidega, ning vähendab hinnangu andmisel subjektiivsust. Meie raamistik võimaldab testimis- ning projektijuhtidel mõista, kas nende ettevõttes kasutusel olev rakendus vastab firma ootustele. Veendumaks loodud hindamisprotsessi kasutatavuses, viisime kvaliteedikontrolli spetsialistide seas läbi täiendava uuringu, mis kinnitas meie ootusi. Meie lõputöö edasi arendamiseks on mitmeid võimalusi. Raamistikust võib luua veebirakenduse, et seda oleks kergem kasutada või laiemalt levitada. Samuti tuleks uurimust laiendada, kaasates ning analüüsides teiste euroopa riikide IT-firmade THS nõudeid. Kindlasti ei saa mainimata jätta, et THS nõudeid tuleks aja möödudes täiendada vastavalt uutele trendidele kvaliteedikontrollis. Lõpetuseks me usume, et käesoleva lõputöö tulemus, THS hindamisraamistik, on praktiline ning vajalik panus tarkvara kvaliteedikontrolli kogukonnale.Software testing has proven its value for software development increasingly over the last decade. With the recognition of the benefits of software testing, several software test management tools (TMT) have emerged on the market. Although there exist different approaches, there is no method for a systematic TMT assessment. This is a problem because to our knowledge, evaluating TMT is rather a subjective task, heavily depending on the evaluators’ opinions rather than based on the objective approach. The same problem applies when test managers are asked to evaluate whether their currently used TMT meets the company’s expectations. In order to understand the importance and neccessity of TMT evaluation we perform a literature study on software testing processes and existing TMT market studies. Then we map together the identified test activities and test artifacts. The results help us formulate and design an online questionnaire and perform a TMT survey within the Estonian IT companies. Based on the survey results, a framework for evaluating TMT software is created. Such a framework could potentially help companies to measure the TMT suitability to company’s goals and to decrease subjectivity of the TMT assessment. The framework also provides test and project managers the understanding whether their current TMTs meet the company’s expectations. We validate the framework with a case study performed among Quality Assurance specialists to collect information on the framework usability. Possibilities for future work based on this thesis are numerous. The framework can be made into an application for ease of use and wider distribution. Expanding the research onto other European countries is another viable choice. Also expanding the TMT requirements based on new trends in testing can be taken into consideration. In conclusion, we believe this thesis contributes to the testing community with a practical TMT evaluation method

A discovery and analysis of influencing factors of pair programming

The exploration into the underlying psychosocial links of pair programming. a new and unorthodox programming paradigm in which two programmers share one keyboard and monitor during real-time programming sessions, is undertaken. These complex psychosocial relationships, along with cognitive process exchanges, ultimately mold the programming output as well as determine the level of communication, satisfaction. confidence and compatibility. Laying the framework for this research, a through review of traditional and contemporary paradigms with a special focus on their limitations and a list of current software development problems are presented. Next, a detailed summary of pair programming and related agile software paradigms, such as extreme programming, which lists pair programming as one of its twelve principles, is given. From earlier pair programming studies, a number of programming benefits have been unveiled and these are listed and discussed. However, a lack of formal studies pertaining to the psychosocial aspects of pair programming exists. Given this void, a field survey is administered to a group of professional programmers and a resulting list of influencing factors on pair programming emerges. From the list, the most popular factor, personality, and two other factors, communication and gender, have been selected in order to study their impact on pair programming product outcome and the level of communication, satisfaction, confidence and compatibility. An experiment focusing on these factors is designed and implemented. From the experimental findings, the personality of the two partners in pair programming is found to have a significant impact on the pair programming output. Also, it is discovered that same gender pairs exhibited an unusually high level of communication, satisfaction and compatibility between each other, especially among female-female pairs. A detailed statistical experiment result based on research hypotheses is reported

Strategies for the intelligent selection of components

It is becoming common to build applications as component-intensive systems - a mixture of fresh code and existing components. For application developers the selection of components to incorporate is key to overall system quality - so they want the `best\u27. For each selection task, the application developer will de ne requirements for the ideal component and use them to select the most suitable one. While many software selection processes exist there is a lack of repeatable, usable, exible, automated processes with tool support. This investigation has focussed on nding and implementing strategies to enhance the selection of software components. The study was built around four research elements, targeting characterisation, process, strategies and evaluation. A Post-positivist methodology was used with the Spiral Development Model structuring the investigation. Data for the study is generated using a range of qualitative and quantitative methods including a survey approach, a range of case studies and quasiexperiments to focus on the speci c tuning of tools and techniques. Evaluation and review are integral to the SDM: a Goal-Question-Metric (GQM)-based approach was applied to every Spiral

Abordagem de selecção de software Commercial Off-The-Shelf suportada por um sistema de apoio à decisão

A selecção de produtos de software Commercial Off-The-Shelf (COTS) tem vindo a adquirir uma importância crescente na engenharia de software. No processo de desenvolvimento de sistemas baseados em COTS é essencial seleccionar os produtos COTS mais adequados, pois uma boa selecção é crucial para o sucesso do sistema final. Existem muitas propostas para modelar o processo de selecção de COTS, no entanto, ainda há espaço para a optimização de processos e de soluções. A principal contribuição desta dissertação é uma abordagem genérica de selecção de COTS e um Sistema de Apoio à Decisão (SAD) para suportá-la, denominado COTS-3S (COTS Selection Support System). Esta abordagem distingue-se das restantes pela sua capacidade de agilizar e simplificar o processo de selecção, sobretudo através da sua utilização em conjunto com o COTS-3S, ajudando a seleccionar produtos COTS com base na sua adequabilidade. A abordagem proposta integra vários conceitos e técnicas, incluindo o suporte à decisão no âmbito da engenharia de software, a tomada de decisão multicritério e a optimização de resolução de problemas. Para validar a aplicabilidade da abordagem proposta em conjunto com o COTS-3S, é apresentado um caso prático no domínio dos sistemas Enterprise Resource Planning (ERP). Os seus resultados são utilizados para demonstrar as potenciais vantagens, desvantagens, benefícios e limitações da utilização da abordagem proposta com o COTS-3S. Para finalizar, são apresentados os objectivos concretizados, realizada uma comparação entre a abordagem proposta e as restantes, descritas as limitações encontradas durante a realização deste trabalho e propostos possíveis trabalhos futuros.Selection of Commercial Off-The-Shelf (COTS) software products has a growing importance in software engineering. In the process of developing COTS-based systems it is essential to select the most suitable COTS products, as a good selection is crucial for the final system’s success. Many proposals are made to model the COTS selection process, however it is still possible to optimize processes and solutions. The main contribution of this dissertation is a generic COTS selection approach and a Decision Support System (DSS) to support it, called COTS-3S (COTS Selection Support System). This approach is distinguished from others by its capability to streamline and simplify the selection process through its use with COTS-3S, helping to select COTS products based on their fitness. The proposed approach integrates several concepts and techniques, including software engineering decision support, multi-criteria decision making, and optimization problem solving. In order to validate the applicability of the proposed approach along with COTS-3S, a practical case in the domain of Enterprise Resource Planning (ERP) systems is presented. Thus, its results are used to show the potential advantages, disadvantages, benefits and limitations of using the proposed approach with COTS-3S. Finally, the pursued objectives are presented, a comparison between the proposed approach and the others is made, the constraints found during the elaboration of this work are described and possible future work is proposed

Systematic construction of goal-oriented COTS taxonomies

El proceso de construir software a partir del ensamblaje e integración de soluciones de software pre-fabricadas, conocidas como componentes COTS (Comercial-Off-The-Shelf) se ha convertido en una necesidad estratégica en una amplia variedad de áreas de aplicación. En general, los componentes COTS son componentes de software que proveen una funcionalidad específica, que están disponibles en el mercado para ser adquiridos e integrados dentro de otros sistemas de software. Los beneficios potenciales de esta tecnología son principalmente la reducción de costes y el acortamiento del tiempo de desarrollo, a la vez que fomenta la calidad. Sin embargo, numerosos retos que van desde problemas técnicos y legales deben ser afrontados para adaptar las actividades tradicionales de ingeniería de software para explotar los beneficios del uso de COTS para el desarrollo de sistemas.Actualmente, existe un incrementalmente enorme mercado de componentes COTS; así, una de las actividades más críticas en el desarrollo de sistemas basados en COTS es la selección de componentes que deben ser integrados en el sistema a desarrollar. La selección está básicamente compuesta de dos procesos principales: La búsqueda de componentes candidatos en el mercado y su posterior evaluación con respecto a los requisitos del sistema. Desafortunadamente, la mayoría de los métodos existentes para seleccionar COTS, se enfocan en el proceso de evaluación, dejando de lado el problema de buscar los componentes en el mercado. La búsqueda de componentes en el mercado no es una tarea trivial, teniendo que afrontar varias características del mercado de COTS, tales como su naturaleza dispersa y siempre creciente, cambio y evolución constante; en este contexto, la obtención de información de calidad acerca de los componentes no es una tarea fácil. Como consecuencia, el proceso de selección de COTS se ve seriamente dañado. Además, las alternativas tradicionales de reuso también carecen de soluciones apropiadas para reusar componentes COTS y el conocimiento adquirido en cada proceso de selección. Esta carencia de propuestas es un problema muy serio que incrementa los riesgos de los proyectos de selección de COTS, además de hacerlos ineficientes y altamente costosos. Esta disertación presenta el método GOThIC (Goal- Oriented Taxonomy and reuse Infrastructure Construction) enfocado a la construcción de infraestructuras de reuso para facilitar la búsqueda y reuso de componentes COTS. El método está basado en el uso de objetivos para construir taxonomías abstractas, bien fundamentadas y estables para lidiar con las características del mercado de COTS. Los nodos de las taxonomías son caracterizados por objetivos, sus relaciones son declaradas como dependencias y varios artefactos son construidos y gestionados para promover la reusabilidad y lidiar con la evolución constante.El método GOThIC ha sido elaborado a través de un proceso iterativo de investigación-acción para identificar los retos reales relacionados con el proceso de búsqueda de COTS. Posteriormente, las soluciones posibles fueron evaluadas e implementadas en varios casos de estudio en el ámbito industrial y académico en diversos dominios. Los resultados más relevantes fueron registrados y articulados en el método GOThIC. La evaluación industrial preliminar del método se ha llevado a cabo en algunas compañías en Noruega.The process of building software systems by assembling and integrating pre-packaged solutions in the form of Commercial-Off-The-Shelf (COTS) software components has become a strategic need in a wide variety of application areas. In general, COTS components are software components that provide a specific functionality, available in the market to be purchased, interfaced and integrated into other software systems. The potential benefits of this technology are mainly its reduced costs and shorter development time, while maintaining the quality. Nevertheless, many challenges ranging form technical to legal issues must be faced for adapting the traditional software engineering activities in order to exploit these benefits.Nowadays there is an increasingly huge marketplace of COTS components; therefore, one of the most critical activities in COTS-based development is the selection of the components to be integrated into the system under development. Selection is basically composed of two main processes, namely: searching of candidates from the marketplace and their evaluation with respect to the system requirements. Unfortunately, most of the different existing methods for COTS selection focus their efforts on evaluation, letting aside the problem of searching components in the marketplace. Searching candidate COTS is not an easy task, having to cope with some challenging marketplace characteristics related to its widespread, evolvable and growing nature; and the lack of available and well-suited information to obtain a quality-assured search. Indeed, traditional reuse approaches also lack of appropriate solutions to reuse COTS components and the knowledge gained in each selection process. This lack of proposals is a serious drawback that makes the whole selection process highly risky, and often expensive and inefficient. This dissertation introduces the GOThIC (Goal- Oriented Taxonomy and reuse Infrastructure Construction) method aimed at building a domain reuse infrastructure for facilitating COTS components searching and reuse. It is based on goal-oriented approaches for building abstract, well-founded and stable taxonomies capable of dealing with the COTS marketplace characteristics. Thus, the nodes of these taxonomies are characterized by means of goals, their relationships declared as dependencies among them and several artifacts are constructed and managed for reusability and evolution purposes. The GOThIC method has been elaborated following an iterative process based on action research premises to identify the actual challenges related to COTS components searching. Then, possible solutions were envisaged and implemented by several industrial and academic case studies in different domains. Successful results were recorded to articulate the synergic GOThIC method solution, followed by its preliminary industrial evaluation in some Norwegian companies

Composable Process Elements for Developing COTS-Based Applications EDSER-5 Position Paper

issued a challenge to the software engineering community to provide better methods for dealing with COTS-based software systems, and to present them a