Finding high-quality grey literature for use as evidence in software engineering research.

Background: Software engineering research often uses practitioners as a source of evidence in their studies. This evidence is usually gathered through empirical methods such as surveys, interviews and ethnographic research. The web has brought with it the emergence of the social programmer. Software practitioners are publishing their opinions online through blog articles, discussion boards and Q&A sites. Mining these online sources of information could provide a new source of evidence which complements traditional evidence sources. There are benefits to the adoption of grey literature in software engineering research (such as bridging the gap between the state–of–art where research typically operates and the state–of–practice), but also significant challenges. The main challenge is finding grey literature which is of high– quality to the researcher given the vast volume of grey literature available on the web. The thesis defines the quality of grey literature in terms of its relevance to the research being undertaken and its credibility. The thesis also focuses on a particular type of grey literature that has been written by soft- ware practitioners. A typical example of such grey literature is blog articles, which are specifically used as examples throughout the thesis. Objectives: There are two main objectives to the thesis; to investigate the problems of finding high–quality grey literature, and to make progress in addressing those problems. In working towards these objectives, we investigate our main research question, how can researchers more effectively and efficiently search for and then select the higher–quality blog–like content relevant to their research? We divide this question into twelve sub–questions, and more formally define what we mean by ‘blog–like content.’ Method: To achieve the objectives, we first investigate how software engineering researchers define and assess quality when working with grey literature; and then work towards a methodology and also a tool–suite which can semi–automate the identification and the quality assessment of relevant grey literature for use as evidence in the researchers study. To investigate how software engineering researchers define and assess quality, we first conduct a literature review of credibility assessment to gather a set of credibility criteria. We then validate those criteria through a survey of software engineering researchers. This gives us an overall model of credibility assessment within software engineering research. We next investigate the empirical challenges of measuring quality and develop a methodology which has been adapted from the case survey methodology and aims to address the problems and challenges identified. Along with the methodology is a suggested tool–suite which is intended to help researchers in automating the application of a subset of the credibility model. The tool–suite developed supports the methodology by, for example, automating tasks in order to scale the analysis. The use of the methodology and tool–suite is then demonstrated through three examples. These examples include a partial evaluation of the methodology and tool–suite. Results: Our literature review of credibility assessment identified a set of criteria that have been used in previous research. However, we also found a lack of definitions for both the criteria and, more generally, the term credibility. Credibility assessment is a difficult and subjective task that is particular to each individual. Research has addressed this subjectivity by conducting studies that look at how particular user groups assess credibility e.g. pensioners, university students, the visually impaired, however none of the studies reviewed software engineering researchers. Informed by the literature review, we conducted a survey which we believe is the first study on the credibility assessment of software engineering researchers. The results of the survey are a more refined set of criteria, but also a set that many (approximately 60%) of the survey participants believed generalise to other types of media (both practitioner–generated and researcher–generated). We found that there are significant challenges in using blog–like content as evidence in research. For example, there are the challenges of identifying the high–quality content from the vast quantity available on the web, and then creating methods of analysis which are scalable to handle that vast quantity. In addressing these challenges, we produce: a set of heuristics which can help in finding higher–quality results when searching using traditional search engines, a validated list of reasoning markers that can aid in assessing the amount of reasoning within a document, a review of the current state of the experience mining domain, and a modifiable classification schema for classifying the source of URLs. With credibility assessment being such a subjective task, there can be no one–size–fits–all method to automating quality assessment. Instead, our methodology is intended to be used as a framework in which the researcher using it can swap out and adapt the criteria that we assess for their own criteria based on the context of the study being undertaken and the personal preference of the researcher. We find from the survey that there are a variety of attitude’s towards using grey literature in software engineering research and not all respondents view the use of grey literature as evidence in the way that we do (i.e. as having the same benefits and threats as other traditional methods of evidence gathering). Conclusion: The work presented in this thesis makes significant progress towards answering our research question and the thesis provides a foundation for future research on automated quality assessment and credibility. Adoption of the tools and methodology presented in this thesis can help more effectively and efficiently search for and select higher–quality blog–like content, but there is a need for more substantial research on the credibility assessment of software engineering researchers, and a more extensive credibility model to be produced. This can be achieved through replicating the literature review systematically, accepting more studies for analysis, and by conducting a more extensive survey with a greater number, and more representative selection, of survey respondents. With a more robust credibility model, we can have more confidence in the criteria that we choose to include within the methodology and tools, as well as automating the assessment of more criteria. Throughout the re- search, there has been a challenge in aggregating the results after assessing each criterion. Future research should look towards the adoption of machine learning methods to aid with this aggregation. We believe that the criteria and measures used by our tools can serve as features to machine learning classifiers which will be able to more accurately assess quality. However, be- fore such work is to take place, there is a need for annotated data–sets to be developed

Data management for production quality deep learning models: Challenges and solutions

Deep learning (DL) based software systems are difficult to develop and maintain in industrial settings due to several challenges. Data management is one of the most prominent challenges which complicates DL in industrial deployments. DL models are data-hungry and require high-quality data. Therefore, the volume, variety, velocity, and quality of data cannot be compromised. This study aims to explore the data management challenges encountered by practitioners developing systems with DL components, identify the potential solutions from the literature and validate the solutions through a multiple case study. We identified 20 data management challenges experienced by DL practitioners through a multiple interpretive case study. Further, we identified 48 articles through a systematic literature review that discuss the solutions for the data management challenges. With the second round of multiple case study, we show that many of these solutions have limitations and are not used in practice due to a combination of four factors: high cost, lack of skill-set and infrastructure, inability to solve the problem completely, and incompatibility with certain DL use cases. Thus, data management for data-intensive DL models in production is complicated. Although the DL technology has achieved very promising results, there is still a significant need for further research in the field of data management to build high-quality datasets and streams that can be used for building production-ready DL systems. Furthermore, we have classified the data management challenges into four categories based on the availability of the solutions.(c) 2022 The Authors. Published by Elsevier Inc. This is an open access article under the CC BY-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)

Automated Test Generation Based on an Applicational Model

Context: As testing is an extremely costly and time-consuming process, tools to automatically generate test cases have been proposed throughout the literature. OutSystems provides a software development environment where with the aid of the visual OutSystems language, developers can create their applications in an agile form, thus improving their productivity. Problem: As OutSystems aims at accelerating software development, automating the test case generation activity would bring great value to their clients. Objectives: The main objectives of this work are to: develop an algorithm that generates, automatically, test cases for OutSystems applications and evaluates the coverage they provide to the code, according to a set of criteria. Methods: The OutSystems language is represented as a graph to which developers can then add pieces of code by dragging nodes to the screen and connecting them to the graph. The methodology applied in this work consists in traversing these graphs with depth and breadth-first search algorithms, employing a boundary-value analysis to identify the test inputs and a cause-effect graphing to reduce the number of redundant inputs generated. To evaluate these test inputs, coverage criteria regarding the control flow of data are analysed according to node, branch, condition, modified condition-decision and multiple condition coverage. Results: This tool is able to generate test inputs that cover 100% of reachable code and the methodologies employed help greatly in reducing the inputs generated, as well as displaying a minimum set of test inputs with which the developer is already able to cover all traversable code. Usability tests also yield very optimistic feedback from users. Conclusions: This work’s objectives were fully met, seen as we have a running tool able to act upon a subset of the OutSystems applicational model. This work provides crucial information for assessing the quality of OutSystems applications, with value for OutSystems developers, in the form of efficiency and visibility

Controller as business manager

https://egrove.olemiss.edu/aicpa_guides/2707/thumbnail.jp

PopArt: Ranked Testing Efficiency

Too often, programmers are under pressure to maximize their confidence in the correctness of their code with a tight testing budget. Should they spend some of that budget on finding “interesting” inputs or spend their entire testing budget on test executions? Work on testing efficiency has explored two competing approaches to answer this question: systematic partition testing (ST), which defines a testing partition and tests its parts, and random testing (RT), which directly samples inputs with replacement. A consensus as to which is better when has yet to emerge. We present Probability Ordered Partition Testing (POPART), a new systematic partition-based testing strategy that visits the parts of a testing partition in decreasing probability order and in doing so leverages any non-uniformity over that partition. We show how to construct a homogeneous testing partition, a requirement for systematic testing, by using an executable oracle and the path partition. A program’s path partition is a naturally occurring testing partition that is usually skewed for the simple reason that some paths execute more frequently than others. To confirm this conventional wisdom, we instrument programs from the Codeflaws repository and find that 80% of them have a skewed path probability distribution. POPART visits the parts of a testing partition in decreasing probability order. We then compare POPART with RT to characterise the configuration space in which each is more efficient. We show that, when simulating Codeflaws, POPART outperforms RT after 100;000 executions. Our results reaffirm RT’s power for very small testing budgets but also show that for any application requiring high (above 90%) probability-weighted coverage POPART should be preferred. In such cases, despite paying more for each test execution, we prove that POPART outperforms RT: it traverses parts whose cumulative probability bounds that of random testing, showing that sampling without replacement pays for itself, given a nonuniform probability over a testing partition

Biometrics

Biometrics uses methods for unique recognition of humans based upon one or more intrinsic physical or behavioral traits. In computer science, particularly, biometrics is used as a form of identity access management and access control. It is also used to identify individuals in groups that are under surveillance. The book consists of 13 chapters, each focusing on a certain aspect of the problem. The book chapters are divided into three sections: physical biometrics, behavioral biometrics and medical biometrics. The key objective of the book is to provide comprehensive reference and text on human authentication and people identity verification from both physiological, behavioural and other points of view. It aims to publish new insights into current innovations in computer systems and technology for biometrics development and its applications. The book was reviewed by the editor Dr. Jucheng Yang, and many of the guest editors, such as Dr. Girija Chetty, Dr. Norman Poh, Dr. Loris Nanni, Dr. Jianjiang Feng, Dr. Dongsun Park, Dr. Sook Yoon and so on, who also made a significant contribution to the book

Call centre employment a qualitative study

This study explores aspects of the nature and experience of call centre employment. In 1972 only 42% of UK households had a home-based telephone (BIFU, 1996). By 2000 98% of UK homes had access to either fixed-line or mobile telephone services (Oftel, 2001). The commercial exploitation of this artifact is now being realised through call centres employing sophisticated information and communications technologies. Virtually unheard of a decade ago, UK call centres provided jobs for an estimated 264,000 people in 2001 (Datamonitor, 1999). They have increasingly attracted public and academic attention, much of the latter focused on issues of employee control and surveilance. This study uses analyses of call centre-related newspaper articles, a survey of Scottish recruitment and employment agencies, covert participant observation, and interviews with agency representatives and call centre employees to explore issues such as recruitment and selection, the nature and experience of employment, and employee turnover in call centres. The ethics of using covert methods are discussed. Four main conclusions emerge from the study. First, call centre employment can be differentiated from other occupations on the basis of recruitment and selection practices, employee skils and differences in work environments, performance monitoring and supervision practices and regulation of workplace behaviour. Second, job characteristics may predispose employees to low levels of job-related well-being and burnout. Third, levels of employee turnover may be linked to occupational novelty and the availability of pre-employment realistic job information. Fourth, automated systems are beginning to replace routine, repetitive, low value tasks, resulting in changes in the nature of call centre employment. Those jobs that remain seem likely to be more demanding with complex tasks and an emphasis on quality rather than quantity of interactions. The implications of the study's findings and conclusions for future research and for call centre employers and their employees are considered

Software Usability

This volume delivers a collection of high-quality contributions to help broaden developers’ and non-developers’ minds alike when it comes to considering software usability. It presents novel research and experiences and disseminates new ideas accessible to people who might not be software makers but who are undoubtedly software users