Automatic feedback and assessment of team-coding assignments in a DevOps context

We describe an automated assessment process for team-coding assignments based on DevOps best practices. This system and methodology includes the definition of Team Performance Metrics measuring properties of the software developed by each team, and their correct use of DevOps techniques. It tracks the progress on each of metric by each group. The methodology also defines Individual Performance Metrics to measure the impact of individual student contributions to increase in Team Performance Metrics. Periodically scheduled reports using these metrics provide students valuable feedback. This process also facilitates the process of assessing the assignments. Although this method is not intended to produce the final grade of each student, it provides very valuable information to the lecturers. We have used it as the main source of information for student and team assessment in one programming course. Additionally, we use other assessment methods to calculate the final grade: written conceptual tests to check their understanding of the development processes, and cross-evaluations. Qualitative evaluation of the students filling relevant questionnaires are very positive and encouraging.Open Access funding provided thanks to the CRUE-CSIC agreement with Springer Nature

Leveraging Final Degree Projects for Open Source Software Contributions

(1) Background: final year students of computer science engineering degrees must carry out a final degree project (FDP) in order to graduate. Students’ contributions to improve open source software (OSS) through FDPs can offer multiple benefits and challenges, both for the students, the instructors and for the project itself. This work reports on a practical experience developed by four students contributing to mature OSS projects during their FDPs, detailing how they addressed the multiple challenges involved, both from the students and teachers perspective. (2) Methods: we followed the work of four students contributing to two established OSS projects for two academic years and analyzed their work on GitHub and their responses to a survey. (3) Results: we obtained a set of specific recommendations for future practitioners and detailed a list of benefits achieved by steering FDP towards OSS contributions, for students, teachers and the OSS projects. (4) Conclusions: we find out that FDPs oriented towards enhancing OSS projects can introduce students into real-world, practical examples of software engineering principles, give them a boost in their confidence about their technical and communication skills and help them build a portfolio of contributions to daily used worldwide open source applications

Software Development Analytics in Practice: A Systematic Literature Review

Context:Software Development Analytics is a research area concerned with providing insights to improve product deliveries and processes. Many types of studies, data sources and mining methods have been used for that purpose. Objective:This systematic literature review aims at providing an aggregate view of the relevant studies on Software Development Analytics in the past decade (2010-2019), with an emphasis on its application in practical settings. Method:Definition and execution of a search string upon several digital libraries, followed by a quality assessment criteria to identify the most relevant papers. On those, we extracted a set of characteristics (study type, data source, study perspective, development life-cycle activities covered, stakeholders, mining methods, and analytics scope) and classified their impact against a taxonomy. Results:Source code repositories, experimental case studies, and developers are the most common data sources, study types, and stakeholders, respectively. Product and project managers are also often present, but less than expected. Mining methods are evolving rapidly and that is reflected in the long list identified. Descriptive statistics are the most usual method followed by correlation analysis. Being software development an important process in every organization, it was unexpected to find that process mining was present in only one study. Most contributions to the software development life cycle were given in the quality dimension. Time management and costs control were lightly debated. The analysis of security aspects suggests it is an increasing topic of concern for practitioners. Risk management contributions are scarce. Conclusions:There is a wide improvement margin for software development analytics in practice. For instance, mining and analyzing the activities performed by software developers in their actual workbench, the IDE

Report on the Second Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE2)

This technical report records and discusses the Second Workshop on Sustainable Software for Science: Practice and Experiences (WSSSPE2). The report includes a description of the alternative, experimental submission and review process, two workshop keynote presentations, a series of lightning talks, a discussion on sustainability, and five discussions from the topic areas of exploring sustainability; software development experiences; credit & incentives; reproducibility & reuse & sharing; and code testing & code review. For each topic, the report includes a list of tangible actions that were proposed and that would lead to potential change. The workshop recognized that reliance on scientific software is pervasive in all areas of world-leading research today. The workshop participants then proceeded to explore different perspectives on the concept of sustainability. Key enablers and barriers of sustainable scientific software were identified from their experiences. In addition, recommendations with new requirements such as software credit files and software prize frameworks were outlined for improving practices in sustainable software engineering. There was also broad consensus that formal training in software development or engineering was rare among the practitioners. Significant strides need to be made in building a sense of community via training in software and technical practices, on increasing their size and scope, and on better integrating them directly into graduate education programs. Finally, journals can define and publish policies to improve reproducibility, whereas reviewers can insist that authors provide sufficient information and access to data and software to allow them reproduce the results in the paper. Hence a list of criteria is compiled for journals to provide to reviewers so as to make it easier to review software submitted for publication as a “Software Paper.

An environment for sustainable research software in Germany and beyond: current state, open challenges, and call for action

Research software has become a central asset in academic research. It optimizes existing and enables new research methods, implements and embeds research knowledge, and constitutes an essential research product in itself. Research software must be sustainable in order to understand, replicate, reproduce, and build upon existing research or conduct new research effectively. In other words, software must be available, discoverable, usable, and adaptable to new needs, both now and in the future. Research software therefore requires an environment that supports sustainability. Hence, a change is needed in the way research software development and maintenance are currently motivated, incentivized, funded, structurally and infrastructurally supported, and legally treated. Failing to do so will threaten the quality and validity of research. In this paper, we identify challenges for research software sustainability in Germany and beyond, in terms of motivation, selection, research software engineering personnel, funding, infrastructure, and legal aspects. Besides researchers, we specifically address political and academic decision-makers to increase awareness of the importance and needs of sustainable research software practices. In particular, we recommend strategies and measures to create an environment for sustainable research software, with the ultimate goal to ensure that software-driven research is valid, reproducible and sustainable, and that software is recognized as a first class citizen in research. This paper is the outcome of two workshops run in Germany in 2019, at deRSE19 - the first International Conference of Research Software Engineers in Germany - and a dedicated DFG-supported follow-up workshop in Berlin

Exploring Security Practices in Infrastructure as Code: An Empirical Study

Cloud computing has become popular thanks to the widespread use of Infrastructure as Code (IaC) tools, allowing the community to conveniently manage and configure cloud infrastructure using scripts. However, the scripting process itself does not automatically prevent practitioners from introducing misconfigurations, vulnerabilities, or privacy risks. As a result, ensuring security relies on practitioners understanding and the adoption of explicit policies, guidelines, or best practices. In order to understand how practitioners deal with this problem, in this work, we perform an empirical study analyzing the adoption of IaC scripted security best practices. First, we select and categorize widely recognized Terraform security practices promulgated in the industry for popular cloud providers such as AWS, Azure, and Google Cloud. Next, we assess the adoption of these practices by each cloud provider, analyzing a sample of 812 open-source projects hosted on GitHub. For that, we scan each project configuration files, looking for policy implementation through static analysis (checkov). Additionally, we investigate GitHub measures that might be correlated with adopting these best practices. The category Access policy emerges as the most widely adopted in all providers, while Encryption in rest are the most neglected policies. Regarding GitHub measures correlated with best practice adoption, we observe a positive, strong correlation between a repository number of stars and adopting practices in its cloud infrastructure. Based on our findings, we provide guidelines for cloud practitioners to limit infrastructure vulnerability and discuss further aspects associated with policies that have yet to be extensively embraced within the industry.Comment: 50 pages, 13 figures, 10 table

The Role of Best Practices to Appraise Open Source Software

Thousands of open source software (OOS) projects are available for collaboration in platforms like Github or Sourceforge.  However, like traditional software, OOS projects have different quality levels.  The developer, or the end-user, need to know the quality of a given project before starting the collaboration  or its usage---they might of course to trust in the package before taking a decision.  In the context of OSS, trustability is a much more sensible concern; mainly end-users usually prefer to pay for  proprietary software, to feel more confident in the package quality.  OSS projects can be assessed like traditional software packages using the well known software metrics.  In this paper we want to go further and propose a finer grain process to do such quality analysis,  precisely tuned for this unique development environment.  As it is known, along the last years, open source communities have created their own standards and \emph{best practices}.  Nevertheless, the classic software metrics do not take into account the \emph{best practices}  established by the community.  We feel that it could be worthwhile to consider this peculiarity as a complementary source of assessment data.  Taking Ruby OSS community and projects as framework, this paper discusses the role of  \emph{best practices} in measuring software quality