Software development process mining: discovery, conformance checking and enhancement

Context. Modern software projects require the proper allocation of human, technical and financial resources. Very often, project managers make decisions supported only by their personal experience, intuition or simply by mirroring activities performed by others in similar contexts. Most attempts to avoid such practices use models based on lines of code, cyclomatic complexity or effort estimators, thus commonly supported by software repositories which are known to contain several flaws. Objective. Demonstrate the usefulness of process data and mining methods to enhance the software development practices, by assessing efficiency and unveil unknown process insights, thus contributing to the creation of novel models within the software development analytics realm. Method. We mined the development process fragments of multiple developers in three different scenarios by collecting Integrated Development Environment (IDE) events during their development sessions. Furthermore, we used process and text mining to discovery developers’ workflows and their fingerprints, respectively. Results. We discovered and modeled with good quality developers’ processes during programming sessions based on events extracted from their IDEs. We unveiled insights from coding practices in distinct refactoring tasks, built accurate software complexity forecast models based only on process metrics and setup a method for characterizing coherently developers’ behaviors. The latter may ultimately lead to the creation of a catalog of software development process smells. Conclusions. Our approach is agnostic to programming languages, geographic location or development practices, making it suitable for challenging contexts such as in modern global software development projects using either traditional IDEs or sophisticated low/no code platforms.Contexto. Projetos de software modernos requerem a correta alocação de recursos humanos, técnicos e financeiros. Frequentemente, os gestores de projeto tomam decisões suportadas apenas na sua própria experiência, intuição ou simplesmente espelhando atividades executadas por terceiros em contextos similares. As tentativas para evitar tais práticas baseiam-se em modelos que usam linhas de código, a complexidade ciclomática ou em estimativas de esforço, sendo estes tradicionalmente suportados por repositórios de software conhecidos por conterem várias limitações. Objetivo. Demonstrar a utilidade dos dados de processo e respetivos métodos de análise na melhoria das práticas de desenvolvimento de software, colocando o foco na análise da eficiência e revelando aspetos dos processos até então desconhecidos, contribuindo para a criação de novos modelos no contexto de análises avançadas para o desenvolvimento de software. Método. Explorámos os fragmentos de processo de vários programadores em três cenários diferentes, recolhendo eventos durante as suas sessões de desenvolvimento no IDE. Adicionalmente, usámos métodos de descoberta e análise de processos e texto no sentido de modelar o fluxo de trabalho dos programadores e as suas características individuais, respetivamente. Resultados. Descobrimos e modelámos com boa qualidade os processos dos programadores durante as suas sessões de trabalho, usando eventos provenientes dos seus IDEs. Revelámos factos desconhecidos sobre práticas de refabricação, construímos modelos de previsão da complexidade ciclomática usando apenas métricas de processo e criámos um método para caracterizar coerentemente os comportamentos dos programadores. Este último, pode levar à criação de um catálogo de boas/más práticas no processo de desenvolvimento de software. Conclusões. A nossa abordagem é agnóstica em termos de linguagens de programação, localização geográfica ou prática de desenvolvimento, tornando-a aplicável em contextos complexos tal como em projetos modernos de desenvolvimento global que utilizam tanto os IDEs tradicionais como as atuais e sofisticadas plataformas "low/no code"

SANTO: Social Aerial NavigaTion in Outdoors

In recent years, the advances in remote connectivity, miniaturization of electronic components and computing power has led to the integration of these technologies in daily devices like cars or aerial vehicles. From these, a consumer-grade option that has gained popularity are the drones or unmanned aerial vehicles, namely quadrotors. Although until recently they have not been used for commercial applications, their inherent potential for a number of tasks where small and intelligent devices are needed is huge. However, although the integrated hardware has advanced exponentially, the refinement of software used for these applications has not beet yet exploited enough. Recently, this shift is visible in the improvement of common tasks in the field of robotics, such as object tracking or autonomous navigation. Moreover, these challenges can become bigger when taking into account the dynamic nature of the real world, where the insight about the current environment is constantly changing. These settings are considered in the improvement of robot-human interaction, where the potential use of these devices is clear, and algorithms are being developed to improve this situation. By the use of the latest advances in artificial intelligence, the human brain behavior is simulated by the so-called neural networks, in such a way that computing system performs as similar as possible as the human behavior. To this end, the system does learn by error which, in an akin way to the human learning, requires a set of previous experiences quite considerable, in order for the algorithm to retain the manners. Applying these technologies to robot-human interaction do narrow the gap. Even so, from a bird's eye, a noticeable time slot used for the application of these technologies is required for the curation of a high-quality dataset, in order to ensure that the learning process is optimal and no wrong actions are retained. Therefore, it is essential to have a development platform in place to ensure these principles are enforced throughout the whole process of creation and optimization of the algorithm. In this work, multiple already-existing handicaps found in pipelines of this computational gauge are exposed, approaching each of them in a independent and simple manner, in such a way that the solutions proposed can be leveraged by the maximum number of workflows. On one side, this project concentrates on reducing the number of bugs introduced by flawed data, as to help the researchers to focus on developing more sophisticated models. On the other side, the shortage of integrated development systems for this kind of pipelines is envisaged, and with special care those using simulated or controlled environments, with the goal of easing the continuous iteration of these pipelines.Thanks to the increasing popularity of drones, the research and development of autonomous capibilities has become easier. However, due to the challenge of integrating multiple technologies, the available software stack to engage this task is restricted. In this thesis, we accent the divergencies among unmanned-aerial-vehicle simulators and propose a platform to allow faster and in-depth prototyping of machine learning algorithms for this drones

Improving Android app security and privacy with developers

Existing research has uncovered many security vulnerabilities in Android applications (apps) caused by inexperienced, and unmotivated developers. Especially, the lack of tool support makes it hard for developers to avoid common security and privacy problems in Android apps. As a result, this leads to apps with security vulnerability that exposes end users to a multitude of attacks. This thesis presents a line of work that studies and supports Android developers in writing more secure code. We first studied to which extent tool support can help developers in creating more secure applications. To this end, we developed and evaluated an Android Studio extension that identifies common security problems of Android apps, and provides developers suggestions to more secure alternatives. Subsequently, we focused on the issue of outdated third-party libraries in apps which also is the root cause for a variety of security vulnerabilities. Therefore, we analyzed all popular 3rd party libraries in the Android ecosystem, and provided developers feedback and guidance in the form of tool support in their development environment to fix such security problems. In the second part of this thesis, we empirically studied and measured the impact of user reviews on app security and privacy evolution. Thus, we built a review classifier to identify security and privacy related reviews and performed regression analysis to measure their impact on the evolution of security and privacy in Android apps. Based on our results we proposed several suggestions to improve the security and privacy of Android apps by leveraging user feedbacks to create incentives for developers to improve their apps toward better versions.Die bisherige Forschung zeigt eine Vielzahl von Sicherheitslücken in Android-Applikationen auf, welche sich auf unerfahrene und unmotivierte Entwickler zurückführen lassen. Insbesondere ein Mangel an Unterstützung durch Tools erschwert es den Entwicklern, häufig auftretende Sicherheits- und Datenschutzprobleme in Android Apps zu vermeiden. Als Folge führt dies zu Apps mit Sicherheitsschwachstellen, die Benutzer einer Vielzahl von Angriffen aussetzen. Diese Dissertation präsentiert eine Reihe von Forschungsarbeiten, die Android-Entwickler bei der Entwicklung von sichereren Apps untersucht und unterstützt. In einem ersten Schritt untersuchten wir, inwieweit die Tool-Unterstützung Entwicklern beim Schreiben von sicherem Code helfen kann. Zu diesem Zweck entwickelten und evaluierten wir eine Android Studio-Erweiterung, die gängige Sicherheitsprobleme von Android-Apps identifiziert und Entwicklern Vorschläge für sicherere Alternativen bietet. Daran anknüpfend, konzentrierten wir uns auf das Problem veralteter Bibliotheken von Drittanbietern in Apps, die ebenfalls häufig die Ursache von Sicherheitslücken sein können. Hierzu analysierten wir alle gängigen 3rd-Party-Bibliotheken im Android-Ökosystem und gaben den Entwicklern Feedback und Anleitung in Form von Tool-Unterstützung in ihrer Entwicklungsumgebung, um solche Sicherheitsprobleme zu beheben. Im zweiten Teil dieser Dissertation untersuchten wir empirisch die Auswirkungen von Benutzer-Reviews im Android Appstore auf die Entwicklung der Sicherheit und des Datenschutzes von Apps. Zu diesem Zweck entwickelten wir einen Review-Klassifikator, welcher in der Lage ist sicherheits- und datenschutzbezogene Reviews zu identifizieren. Nachfolgend untersuchten wir den Einfluss solcher Reviews auf die Entwicklung der Sicherheit und des Datenschutzes in Android-Apps mithilfe einer Regressionsanalyse. Basierend auf unseren Ergebnissen präsentieren wir verschiedene Vorschläge zur Verbesserung der Sicherheit und des Datenschutzes von Android-Apps, welche die Reviews der Benutzer zur Schaffung von Anreizen für Entwickler nutzen

Overcoming Language Dichotomies: Toward Effective Program Comprehension for Mobile App Development

Mobile devices and platforms have become an established target for modern software developers due to performant hardware and a large and growing user base numbering in the billions. Despite their popularity, the software development process for mobile apps comes with a set of unique, domain-specific challenges rooted in program comprehension. Many of these challenges stem from developer difficulties in reasoning about different representations of a program, a phenomenon we define as a "language dichotomy". In this paper, we reflect upon the various language dichotomies that contribute to open problems in program comprehension and development for mobile apps. Furthermore, to help guide the research community towards effective solutions for these problems, we provide a roadmap of directions for future work.Comment: Invited Keynote Paper for the 26th IEEE/ACM International Conference on Program Comprehension (ICPC'18

Holistic recommender systems for software engineering

The knowledge possessed by developers is often not sufficient to overcome a programming problem. Short of talking to teammates, when available, developers often gather additional knowledge from development artifacts (e.g., project documentation), as well as online resources. The web has become an essential component in the modern developer’s daily life, providing a plethora of information from sources like forums, tutorials, Q&A websites, API documentation, and even video tutorials. Recommender Systems for Software Engineering (RSSE) provide developers with assistance to navigate the information space, automatically suggest useful items, and reduce the time required to locate the needed information. Current RSSEs consider development artifacts as containers of homogeneous information in form of pure text. However, text is a means to represent heterogeneous information provided by, for example, natural language, source code, interchange formats (e.g., XML, JSON), and stack traces. Interpreting the information from a pure textual point of view misses the intrinsic heterogeneity of the artifacts, thus leading to a reductionist approach. We propose the concept of Holistic Recommender Systems for Software Engineering (H-RSSE), i.e., RSSEs that go beyond the textual interpretation of the information contained in development artifacts. Our thesis is that modeling and aggregating information in a holistic fashion enables novel and advanced analyses of development artifacts. To validate our thesis we developed a framework to extract, model and analyze information contained in development artifacts in a reusable meta- information model. We show how RSSEs benefit from a meta-information model, since it enables customized and novel analyses built on top of our framework. The information can be thus reinterpreted from an holistic point of view, preserving its multi-dimensionality, and opening the path towards the concept of holistic recommender systems for software engineering

Utilizing static and dynamic software analysis to aid cost estimation, software visualization, and test quality management

The main results presented in the thesis are related to the semi- or fully-automated analysis of the software and its development processes. My overall research goal is to provide meaningful insights, methods, and practical tools to help the work of stakeholders during various phases of software development. The thesis statements have been grouped into three major thesis points, namely "Measuring, predicting, and comparing the productivity of developer teams"; "Providing immersive methods for software and unit test visualization"; and "Spotting the structures in the package hierarchy that required attention using test coverage data"

VIRHUS: uma plataforma computacional para a simulação de sinais fisiológicos de humanos virtuais

The ability to access bio-signals of participants for research activity is limited specially for informal settings like academic projects. These limitations can be in part overcome it by using software to simulated good enough physiological data. In this work we propose and develop a computational platform to simulate (biological signals of ) virtual humans as a service. The system adopts the concept of digital twin to structure the simulation processes. In this case, the system is not sensing real participants, rather uses pre-recorded signals as inputs to auto-encoders that generate realistic synthetic signal for a virtual human, i.e., a digital twin. The pre-recorded signals used were the electrocardiogram, electrodermal activity and electromyography signals which were labeled with the ongoing emotion. The system, VIRHUS, offers an interactive web environment to create the required virtual humans and manage the simulation processes. A scalable backend takes care of the asynchronous generation of signals, that can be streamed to endpoints (and consumed by external applications) or exported as files, for convenience. As a proof of concept, the “virtual human” data can be parameterized to include emotional traits in the bio-signals (happy, sad,...), generating meaningful variations in data for applications developers.A capacidade de aceder aos biossinais de participantes para actividades de investigação é limitada, especialmente em contextos informais, tais como projectos académicos. Estas limitações podem ser parcialmente ultrapassadas através da utilização de software para simular dados fisiológicos suficientemente fidedignos. Neste trabalho, propomos e desenvolvemos uma plataforma computacional para simular (sinais biológicos de ) seres humanos virtuais como um serviço. O sistema adopta o conceito de réplica digital ("digital twin") para estruturar os processos de simulação. Neste caso, o sistema não está a monitorar participantes reais, mas utiliza sinais pré-gravados como entradas para autocodificadores que geram um sinal sintético realista para um ser humano virtual, ou seja, uma réplica digital. Os sinais pré-gravados utilizados foram o electrocardiograma, a actividade electrodérmica e os sinais electromiográficos que foram marcados com a emoção em progresso. O sistema, VIRHUS, fornece um ambiente web interactivo para criar os seres humanos virtuais necessários e gerir os processos de simulação. Um backend escalável cuida da geração assíncrona de sinais, que podem ser transmitidos para pontos de acesso programático (e consumidos por aplicações externas) ou exportados como ficheiros por conveniência. Como prova de conceito, os dados "humanos virtuais" podem ser parametrizados para incluir traços emocionais nos biossinais (feliz, triste,...), gerando variações significativas nos dados para os programadores de aplicações.Mestrado em Engenharia Informátic