Can a Machine Replace Humans in Building Regular Expressions? A Case Study

Regular expressions are routinely used in a variety of different application domains. But building a regular expression involves a considerable amount of skill, expertise, and creativity. In this work, the authors investigate whether a machine can surrogate these qualities and automatically construct regular expressions for tasks of realistic complexity. They discuss a large-scale experiment involving more than 1,700 users on 10 challenging tasks. The authors compare the solutions constructed by these users to those constructed by a tool based on genetic programming that they recently developed and made publicly available. The quality of automatically constructed solutions turned out to be similar to the quality of those constructed by the most skilled user group; the time for automatic construction was likewise similar to the time required by human users

Benchmarking and Explaining Large Language Model-based Code Generation: A Causality-Centric Approach

While code generation has been widely used in various software development scenarios, the quality of the generated code is not guaranteed. This has been a particular concern in the era of large language models (LLMs)- based code generation, where LLMs, deemed a complex and powerful black-box model, is instructed by a high-level natural language specification, namely a prompt, to generate code. Nevertheless, effectively evaluating and explaining the code generation capability of LLMs is inherently challenging, given the complexity of LLMs and the lack of transparency. Inspired by the recent progress in causality analysis and its application in software engineering, this paper launches a causality analysis-based approach to systematically analyze the causal relations between the LLM input prompts and the generated code. To handle various technical challenges in this study, we first propose a novel causal graph-based representation of the prompt and the generated code, which is established over the fine-grained, human-understandable concepts in the input prompts. The formed causal graph is then used to identify the causal relations between the prompt and the derived code. We illustrate the insights that our framework can provide by studying over 3 popular LLMs with over 12 prompt adjustment strategies. The results of these studies illustrate the potential of our technique to provide insights into LLM effectiveness, and aid end-users in understanding predictions. Additionally, we demonstrate that our approach provides actionable insights to improve the quality of the LLM-generated code by properly calibrating the prompt

Does Automated Unit Test Generation Really Help Software Testers? A Controlled Empirical Study

Work on automated test generation has produced several tools capable of generating test data which achieves high structural coverage over a program. In the absence of a specification, developers are expected to manually construct or verify the test oracle for each test input. Nevertheless, it is assumed that these generated tests ease the task of testing for the developer, as testing is reduced to checking the results of tests. While this assumption has persisted for decades, there has been no conclusive evidence to date confirming it. However, the limited adoption in industry indicates this assumption may not be correct, and calls into question the practical value of test generation tools. To investigate this issue, we performed two controlled experiments comparing a total of 97 subjects split between writing tests manually and writing tests with the aid of an automated unit test generation tool, EvoSuite. We found that, on one hand, tool support leads to clear improvements in commonly applied quality metrics such as code coverage (up to 300% increase). However, on the other hand, there was no measurable improvement in the number of bugs actually found by developers. Our results not only cast some doubt on how the research community evaluates test generation tools, but also point to improvements and future work necessary before automated test generation tools will be widely adopted by practitioners

A survey on utilization of data mining approaches for dermatological (skin) diseases prediction

Due to recent technology advances, large volumes of medical data is obtained. These data contain valuable information. Therefore data mining techniques can be used to extract useful patterns. This paper is intended to introduce data mining and its various techniques and a survey of the available literature on medical data mining. We emphasize mainly on the application of data mining on skin diseases. A categorization has been provided based on the different data mining techniques. The utility of the various data mining methodologies is highlighted. Generally association mining is suitable for extracting rules. It has been used especially in cancer diagnosis. Classification is a robust method in medical mining. In this paper, we have summarized the different uses of classification in dermatology. It is one of the most important methods for diagnosis of erythemato-squamous diseases. There are different methods like Neural Networks, Genetic Algorithms and fuzzy classifiaction in this topic. Clustering is a useful method in medical images mining. The purpose of clustering techniques is to find a structure for the given data by finding similarities between data according to data characteristics. Clustering has some applications in dermatology. Besides introducing different mining methods, we have investigated some challenges which exist in mining skin data

Algorithms for Automatic Label Placement

Práce popisuje problém automatického umísťování popisků do mapy. Jednotlivé bodové, čárové a plošné objekty v mapě je třeba označit odpovídajícími textovými či obrázkovými popisky. Tyto popisky je nutné rozmístit tak, aby se vzájemně nepřekrývaly a zároveň byly jasně přiřaditelné k odpovídajícím objektům. O problému je známo, že je NP-těžký a nalezení optimálního rozmístění všech popisků je výpočetně velmi náročné i pro nejjednodušší mapy. Pozornost je věnována umísťování popisků označujících bodové a čárové objekty, včetně prvního kroku obnášejícího přípravu možných pozic pro umístění těchto popisků, při dodržení běžných kartografických pravidel pro rozmísťování popisků. Následně jsou na problém aplikovány tři různé druhy algoritmů -- greedy ("hladové") algoritmy v kombinaci s lokálním prohledáváním, matematická optimalizace (v podobě 0-1 celočíselného programování) a genetické algoritmy. Popsané algoritmy jsou v softwarové části práce implementovány a na závěr porovnány na několika různých datových sadách, vycházejících z reálných geografických podkladů a z náhodně vygenerovaných map. Závěrečné srovnání se zaměřuje na kvalitu výsledného rozmístění (dle metrik definovaných v práci), času potřebnému k nalezení řešení a také na determinističnost daných algoritmů.Thesis describes the problem of automatic map label placement. Various point, line or area features in maps must be marked with matching text or graphic labels. These labels have to be placed so they do not overlap with each other and they are clearly associable with corresponding map features. The problem is known to be NP-hard and finding optimal positions of all map labels is highly computationally expensive, even for the simplest maps. Focus is given to the placement of labels describing point and line map features, including the initial phase of enumerating possible label positions, respecting the basic cartographic rules common for those labels. Afterwards, three different algorithm types are applied to the problem itself -- greedy algorithms (in combination with local search optimization), mathematical optimization (0-1 integer programming) and genetic algorithms. Ultimately, the described algorithms are implemented in the software part of the work and compared on various data sets, based on both real world geographical data and randomly generated maps. The final comparison focuses especially on the quality of the result (scored by the metrics defined in the thesis), time needed to find the solution and determinism of the given algorithms

Holistic recommender systems for software engineering

The knowledge possessed by developers is often not sufficient to overcome a programming problem. Short of talking to teammates, when available, developers often gather additional knowledge from development artifacts (e.g., project documentation), as well as online resources. The web has become an essential component in the modern developer’s daily life, providing a plethora of information from sources like forums, tutorials, Q&A websites, API documentation, and even video tutorials. Recommender Systems for Software Engineering (RSSE) provide developers with assistance to navigate the information space, automatically suggest useful items, and reduce the time required to locate the needed information. Current RSSEs consider development artifacts as containers of homogeneous information in form of pure text. However, text is a means to represent heterogeneous information provided by, for example, natural language, source code, interchange formats (e.g., XML, JSON), and stack traces. Interpreting the information from a pure textual point of view misses the intrinsic heterogeneity of the artifacts, thus leading to a reductionist approach. We propose the concept of Holistic Recommender Systems for Software Engineering (H-RSSE), i.e., RSSEs that go beyond the textual interpretation of the information contained in development artifacts. Our thesis is that modeling and aggregating information in a holistic fashion enables novel and advanced analyses of development artifacts. To validate our thesis we developed a framework to extract, model and analyze information contained in development artifacts in a reusable meta- information model. We show how RSSEs benefit from a meta-information model, since it enables customized and novel analyses built on top of our framework. The information can be thus reinterpreted from an holistic point of view, preserving its multi-dimensionality, and opening the path towards the concept of holistic recommender systems for software engineering