3 research outputs found
Contrastive Learning for API Aspect Analysis
We present a novel approach - CLAA - for API aspect detection in API reviews
that utilizes transformer models trained with a supervised contrastive loss
objective function. We evaluate CLAA using performance and impact analysis. For
performance analysis, we utilized a benchmark dataset on developer discussions
collected from Stack Overflow and compare the results to those obtained using
state-of-the-art transformer models. Our experiments show that contrastive
learning can significantly improve the performance of transformer models in
detecting aspects such as Performance, Security, Usability, and Documentation.
For impact analysis, we performed empirical and developer study. On a randomly
selected and manually labeled 200 online reviews, CLAA achieved 92% accuracy
while the SOTA baseline achieved 81.5%. According to our developer study
involving 10 participants, the use of 'Stack Overflow + CLAA' resulted in
increased accuracy and confidence during API selection. Replication package:
https://github.com/shahariar-shibli/Contrastive-Learning-for-API-Aspect-AnalysisComment: Accepted in the 38th IEEE/ACM International Conference on Automated
Software Engineering (ASE2023
AutoML from Software Engineering Perspective: Landscapes and Challenges
Machine learning (ML) has been widely adopted in modern software, but the manual configuration of ML (e.g., hyper-parameter configuration) poses a significant challenge to software developers. Therefore, automated ML (AutoML), which seeks the optimal configuration of ML automatically, has received increasing attention from the software engineering community. However, to date, there is no comprehensive understanding of how AutoML is used by developers and what challenges developers encounter in using AutoML for software development. To fill this knowledge gap, we conduct the first study on understanding the use and challenges of AutoML from software developers’ perspective. We collect and analyze 1,554 AutoML downstream repositories, 769 AutoML-related Stack Overflow questions, and 1,437 relevant GitHub issues. The results suggest the increasing popularity of AutoML in a wide range of topics, but also the lack of relevant expertise. We manually identify specific challenges faced by developers for AutoML-enabled software. Based on the results, we derive a series of implications for AutoML framework selection, framework development, and research