AutoML from Software Engineering Perspective: Landscapes and Challenges

Machine learning (ML) has been widely adopted in modern software, but the manual configuration of ML (e.g., hyper-parameter configuration) poses a significant challenge to software developers. Therefore, automated ML (AutoML), which seeks the optimal configuration of ML automatically, has received increasing attention from the software engineering community. However, to date, there is no comprehensive understanding of how AutoML is used by developers and what challenges developers encounter in using AutoML for software development. To fill this knowledge gap, we conduct the first study on understanding the use and challenges of AutoML from software developers’ perspective. We collect and analyze 1,554 AutoML downstream repositories, 769 AutoML-related Stack Overflow questions, and 1,437 relevant GitHub issues. The results suggest the increasing popularity of AutoML in a wide range of topics, but also the lack of relevant expertise. We manually identify specific challenges faced by developers for AutoML-enabled software. Based on the results, we derive a series of implications for AutoML framework selection, framework development, and research

HC3 Plus: A Semantic-Invariant Human ChatGPT Comparison Corpus

ChatGPT has gained significant interest due to its impressive performance, but people are increasingly concerned about its potential risks, particularly around the detection of AI-generated content (AIGC), which is often difficult for untrained humans to identify. Current datasets utilized for detecting ChatGPT-generated text primarily center around question-answering, yet they tend to disregard tasks that possess semantic-invariant properties, such as summarization, translation, and paraphrasing. Our primary studies demonstrate that detecting model-generated text on semantic-invariant tasks is more difficult. To fill this gap, we introduce a more extensive and comprehensive dataset that considers more types of tasks than previous work, including semantic-invariant tasks. In addition, the model after a large number of task instruction fine-tuning shows a strong powerful performance. Owing to its previous success, we further instruct fine-tuning Tk-instruct and built a more powerful detection system. Experimental results show that our proposed detector outperforms the previous state-of-the-art RoBERTa-based detector

Adonis: Practical and Efficient Control Flow Recovery through OS-Level Traces

Control flow recovery is critical to promise the software quality, especially for large-scale software in production environment. However, the efficiency of most current control flow recovery techniques is compromised due to their runtime overheads along with deployment and development costs. To tackle this problem, we propose a novel solution, Adonis, which harnesses OS-level traces, such as dynamic library calls and system call traces, to efficiently and safely recover control flows in practice. Adonis operates in two steps: it first identifies the call-sites of trace entries, then it executes a pair-wise symbolic execution to recover valid execution paths. This technique has several advantages. First, Adonis does not require the insertion of any probes into existing applications, thereby minimizing runtime cost. Second, given that OS-level traces are hardware-independent, Adonis can be implemented across various hardware configurations without the need for hardware-specific engineering efforts, thus reducing deployment cost. Third, as Adonis is fully automated and does not depend on manually created logs, it circumvents additional development cost. We conducted an evaluation of Adonis on representative desktop applications and real-world IoT applications. Adonis can faithfully recover the control flow with 86.8% recall and 81.7% precision. Compared to the state-of-the-art log-based approach, Adonis can not only cover all the execution paths recovered, but also recover 74.9% of statements that cannot be covered. In addition, the runtime cost of Adonis is 18.3× lower than the instrument-based approach; the analysis time and storage cost (indicative of the deployment cost) of Adonis is 50× smaller and 443× smaller than the hardware-based approach, respectively. To facilitate future replication and extension of this work, we have made the code and data publicly available

InfoEntropy Loss to Mitigate Bias of Learning Difficulties for Generative Language Models

Generative language models are usually pretrained on large text corpus via predicting the next token (i.e., sub-word/word/phrase) given the previous ones. Recent works have demonstrated the impressive performance of large generative language models on downstream tasks. However, existing generative language models generally neglect an inherent challenge in text corpus during training, i.e., the imbalance between frequent tokens and infrequent ones. It can lead a language model to be dominated by common and easy-to-learn tokens, thereby overlooking the infrequent and difficult-to-learn ones. To alleviate that, we propose an Information Entropy Loss (InfoEntropy Loss) function. During training, it can dynamically assess the learning difficulty of a to-be-learned token, according to the information entropy of the corresponding predicted probability distribution over the vocabulary. Then it scales the training loss adaptively, trying to lead the model to focus more on the difficult-to-learn tokens. On the Pile dataset, we train generative language models at different scales of 468M, 1.2B, and 6.7B parameters. Experiments reveal that models incorporating the proposed InfoEntropy Loss can gain consistent performance improvement on downstream benchmarks

Efficient Silicon Metasurfaces for Visible Light

Dielectric metasurfaces require high refractive index contrast materials for optimum performance. This requirement imposes a severe restraint; either devices have been demonstrated at wavelengths of 700 nm and above using high-index semiconductors such as silicon, or they use lower index dielectric materials such as TiO2 or Si3N4 and operate in the visible wavelength regime. Here, we show that the high refractive index of silicon can be exploited at wavelengths as short as 532 nm by demonstrating a crystalline silicon metasurface with a transmission efficiency of 71% at this wavelength and a diffraction efficiency of 95% into the desired diffraction order. The metasurfaces consist of a graded array of silicon posts arranged in a square lattice on a quartz substrate. We show full 2π phase control, and we experimentally demonstrate polarization-independent beam deflection at 532 nm wavelength. Our results open a new way for realizing efficient metasurfaces based on silicon for the technologically all-important display applications

Correlation-driven eightfold magnetic anisotropy in a two-dimensional oxide monolayer.

Engineering magnetic anisotropy in two-dimensional systems has enormous scientific and technological implications. The uniaxial anisotropy universally exhibited by two-dimensional magnets has only two stable spin directions, demanding 180° spin switching between states. We demonstrate a previously unobserved eightfold anisotropy in magnetic SrRuO3 monolayers by inducing a spin reorientation in (SrRuO3)1/(SrTiO3) N superlattices, in which the magnetic easy axis of Ru spins is transformed from uniaxial 〈001〉 direction (N < 3) to eightfold 〈111〉 directions (N ≥ 3). This eightfold anisotropy enables 71° and 109° spin switching in SrRuO3 monolayers, analogous to 71° and 109° polarization switching in ferroelectric BiFeO3. First-principle calculations reveal that increasing the SrTiO3 layer thickness induces an emergent correlation-driven orbital ordering, tuning spin-orbit interactions and reorienting the SrRuO3 monolayer easy axis. Our work demonstrates that correlation effects can be exploited to substantially change spin-orbit interactions, stabilizing unprecedented properties in two-dimensional magnets and opening rich opportunities for low-power, multistate device applications