Learning to Program with Natural Language

Large Language Models (LLMs) have shown remarkable performance in various basic natural language tasks, which raises hope for achieving Artificial General Intelligence. For completing the complex task, we still need a program for the task first and then ask LLMs to follow the program to generate the specific solution. We propose using natural language as a new programming language to describe task procedures, making them easily understandable to both humans and LLMs. ~The LLM is capable of directly generating natural language programs, but these programs may still contain factual errors or incomplete steps. Therefore, we further propose the Learning to Program (\text{LP}) method to ask LLMs themselves to learn the natural language program based on the training dataset of the complex task first and then use the learned program to guide the inference. Our experiments on the reasoning tasks of five different reasoning types (8 datasets) demonstrate the effectiveness of our approach. Further, our analysis experiment shows that the learned program can be directly used to guide another LLM to improve its performance, which reveals a new transfer learning paradigm.Comment: Work in progres

Random Silicon Sampling: Simulating Human Sub-Population Opinion Using a Large Language Model Based on Group-Level Demographic Information

Large language models exhibit societal biases associated with demographic information, including race, gender, and others. Endowing such language models with personalities based on demographic data can enable generating opinions that align with those of humans. Building on this idea, we propose "random silicon sampling," a method to emulate the opinions of the human population sub-group. Our study analyzed 1) a language model that generates the survey responses that correspond with a human group based solely on its demographic distribution and 2) the applicability of our methodology across various demographic subgroups and thematic questions. Through random silicon sampling and using only group-level demographic information, we discovered that language models can generate response distributions that are remarkably similar to the actual U.S. public opinion polls. Moreover, we found that the replicability of language models varies depending on the demographic group and topic of the question, and this can be attributed to inherent societal biases in the models. Our findings demonstrate the feasibility of mirroring a group's opinion using only demographic distribution and elucidate the effect of social biases in language models on such simulations.Comment: 25 pages, 4 figures, 19 Table

MOOCs as a Research Agenda: Changes Over Time

MOOCs (massive open online courses) have attracted considerable attention from researchers. Fueled by constant change and developments in educational technology, the trends of MOOCs have varied greatly over the years. To detect and visualize the developments and changes in MOOC research, 4,652 articles published between 2009 and 2021 were retrieved from Web of Science and Scopus with the aid of CiteSpace. This study sought to explore the number of publications, co-citation network, cluster analysis, timeline analysis, burstness analysis, and dual-map overlays based on co-citation relationships. The first finding was that the number of publications on MOOCs had increased consistently, and grew especially quickly between 2013 and 2015. Second, the main topic of the top 10 co-cited studies revolved around the problem of learner continuance. Third, blended programs, task-technology fit, and comparative analysis have emerged as popular subjects. Fourth, the development of MOOC research has followed distinct phases, with 2009 to 2012 the starting phase, 2013 to 2015 the high growth phase, 2016 to 2018 the plateau phase, and 2019 to 2021 another peak phase. Lastly, both cluster analysis and dual-map overlays provided empirical evidence of cross-disciplinary research. Our findings provided an in-depth and dynamic understanding of the development and evolution of MOOC research and also proposed novel ideas for future studies

Exploring Implicit Feedback for Open Domain Conversation Generation

User feedback can be an effective indicator to the success of the human-robot conversation. However, to avoid to interrupt the online real-time conversation process, explicit feedback is usually gained at the end of a conversation. Alternatively, users' responses usually contain their implicit feedback, such as stance, sentiment, emotion, etc., towards the conversation content or the interlocutors. Therefore, exploring the implicit feedback is a natural way to optimize the conversation generation process. In this paper, we propose a novel reward function which explores the implicit feedback to optimize the future reward of a reinforcement learning based neural conversation model. A simulation strategy is applied to explore the state-action space in training and test. Experimental results show that the proposed approach outperforms the Seq2Seq model and the state-of-the-art reinforcement learning model for conversation generation on automatic and human evaluations on the OpenSubtitles and Twitter datasets

Structural Topic Model Analysis of Mask-Wearing Issue Using International News Big Data

Media plays an important role in the acquisition of health information worldwide. This was particularly evident in the face of the COVID-19 epidemic. Relatedly, it is practical and desirable for people to wear masks for health, fashion, and religious regions. However, depending on cultural differences, people naturally accept wearing a mask, or they look upon it negatively. In 2020, the COVID-19 pandemic led to widespread mask-wearing mandates worldwide. In the case of COVID-19, wearing a mask is strongly recommended, so by analyzing the news data before and after the spread of the epidemic, it is possible to see how the direction of crisis management is being structured. In particular, by utilizing big data analysis of international news data, discourses around the world can be analyzed more deeply. This study collected and analyzed 58,061 international news items related to mask-wearing from 1 January 2019 to 31 December 2020. The collected dataset was compared before and after the World Health Organization’s pandemic declaration by applying structural topic model analysis. The results revealed that prior to the declaration, issues related to the COVID-19 outbreak were emphasized, but afterward, issues related to movement restrictions, quarantine management, and local economic impacts emerged

PPTC-R benchmark: Towards Evaluating the Robustness of Large Language Models for PowerPoint Task Completion

The growing dependence on Large Language Models (LLMs) for finishing user instructions necessitates a comprehensive understanding of their robustness to complex task completion in real-world situations. To address this critical need, we propose the PowerPoint Task Completion Robustness benchmark (PPTC-R) to measure LLMs' robustness to the user PPT task instruction and software version. Specifically, we construct adversarial user instructions by attacking user instructions at sentence, semantic, and multi-language levels. To assess the robustness of Language Models to software versions, we vary the number of provided APIs to simulate both the newest version and earlier version settings. Subsequently, we test 3 closed-source and 4 open-source LLMs using a benchmark that incorporates these robustness settings, aiming to evaluate how deviations impact LLMs' API calls for task completion. We find that GPT-4 exhibits the highest performance and strong robustness in our benchmark, particularly in the version update and the multilingual settings. However, we find that all LLMs lose their robustness when confronted with multiple challenges (e.g., multi-turn) simultaneously, leading to significant performance drops. We further analyze the robustness behavior and error reasons of LLMs in our benchmark, which provide valuable insights for researchers to understand the LLM's robustness in task completion and develop more robust LLMs and agents. We release the code and data at \url{https://github.com/ZekaiGalaxy/PPTCR}.Comment: LLM evaluation, Multi-turn, Multi-language, Multi-modal benchmar

What Motivates Users to Keep Using Social Mobile Payments?

Due to the rapid diffusion of social mobile payment (SMP), the current research explores the post-adoption behavior of SMP users. It proposes a research model to determine the core predictors of users’ continuance intentions to use SMPs. Through the analysis of survey data from South Korea, it indicates that satisfaction strongly and positively affects users’ continuance intentions. Moreover, satisfaction is influenced by perceived usefulness (PU), security, and enjoyment. Interestingly, although perceived ease of use (PEU) does not directly affect satisfaction, it can indirectly influence satisfaction via users’ PU and perceived enjoyment. Furthermore, perceived ubiquity has strong effects on users’ PU and PEU. The study also discusses meaningful implications, and provides suggestions for future studies

Teammate Familiarity in Distributed Computer-Supported Collaborative Learning: The Mediating Role of Social Presence

Owing to the limitations of computer-mediated communication (CMC), distributed CSCL (Computer-supported collaborative learning) has not always been as effective as desired. Despite recognizing the significance of group composition, the exploration of the function of teammate familiarity in distributed educational settings is restricted. This study explored the influence of teammate familiarity and social presence in a distributed CSCL setting by conducting an online survey of 288 Korean university students with experience in distributed CSCL. The results indicate that teammate familiarity increased the social presence experienced by students among their peers. Social presence subsequently enhanced teamwork satisfaction and, ultimately, increased self-assessed knowledge gain. More importantly, the relationship between teammate familiarity and teamwork satisfaction was mediated by social presence. Social media platforms and class webpages were the most widely used channels for students to get to know their teammates. Our study provided insights for improving the effectiveness of distributed CSCL and a framework for investigating social presence in satisfaction building in various contexts, including online education

Bibliometric study on environmental, social, and governance research using CiteSpace

This paper offers an overview of the status of and emerging trends in environmental, social, and governance (ESG) research through a bibliometric approach using CiteSpace. In particular, our study aimed to elucidate the overall intellectual structure of the environmental, social, and governance academic field. To this end, we performed a topic search related to the environmental, social, and governance field and gathered published articles (2007–2021) from the Web of Science. Subsequently, we identified productive authors, institutes, and countries/regions to determine main research forces in the environmental, social, and governance field. Additionally, we conducted a co-citation analysis to identify highly cited authors, journals, and literatures in the environmental, social, and governance field. Furthermore, we performed a literature-co-citation-based cluster analysis and literature citation burst analysis to confirm the main themes and hotspots of the environmental, social, and governance field. These analyses can contribute to the investigations of key contributing forces in the environmental, social, and governance field at the author, institution, country/region, and journal levels and provide insights into the knowledge structures and orientations of the environmental, social, and governance field for future research.Published versionThis research was supported by a National Research Foundation of Korea (NRF) grant funded by the Korean Government (NRF2020R1A2C1014957)