Deep Learning Approach for cognitive competency assessment in Computer Programming subject

This research examines the competencies that are essential for an lecturer or instructor to evaluate the student based on automated assessments. The competencies are the skills, knowledge, abilities and behavior that are required to perform the task given, whether in a learning or a working environment. The significance of this research is that it will assist students who are having difficulty learning a Computer Programming Language course to identify their flaws using a Deep Learning Approach. As a result, higher education institutions have a problem with assessing students based on their competency level because; they still use manual assessment to mark the assessment. In order to measure intelligence, it is necessary to identify the cluster of abilities or skills of the type in which intelligence expresses itself. This grouping of skills and abilities referred to as "competency". Then, an automated assessment is a problem-solving activity in which the student and the computer interact with no other human intervention. This review focuses on collecting different techniques that have been used. In addition, the review finding shows the main gap that exists within the context of the studied areas, which contributes to our key research topic of interest

Research on Risk Prediction and Early Warning of Human Resource Management Based on Machine Learning and Ontology Reasoning

Talent is the first resource, the development of the enterprise to retain key talent is essential, the main research is based on machine learning and ontological reasoning, human resources analysis and management risk prediction and early warning methods, first of all, according to the specific situation and the target case, through the calculation of the similarity of the concept name and attribute of the similarity assessment of the source case in the case library, the matching of knowledge-based employees of the company\u27s case for the similarity prediction and human resources management risk prediction research. Then, according to the evaluation results, we can find out the most suitable job matches in specific risk problems and situations. This is a solution to the target cases and criteria for companies to evaluate candidates. Second, we have successfully developed and implemented a prediction model that applies machine learning to the early warning study of risk prediction for HR management. The model is optimized with a cross-validation function, and the convergence of the model training is accelerated by the regularization of Newton\u27s iterative method. Finally, our prediction model achieved 82% yield. Ontological reasoning and machine learning are promising in human resource management risk prediction and warning, which is proved by the high accuracy rate verified by examples. Finally, we analyze the proposed results of HRM risk prediction and early warning to contribute to the improvement of risk control and suggest measures for possible risks

Learning Latent Characteristics of Data and Models using Item Response Theory

A supervised machine learning model is trained with a large set of labeled training data, and evaluated on a smaller but still large set of test data. Especially with deep neural networks (DNNs), the complexity of the model requires that an extremely large data set is collected to prevent overfitting. It is often the case that these models do not take into account specific attributes of the training set examples, but instead treat each equally in the process of model training. This is due to the fact that it is difficult to model latent traits of individual examples at the scale of hundreds of thousands or millions of data points. However, there exist a set of psychometric methods that can model attributes of specific examples and can greatly improve model training and evaluation in the supervised learning process. Item Response Theory (IRT) is a well-studied psychometric methodology for scale construction and evaluation. IRT jointly models human ability and example characteristics such as difficulty based on human response data. We introduce new evaluation metrics for both humans and machine learning models build using IRT, and propose new methods for applying IRT to machine learning-scale data. We use IRT to make contributions to the machine learning community in the following areas: (i) new test sets for evaluating machine learning models with respect to a human population, (ii) new insights about how deep-learning models learn by tracking example difficulty and training conditions, and (iii) new methods for data selection and curriculum building to improve model training efficiency, (iv) a new test of electronic health literacy built with questions extracted from de-identified patient Electronic Health Records (EHRs). We first introduce two new evaluation sets built and validated using IRT. These tests are the first IRT test sets to be applied to natural language processing tasks. Using IRT test sets allows for more comprehensive comparison of NLP models. Second, by modeling the difficulty of test set examples, we identify patterns that emerge when training deep neural network models that are consistent with human learning patterns. Specifically, as models are trained with larger training sets, they learn easy test set examples more quickly than hard examples. Third, we present a method for using soft labels on a subset of training data to improve deep learning model generalization. We show that fine-tuning a trained deep neural network with as little as 0.1% of the training data can improve model generalization in terms of test set accuracy. Fourth, we propose a new method for estimating IRT example and model parameters that allows for learning parameters at a much larger scale than previously available to accommodate the large data sets required for deep learning. This allows for learning IRT models at machine learning scale, with hundreds of thousands of examples and large ensembles of machine learning models. The response patterns of machine learning models can be used to learn IRT example characteristics instead of human response patterns. Fifth, we introduce a dynamic curriculum learning process that estimates model competency during training to adaptively select training data that is appropriate for learning at the given epoch. Finally, we introduce the ComprehENotes test, the first test of EHR comprehension for humans. The test is an accurate measure for identifying individuals with low EHR note comprehension ability, and validates the effectiveness of previously self-reported patient comprehension evaluations

Computing Competencies for Undergraduate Data Science Curricula: ACM Data Science Task Force

At the August 2017 ACM Education Council meeting, a task force was formed to explore a process to add to the broad, interdisciplinary conversation on data science, with an articulation of the role of computing discipline-specific contributions to this emerging field. Specifically, the task force would seek to define what the computing/computational contributions are to this new field, and provide guidance on computing-specific competencies in data science for departments offering such programs of study at the undergraduate level. There are many stakeholders in the discussion of data science – these include colleges and universities that (hope to) offer data science programs, employers who hope to hire a workforce with knowledge and experience in data science, as well as individuals and professional societies representing the fields of computing, statistics, machine learning, computational biology, computational social sciences, digital humanities, and others. There is a shared desire to form a broad interdisciplinary definition of data science and to develop curriculum guidance for degree programs in data science. This volume builds upon the important work of other groups who have published guidelines for data science education. There is a need to acknowledge the definition and description of the individual contributions to this interdisciplinary field. For instance, those interested in the business context for these concepts generally use the term “analytics”; in some cases, the abbreviation DSA appears, meaning Data Science and Analytics. This volume is the third draft articulation of computing-focused competencies for data science. It recognizes the inherent interdisciplinarity of data science and situates computing-specific competencies within the broader interdisciplinary space

Exploring the Potential of Large Language models in Traditional Korean Medicine: A Foundation Model Approach to Culturally-Adapted Healthcare

Introduction: Traditional Korean medicine (TKM) emphasizes individualized diagnosis and treatment, making AI modeling difficult due to limited data and implicit processes. GPT-3.5 and GPT-4, large language models, have shown impressive medical knowledge despite lacking medicine-specific training. This study aimed to assess the capabilities of GPT-3.5 and GPT-4 for TKM using the Korean National Licensing Examination for Korean Medicine Doctors. Methods: GPT-3.5 (February 2023) and GPT-4 (March 2023) models answered 340 questions from the 2022 examination across 12 subjects. Each question was independently evaluated five times in an initialized session. Results: GPT-3.5 and GPT-4 achieved 42.06% and 57.29% accuracy, respectively, with GPT-4 nearing passing performance. There were significant differences in accuracy by subjects, with 83.75% accuracy for neuropsychiatry compared to 28.75% for internal medicine (2). Both models showed high accuracy in recall-based and diagnosis-based questions but struggled with intervention-based ones. The accuracy for questions that require TKM-specialized knowledge was relatively lower than the accuracy for questions that do not GPT-4 showed high accuracy for table-based questions, and both models demonstrated consistent responses. A positive correlation between consistency and accuracy was observed. Conclusion: Models in this study showed near-passing performance in decision-making for TKM without domain-specific training. However, limits were also observed that were believed to be caused by culturally-biased learning. Our study suggests that foundation models have potential in culturally-adapted medicine, specifically TKM, for clinical assistance, medical education, and medical research.Comment: 31 pages, 6 figure

Current and Future Artificial Intelligence (AI) Curriculum in Business School: A Text Mining Analysis

As artificial intelligence (AI) becomes one of the most important driving forces in industrial innovations, more business schools, mostly in graduate programs, are introducing AI in their curricula, particularly in information systems (IS) curricula. However, there appears to be a paucity of research on the AI curriculum. This study examines the current status of the AI curriculum in both undergraduate and graduate business schools and provides recommendations for future AI curriculum development. The study develops a technical competency model for AI curriculum based on both MSIS2016 - Global Competency Model for Graduate Degree Programs in Information Systems and IS2020 - A Competency Model for Undergraduate Programs in Information Systems and the AI technical competencies. Using text mining analysis, we collected and analyzed AI courses from the top 46 business schools at both undergraduate and graduate levels, ranked by US News in 2020. The findings indicate that machine learning is at the core of the AI curriculum in business, and most AI curricula are a hybrid of AI and data analytics. This acknowledges that the AI curriculum is still at its early stage, and business schools are closely adhering to the industrial development trend. The proposed technical competency model for AI curriculum can serve as a guideline for future AI curriculum development in business schools. We hope this study provides systematic insight into AI curriculum and offers recommendations for business education, in IS programs specifically

A Comprehensive Survey of Artificial Intelligence Techniques for Talent Analytics

In today's competitive and fast-evolving business environment, it is a critical time for organizations to rethink how to make talent-related decisions in a quantitative manner. Indeed, the recent development of Big Data and Artificial Intelligence (AI) techniques have revolutionized human resource management. The availability of large-scale talent and management-related data provides unparalleled opportunities for business leaders to comprehend organizational behaviors and gain tangible knowledge from a data science perspective, which in turn delivers intelligence for real-time decision-making and effective talent management at work for their organizations. In the last decade, talent analytics has emerged as a promising field in applied data science for human resource management, garnering significant attention from AI communities and inspiring numerous research efforts. To this end, we present an up-to-date and comprehensive survey on AI technologies used for talent analytics in the field of human resource management. Specifically, we first provide the background knowledge of talent analytics and categorize various pertinent data. Subsequently, we offer a comprehensive taxonomy of relevant research efforts, categorized based on three distinct application-driven scenarios: talent management, organization management, and labor market analysis. In conclusion, we summarize the open challenges and potential prospects for future research directions in the domain of AI-driven talent analytics.Comment: 30 pages, 15 figure