Evaluating Human-Language Model Interaction

Many real-world applications of language models (LMs), such as writing assistance and code autocomplete, involve human-LM interaction. However, most benchmarks are non-interactive in that a model produces output without human involvement. To evaluate human-LM interaction, we develop a new framework, Human-AI Language-based Interaction Evaluation (HALIE), that defines the components of interactive systems and dimensions to consider when designing evaluation metrics. Compared to standard, non-interactive evaluation, HALIE captures (i) the interactive process, not only the final output; (ii) the first-person subjective experience, not just a third-party assessment; and (iii) notions of preference beyond quality (e.g., enjoyment and ownership). We then design five tasks to cover different forms of interaction: social dialogue, question answering, crossword puzzles, summarization, and metaphor generation. With four state-of-the-art LMs (three variants of OpenAI's GPT-3 and AI21 Labs' Jurassic-1), we find that better non-interactive performance does not always translate to better human-LM interaction. In particular, we highlight three cases where the results from non-interactive and interactive metrics diverge and underscore the importance of human-LM interaction for LM evaluation.Comment: Authored by the Center for Research on Foundation Models (CRFM) at the Stanford Institute for Human-Centered Artificial Intelligence (HAI

Games based learning in mathematics education: A systematic literature review

This study aims to describe and evaluate the effects of game-based mathematics learning. Systematic Literature Review (SLR) is used as a method in this research by collecting primary studies through checking the Google Scholar database, Education Resources Information Center (ERIC) from 2013 to 2022 with the keywords "games and learning mathematics" or "games and mathematics education", resulting in 25 relevant primary studies with inclusion criteria. The analysis technique used for the articles or primary studies that have been collected is that each primary study is recorded, then classified by title and year of study. Furthermore, each of the articles included in this study was analyzed to obtain the findings and discussion required to draw conclusions from this SLR study. It shows several results from the 25 articles used as material for the analysis of game-based mathematics learning, namely: 1) There are two types of games that are applied in the process of learning mathematics, namely traditional games, and modern games. 2) In general, the influence of the application of games, both traditional and modern, on learning mathematics is to improve the quality of learning with better indicators of teacher skills and student activity. From the results of this analysis, it can be concluded that the application of games in the process of learning mathematics is effective, because it affects the quality of learning mathematics in schools, especially by increasing student competence. The findings of this study also contribute to providing information related to the trend of game-based mathematics learning in Indonesia, which is expected to become recommendations and ideas for further research.Penelitian ini bertujuan untuk mendeskripsikan dan mengevaluasi pengaruh pembelajaran matematika berbasis permainan. Systematic Literature Review (SLR) digunakan sebagai metode dalam penelitian ini dengan mengumpulkan studi primer melalui pemeriksaan database Google Scholar, Education Resources Information Center (ERIC) dari tahun 2013 hingga 2022 dengan kata kunci “permainan dan pembelajaran matematika” atau “game and mathematic education” sehingga menghasilkan 25 studi primer yang relevan dengan kriteria inklusi. Teknik analisis yang digunakan terhadap artikel atau studi primer yang telah dikumpulkan yaitu setiap studi primer didata, lalu diklasifikasikan berdasarkan judul, dan tahun studi. Selanjutnya, dilakukan analisis satu per satu artikel yang disertakan dalam studi ini untuk memperoleh temuan dan pembahasan hingga kesimpulan dalam penelitian SLR ini. Dari 25 artikel yang digunakan sebagai bahan analisis tentang pembelajaran matematika berbasis permainan menunjukkan beberapa hasil, yaitu: 1) Terdapat dua jenis permainan yang diterapkan dalam proses pembelajaran matematika, yaitu permainan tradisional dan permainan modern. 2) Secara general pengaruh penerapan permainan, baik tradisional maupun modern terhadap pembelajaran matematika yaitu meningkatkan kualitas pembelajaran dengan indikator keterampilan guru dan aktivitas siswa yang lebih baik. Dari hasil analisis tersebut dapat disimpulkan bahwa penerapan permainan dalam proses pembelajaran matematika sudah efektif, karena berpengaruh terhadap kualitas belajar matematika di sekolah, khususnya peningkatan kompetensi siswa. Temuan penelitian ini juga berkontribusi memberikan informasi yang berkaitan dengan tren pembelajaran matematika berbasis permainan di Indonesia sehingga diharapkan dapat menjadi rekomendasi maupun ide untuk penelitian selanjutnya

Kenyon College Magazine - Summer 2022

https://digital.kenyon.edu/kcab/1302/thumbnail.jp

ACMS 18th Biennial Conference Proceedings

Association of Christians in the Mathematical Sciences 18th Biennial Conference Proceedings, June 1-4, 2011, Westmont College, Santa Barbara, CA

Symphony. Alleviating depression symptoms through science based video gaming

A depressão representa um grande encargo econômico na Europa e em todo o mundo, é esperado que se torne a segunda doença mental mais comum em todo o mundo até 2020. Os tratamentos de primeira linha incluem medicação e psicoterapia, os quais podem produzir efeitos colaterais indesejáveis, ou serem ineficazes por várias razões que vão da escassez de profissionais treinados, até ao desconforto dos pacientes com os resultados. Terapias alternativas, como a música e terapias baseadas em tecnologia, têm sido propostas como tratamentos alternativos. Esta tese propõe o uso de um jogo casual afim de ajudar pessoas com sintomatologia depressiva. Foi realizado um estudo inicial com uma amostra de indivíduos com sintomas de depressão, leve a moderada, para entender a sua relação com videojogos, tendências de jogos e qual o tipo preferido de jogos. Concluímos que jogando no seu tempo livre, sozinhos e num computador ou dispositivo móvel tiveram maior preferência. Com base na informação coletada, criamos um jogo para dispositivos móveis chamado Symphony, baseado em resolver puzzles e audição musical, como meio de fornecer dicas para o bem-estar mental. Tentamos também promover a estimulação cognitiva, a melhoria de humor, e estratégias de regulação de emoções. Um teste de usabilidade foi realizado com 5 indivíduos da população geral para testar sua jogabilidade, onde eles preencheram questionários pré e pós-intervenção para avaliar seu humor e perceção do jogo. Uma experiência final foi realizada com uma amostra da população-alvo de 8 voluntários, que também completaram questionários pré e pós-intervenção para avaliar o impacto de jogar Symphony em seus níveis de depressão, humor e também para conhecer sua perceção do jogo. Os dados dos questionários e os comentários e observações informais dos terapeutas, revelaram respostas emocionais positivas e mudanças de humor resultantes de jogar Symphony. As quais, sugerem melhorias para tornar Symphony mais eficaz e adaptável a cada indivíduo.Depression is a heavy economic burden in Europe and worldwide, and it is expected to reach second place as the most common mental illness worldwide by 2020. First-line treatments include medication and psychotherapy, but they may either bring undesirable side-effects or be ineffective for several reasons that go from the shortage of trained practitioners to lack of compliance. Alternative therapies, such as music and technology based therapies, have been proposed as adjunctive treatments. This thesis proposes using a casual-game to aid people with depressive symptomatology. We conducted an initial study with a sample of individuals with mild to moderate depression symptoms to understand their regards to video-games, gaming tendencies, and preferred type of games. Playing in their free time, alone and on a computer or mobile device were mostly preferred, so we created a mobile casual game called Symphony based on puzzle-playing and music-listening to give tips for their mental well-being, promote cognitive stimulation, mood improvement, and emotion regulation strategies. A usability test conducted with 5 individuals from the general population was conducted to test its playability, where they completed pre- and post-intervention questionnaires to assess their mood and perception of the game. A final experiment was conducted with a sample of the target population of 8 volunteers, who also completed pre- and post-intervention questionnaires to assess the impact of playing Symphony on their depression levels, mood, and also to gather their perception of the game. Questionnaire data and therapists’ informal comments and observations reveal overall positive emotional responses and mood changes resulting from playing Symphony, and suggest improvements to make Symphony more effective and adaptable to each individual

Hypatiamat - I want to solve questions about...

Dissertação de mestrado em Informatics EngineeringHypatiamat is a Portuguese project comprised of several applications that aim to develop the Math skills of students from the 1st through 9th grades (Basic Education). The ingraining of mental calculation strategies, numbering systems, and logical operations lead to a better success rate in this subject in later years. One of the project’s components is the online platform (https://www.hypatiamat.com), which aims to foster autonomous learning through more interactive practices due to the current ease of technological access in this age group, by trying to appropriate teaching to everyday life. Several tools are made available, such as videos, tutorials, explanations, questions, etc. on various Math topics that students can easily access at any time. Teachers that aim to enhance their students’ learning process using this digital approach can exercise it in multiple applications provided by the platform, where the interactions are carried out and controlled through these means. The monolithic architecture (written in PHP) has received contributions from multiple developers over the years in order to address the scalability issues introduced with this platform’s growing popularity, which thus far demanded manual efforts for maintenance and content insertion. As such, there has been an incremental process of modernization, turning the various constituent applications into distinct microservices. "I Want to Solve Questions About..." is one of these applications where students are provided with a large selection of questions in the form of mini-games (multiple choice, true or false, ...), regarding the themes mentioned above. The first objective of the dissertation is to develop a back-office that allows the teachers in charge of the project to manage existing questions as well as add new ones for the students, since the current process requires updating the database manually. The second one is the modernization of the application’s interface at the technological level, by making use of adequate frameworks and programming languages and at the user level, by making an effort to maintain the intuitive workflow that led to its popularity but with a modernized design, in order to be consistent with other online tools.O Hypatiamat é um projeto português constituído por várias aplicações que visa desenvolver as aptidões, na disciplina de Matemática, de alunos do 1º ao 9º ano de escolaridade (Educação Básica). O enraizamento de estratégias de cálculo mental, sistemas de numeração e operações lógicas originam uma melhor taxa de sucesso nesta disciplina em anos posteriores. Uma das componentes deste projeto é a plataforma online (https://www.hypatiamat.com), cujo propósito é fomentar a aprendizagem autónoma através de práticas mais interativas, devido à facilidade de acesso tecnológico atual desta faixa etária, tentando apropriar o ensino ao quotidiano. São disponibilizadas várias ferramentas, tais como vídeos, tutoriais, explicações, questões, etc sobre os vários temas da Matemática (Ensino Básico) que os alunos podem facilmente aceder a qualquer momento. Professores que pretendam enriquecer a aprendizagem dos seus alunos com esta metodologia digital podem exercê-lo nas várias aplicações que a plataforma disponibiliza, onde a interação é realizada e controlada através destes meios. A arquitetura monolítica (escrita em PHP) tem recebido contribuições de vários desenvolvedores ao longo dos anos de modo a colmatar os problemas de escalabilidade introduzidos com a popularidade crescente desta plataforma, que até agora exigia esforço manual para manutenção e inserção de conteúdo. Assim, tem existido um processo incremental de modernização, tornando as várias aplicações constituintes em microsserviços distintos. A "Quero resolver questões de..." é uma destas aplicações, onde são disponibilizadas aos alunos várias questões, sob a forma de mini-jogos (escolha múltipla, verdadeiro ou falso, ...), relativas aos temas mencionados anteriormente. O primeiro objetivo da dissertação é o desenvolvimento de um backoffice que permita aos professores responsáveis gerirem as questões existentes assim como adicionarem novas para os alunos, visto que o processo atual obriga a atualização manual na base de dados. O segundo é a modernização da interface da aplicação ao nível: tecnológico, utilizando frameworks e linguagens de programação adequadas ao problema; do utilizador, de modo a manter o fluxo intuitivo que gerou a sua popularidade mas tendo em conta um design mais atualizado para manter a consistência com outras ferramentas online

Digital Technologies for Teaching English as a Foreign/Second Language: a collective monograph

Колективна монографія розкриває різні аспекти використання цифрових технологій у навчанні англійської мови як іноземної/другої мови (цифровий сторітелінг, мобільні застосунки, інтерактивне навчання і онлайн-ігри, тощо) та надає освітянам і дослідникам ресурс для збагачення їхньої професійної діяльності. Окрема увага приділена цифровим інструментам для впровадження соціально-емоційного навчання та інклюзивної освіти на уроках англійської мови. Для вчителів англійської мови, методистів, викладачів вищих закладів освіти, науковців, здобувачів вищої освіти