Validating the Automated Assessment of Participation and of Collaboration in Chat Conversations

International audienceAs Computer Supported Collaborative Learning (CSCL) gains a broader usage as a viable alternative to classic educational scenarios, the need for automated tools capable of supporting tutors in the time consuming process of analyzing conversations becomes more stringent. Moreover, in order to fully explore the benefits of such scenarios, a clear demarcation must be made between participation or active involvement, and collaboration that presumes the intertwining of ideas or points of view with other participants. Therefore, starting from a cohesion-based model of the discourse, we propose two computational models for assessing collaboration and participation. The first model is based on the cohesion graph and can be perceived as a longitudinal analysis of the ongoing conversation, thus accounting for participation from a social knowledge-building perspective. In the second approach, collaboration is regarded from a dialogical perspective as the intertwining or overlap of voices pertaining to different speakers, therefore enabling a transversal analysis of subsequent discussion slices

Concept-based topic model improvement

We propose a system which employs conceptual knowledge to improve topic models by removing unrelated words from the simplified topic description. We use WordNet to detect which topical words are not conceptually similar to the others and then test our assumptions against human judgment. Results obtained on two different corpora in different test conditions show that the words detected as unrelated had a much greater probability than the others to be chosen by human evaluators as not being part of the topic at all. We prove that there is a strong correlation between the said probability and an automatically calculated topical fitness and we discuss the variation of the correlation depending on the method and data used. © 2011 Springer-Verlag Berlin Heidelberg

Document Cohesion Flow: Striving towards Coherence

Text cohesion is an important element of discourseprocessing. This paper presents a new approach to modeling,quantifying, and visualizing text cohesion using automatedcohesion flow indices that capture semantic links amongparagraphs. Cohesion flow is calculated by applyingCohesion Network Analysis, a combination of semanticdistances, Latent Semantic Analysis, and Latent DirichletAllocation, as well as Social Network Analysis. Experimentsperformed on 315 timed essays indicated that cohesion flowindices are significantly correlated with human ratings of textcoherence and essay quality. Visualizations of the globalcohesion indices are also included to support a more facileunderstanding of how cohesion flow impacts coherence interms of semantic dependencies between paragraphs

Improving topic evaluation using conceptual knowledge

The growing number of statistical topic models led to the need to better evaluate their output. Traditional evaluation means estimate the model's fitness to unseen data. It has recently been proven than the output of human judgment can greatly differ from these measures. Thus the need for methods that better emulate human judgment is stringent. In this paper we present a system that computes the conceptual relevance of individual topics from a given model on the basis of information drawn from a given concept hierarchy, in this case WordNet. The notion of conceptual relevance is regarded as the ability to attribute a concept to each topic and separate words related to the topic from the unrelated ones based on that concept. In multiple experiments we prove the correlation between the automatic evaluation method and the answers received from human evaluators, for various corpora and difficulty levels. By changing the evaluation focus from a statistical one to a conceptual one we were able to detect which topics are conceptually meaningful and rank them accordingly

ReaderBench: An Integrated Cohesion-Centered Framework

Dascalu, M., Stavarache, L.L., Dessus, P., Trausan-Matu, S., McNamara, D.S., & Bianco, M. (2015). ReaderBench: An Integrated Cohesion-Centered Framework. In G. Conole, T. Klobucar, C. Rensing, J. Konert & É. Lavoué (Eds.), 10th European Conf. on Technology Enhanced Learning (pp. 505–508). Toledo, Spain: Springer.ReaderBench is an automated software framework designed to support both students and tutors by making use of text mining techniques, advanced natural language processing, and social network analysis tools. ReaderBench is centered on comprehension prediction and assessment based on a cohesion-based representation of the discourse applied on different sources (e.g., textual materials, behavior tracks, metacognitive explanations, Computer Supported Collaborative Learning – CSCL – conversations). Therefore, Reader‐Bench can act as a Personal Learning Environment (PLE) which incorporates both individual and collaborative assessments. Besides the a priori evaluation of textual materials’ complexity presented to learners, our system supports the identification of reading strategies evident within the learners’ self-explanations or summaries. Moreover, ReaderBench integrates a dedicated cohesion-based module to assess participation and collaboration in CSCL conversations.This study is part of the RAGE project. The RAGE project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 644187. This publication reflects only the author's view. The European Commission is not responsible for any use that may be made of the information it contains

Document Cohesion Flow: Striving towards Coherence

Abstract Text cohesion is an important element of discourse processing. This paper presents a new approach to modeling, quantifying, and visualizing text cohesion using automated cohesion flow indices that capture semantic links among paragraphs. Cohesion flow is calculated by applying Cohesion Network Analysis, a combination of semantic distances, Latent Semantic Analysis, and Latent Dirichlet Allocation, as well as Social Network Analysis. Experiments performed on 315 timed essays indicated that cohesion flow indices are significantly correlated with human ratings of text coherence and essay quality. Visualizations of the global cohesion indices are also included to support a more facile understanding of how cohesion flow impacts coherence in terms of semantic dependencies between paragraphs

Methods of negotiation in online collaborative mathematics problem-solving

The Virtual Math Teams (VMT) project is an NSF-funded research program through which researchers at the College of Information Science and Technology (IST) and the Math Forum investigate the innovative use of online collaborative environments to support effective K-12 mathematics learning. Negotiation is an indispensable element in collaboration, especially in the resolution of situations whose complexity is such that understanding them is enhanced by multiple perspectives made possible through collaboration. Some mathematics problems which are described as ‘ill-structured’ because they admit of several possible interpretations and multiple solutions, are candidates for collaborative problem-solving. Negotiation is necessary to make this collaborative approach possible, inasmuch as sense- and meaning-making associated with this particular approach to problem-solving has for its distinctive characteristic, the dynamics of a group rather than the jig-saw style of cooperating individuals. A research methodology based on conversation analysis is used to identify how negotiation is conducted in online mathematics problem-solving by locating its methods as negotiation is initiated, processed and concluded. Initial findings show: - how changes in the tempo of turn-taking indicate shifts in the conduct of problem-solving, - how repetition and ‘forced conclusions’ are used to negotiate a choice among several problems at hand, - how intervals and transitions are used by participants to indicate preferred responses and responders, - how referencing tools impact negotiation. Findings from this research can assist in the design of CSCL environments, especially in the transformation of generic communication applications to collaborative technology

Virtual math teams (VMT): Continuity and sustainability of collaborative knowledge building

How can we support online groups to better sustain their collaborative learning activity over time? In this poster, we present a qualitative case study of group creativity online in the domain of mathematics. We define collaborative learning work broadly, ranging from the micro-level co-construction of novel resources for collective problem solving to the innovative reuse and expansion of ideas across multiple virtual teams. Based on our analysis of the collaborative interactions of five virtual math teams we characterize the relationship between "synchronic" aspects of collaborative knowledge building (i.e. single episode interactions) and their "diachronic" evolution across time and across collectivities. Our analysis indicates that the synergy between these two types of interactions and the resulting creative engagement of the teams relies on three fundamental processes: (1) referencing and the “configuration of indexicals”, (2) collective remembering, and (3) bridging across discontinuities. In addition we also reflect on the aspects of the online environment used by these virtual teams which promote, support or hinder diachronic and synchronic interactions leading to successful knowledge building

Analisis Pelaksanaan Audit Internal terhadap Pengendalian Internal Penerimaan & Pengeluaran Kas dalam Rangka Menemukan Fraud - Kasus pada PT XYZ

Penelitian ini bertujuan untuk mengetahui apakah teknik audit internal yang diterapkan oleh PT XYZ di Jakarta sebagai aktivitas pemeriksaan dan penilaian Perusahaan secara menyeluruh yang membantu semua tingkatan manajemen dalam melaksanakan tanggung jawabnya telah berjalan secara memadai pada penerimaan kas dan pengeluaran kas. Dalam penelitian ini metode analisis data yang digunakan adalah pendekatan kualitatif. Data primer didapat dari responden melalui observasi, kuesioner, dan wawancara yang berprofesi sebagai auditor internal. Data sekunder didapat melalui dokumen yang berasal dari Perusahaan. Berdasarkan penelitian yang telah dilakukan pada PT XYZ di Jakarta, dapat disimpulkan bahwa inspeksi sebagai teknik audit internal yang digunakan dalam menemukan fraud pada penerimaan kas dan pengeluaran kas telah berjalan secara memadai. Kata kunci: teknik audit internal, penerimaan kas, pengeluaran kas This research aims to determine whether the internal audit technique applied by XYZ Company in Jakarta as the activity and examination of the company\u27s overall assessment that helps all levels of management in carrying out its responsibilities adequately run well on the cash receipt and cash disbursement. In this research, data analysis method used is the qualitative approach. The primary data obtained from the respondents by doing observation, questionnaires, and interviews who works as internal auditor. And then, the secondary data obtained through documents derived from the company. Based on the research which has been done at the XYZ Company in Jakarta, it could be concluded that inspection as the internal audit technique which used in detecting fraud at cash receipt and cash disbursement has been adequately well. Keywords: Internal audit technique, cash receipt, cash disbursement Daftar Pustaka Agoes, Sukrisno. (2008). Auditing: (Pemeriksaan Akuntan) oleh Kantor Akuntan Publik, Jilid I, Edisi 3. Jakarta: Fakultas Ekonomi Universitas Indonesia. Arens, Alvin A., Elder, Randal J., Beasley, Mark S. (2014). Auditing and Assurance Services: An Integrated Approach, Edisi 15. England: Pearson Education Limited. Badan Pengendalian Internal Universitas Muhammadiyah Malang (26 Maret, 2013). Identifikasi Bentuk-Bentuk Penggelapan. 25 September, 2014. http://bpi.umm.ac.id/home.php?c=1710-5 Badan Pengawas Keuangan Dan Pembangunan. (2008). Fraud Auditing. Edisi 5. Bogor: Pusdiklatwas BPKP. Boynton, William C., Jhonson, Raymond N., & Kell, Walter G. (2003). Modern Auditing, Jilid II, Edisi 7. Jakarta: Erlangga. Bungin, Burhan. (2010). Penelitian Kualitatif: Komunikasi, Ekonomi, Kebijakan Publik dan Ilmu Sosial Lainnya. Jakarta: Kencana Prenama Media Group. Champion, Dean J. (1990). Basic Statistic For Social Research. San Fransisco: Chandler Publisher. Ekawati, Rian. (5 Juni, 2013). Akuntansi dan Pengendalian Intern Terhadap Kas. 22 September, 2014. http://www.slideshare.net/ekawatirian/bab-1-akuntansi-dan-pengendalian-intern-terhadap-kas Fahmi, Wahyudin, & Yuliawati. (8 Desember 2006). Bapepam: Akuntan dan Great River Terlibat Konspirasi. 9 Oktober, 2014. http://www.tempo.co/read/news/2006/12/08/05689121/Bapepam-Akuntan-dan-Great-River-Terlibat-Konspirasi Halim, Abdul. (2001). Auditing I: Dasar-Dasar Audit Laporan Keuangan. Yogyakarta: Unit Penerbit dan Percetakan AMP YKPN. Hall, James A. (2004). Sistem Informasi Akuntansi, Buku I. (Amir Abadi Jusuf, Penerjemah). Jakarta: Salemba Empat. Harahap, Sofyan S. (2001). Analisis Kritis Atas Laporan Keuangan, Edisi Revisi. Jakarta: Raja Garafindo Pustaka Utama. Harrison, Walter T., Horngren, Charles T., Thomas, C. William., Suwardy, Themin. (2012). Akuntansi Keuangan International Financial Reporting Standards-IFRS, Edisi 8, Jilid I. (Gina Gania, Penerjemah). Jakarta: Erlangga Horngren, Charles T., Harrison, Walter T., Oliver, M. Suzanne. (2009). Accounting, Edisi 8. New Jersey: Pearson. Institut Akuntan Publik Indonesia. (2008). Panduan Audit Entitas Bisnis Kecil. Jakarta: Salemba Empat. Karni, Soejono. (2000). Auditing Audit Khusus & Audit Forensik dalam Praktik. Jakarta: FE UI. Kieso, Donald E., Weygandt, Jerry J., & Warfield, Terry D. (2002). Akuntansi Intermediate, Edisi 10, Jilid 1. (Emil Salim, Penerjemah). Jakarta: Erlangga. Konsorsium Organisasi Profesi Audit Internal. (2004). Standar Profesional Audit Internal, Jilid 1. Jakarta: Yayasan Pendidikan Internal Audit. Kusnadi. (2000). Akuntansi Keuangan Menengah (Intermediate), Prinsip, Prosedur & metode, Edisi I. Malang: Universitas Brawijaya. La Midjan, & Susanto, Azhar. (2001). Sistem Informasi Akuntansi. Jakarta: PT Gramedia Pustaka Utama. Kumaat, Valery G. (2011). Internal Audit. Jakarta: Erlangga Marshall, Romney B., & Steinbart, Paul J. (2004). Akuntansi Sistem Informasi, Buku I, Edisi 9. (Widyanahar, Penerjemah). Jakarta: Salemba Empat. Messier, William F., Glover, Steven M., & Prawitt, Douglas F. (2005). Jasa Audit dan Assurance Pendekatan Sistematis, Buku 2, Edisi 4. Jakarta: Salemba Empat. Moleong, Lexy J. (2008). Metodologi Penelitian Kualitatif. Bandung: Remaja Rosdakarya. Moeller, R., Brinks. (2005). Modern Internal Auditing. New Jersey: John Wiley & Sons. Mulyadi, Puradiredja. (2002). Auditing, Edisi 6. Jakarta: Salemba Empat. Mulyadi. (2008). Sistem Akuntansi, Edisi 3, Cetakan 4. Jakarta: Salemba Empat. Mulyadi. (2010). Auditing, Jilid I, Cetakan 7. Jakarta: Salemba Empat. Undang-Undang Republik Indonesia Nomor 19 Tahun 2003 Tentang Badan Usaha Milik Negara Peraturan Pemerintah Republik Indonesia Nomor 45 Tahun 2005 Tentang Pendirian, Pengurusan, Pengawasan Dan Pembubaran Badan Usaha Milik Negara Pickett, K.H. Spencer. (2005). The Essential Handbook of Internal Auditing. New Jersey: John Wiley and Sons. Prianti, Martina. (7 November, 2007). Penggelapan Pajak Asian Agri Harus Masuk Pengadilan: Semua Aset Perusahaan Pemilik Asian Agri Bisa Dibekukan. 26 September, 2014. http://www.ortax.org/ortax/?mod =berita&page=show&id=548&q=agri&hlm=63 Sawyer, Lawrence B., Dittenhofer, Mortimer A., & Scheiner, James H. (2005). Audit Internal Sawyer, Edisi 5, Buku 1. (Desi Adhariani, Penerjemah). Jakarta: Salemba Empat. Sekaran, Uma. (2011). Metode Penelitian untuk Bisnis, Edisi 4, Buku 2. (Kwan Men Yon, Penerjemah). Jakarta: Salemba Empat. Smith, Jay M., Skousen, K. Fred., (1996). Akuntansi Intermediate, Edisi 9, Jilid I. Jakarta: Erlangga. Soemarso. (2002). Akuntansi Suatu Pengantar. Jakarta: Salemba Empat. Subaweh, Imam. (20 juni, 2014), Teknik Audit Berbantuan Komputer. 26 September, 2014. http://imas.staff.gunadarma.ac.id/Downloads/files/3559 5/3-Teknik+Audit.pdf Sugiyono. (2010). Metode Penelitian Kuantitatif Kualitatif R & D. Bandung: Alfabeta. Sugiyono. (2012). Memahami Penelitian Kualitatif. Bandung: Alfabeta. Tuanakotta, Theodorus M. (2001). Teori Akuntansi, Buku 2, Edisi 8. Jakarta: Fakultas Ekonomi Universitas Indonesia. The International Professional Practices Framework. (2009). Definisi Audit Internal Pada IPPF 2009. 5 September, 2014. http://auditorinternal.com/2010/01/18/definisi-audit-internal-pada-ippf-2009/ Tugiman, Hiro. 2006. Standar Profesional Audit Internal. Yogyakarta: Kanisius. Tunggal, Amin W. (2012). The Fraud Audit Mencegah dan Mendeteksi Kecurangan Akuntansi. Jakarta: Harvarindo

ReaderBench Learns Dutch: Building a Comprehensive Automated Essay Scoring System for Dutch Language

Automated Essay Scoring has gained a wider applicability and usage with the integration of advanced Natural Language Processing techniques which enabled in-depth analyses of discourse in order capture the specificities of written texts. In this paper, we introduce a novel Automatic Essay Scoring method for Dutch language, built within the Readerbench framework, which encompasses a wide range of textual complexity indices, as well as an automated segmentation approach. Our method was evaluated on a corpus of 173 technical reports automatically split into sections and subsections, thus forming a hierarchical structure on which textual complexity indices were subsequently applied. The stepwise regression model explained 30.5% of the variance in students’ scores, while a Discriminant Function Analysis predicted with substantial accuracy (75.1%) whether they are high or low performance students.This study is part of the RAGE project. The RAGE project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No 644187. This publication reflects only the author's view. The European Commission is not responsible for any use that may be made of the information it contains