ASSESSMENT OF RUBRIC-BASED EVALUATION BY NONPARAMETRIC MULTIPLE COMPARISONS IN FIRST-YEAR EDUCATION IN A JAPANESE UNIVERSITY

The rubrics have become a widely referenced and utilized form of assessment on campuses across internationally. But rubric can be an asset in any classroom and at any education level but it needs to be implemented correctly. Our research question in this study is whether students were evaluated consistently and equally from teacher to teacher using rubric. To answer this research question, we performed statistical estimation using nonparametric multiple comparisons. This article reports on a normalizing rubric evaluation by nonparametric multiple comparisons in a first-year course called “Manaburu I” offered at Kobe Tokiwa University. “Manaburu” is a word coined by us: “manabu” ‘learn’ in Japanese + English able. Thus, “Manaburu” refers to Self-Directed Learning I. In the course, about 20 teachers teach about 350 students (16–17 students per teacher). Students are organized into groups of about 6. It is of course difficult for 20 teachers to evaluate their students consistently among them, making this course an appropriate site for the evaluation. We constructed a rubric for the course, under which teachers were meant to evaluate students, and presented it to both teachers and students. Our research question was whether teachers evaluated students consistently and equally according to the Steel–Dwass estimation method, a strict statistical estimation method for nonparametric multiple comparisons. The results show that teachers do not evaluate students equally. Suggestions for future research, more attention to validity and reliability, a closer focus on learning and research on rubric use in higher education

THE FIRST-YEAR EXPERIENCE INCORPORATING THE ORGANIZATIONAL DEVELOPMENT APPROACH AT KOBE TOKIWA UNIVERSITY

In Japan, the first-year experience that rapidly gained attention at the beginning of the twenty-first century was clearly positioned in 2008 as formal undergraduate educational programs (The Central Council for Education, 2008). The term “first-year experience” is defined as a “comprehensive educational program primarily created for freshmen” to promote their smooth transition from high school to university and to create successful academic and social experiences at university (The Central Council for Education, 2008). Thus, the first-year experience is a specific program with activities implemented by diverse universities to fit the unique needs of their first-year university students (Tachi, 2008). One important issue of the first-year experience within the undergraduate program has been identifying ways to guarantee the quality of education. Kobe Tokiwa University’s four departments (medical technology, nursing, child education, and dental hygiene) require students to gain strong abilities to collaborate and cooperate in teams to be responsible for future team medical care or a school as a team. Therefore, in 2018, the university implemented a first-year experience program that incorporated the organizational development approach instead of the conventional human resource development approach. This article shares our experiences using a first-year experience program that incorporates the organizational development approach, and we discuss the potential of this approach for the first-year experience. To estimate the effectiveness of organizational development approach in FYE, we analyzed and compared the interim data that were reported on students in 2017 and 2018 using a text mining method. By introducing this “Organizational Development” approach into the students’ first-year educational curriculum, results suggest that it is possible to “deepen self-understanding” and “cooperate in self-understanding of others” at an early stage of a student’s enrollment. It is thought that this approach could become another effective method for universities to use for training professional persons as interpersonal aid workers

ANALYZING STUDENTS’ COURSE EVALUATIONS USING TEXT MINING: VISUALIZATION OF OPEN-ENDED RESPONSES IN A CO-OCCURRENCE NETWORK

Japan’s Standards for Establishment of Universities states, “A university shall conduct organized training and research to improve the content and methodology used in courses at said university.” Based on this, most of Japan’s universities have recently implemented course evaluations by students. Student course evaluations are intended to quantify and provide an understanding of students’ satisfaction with their courses, and all universities are implementing them as one way to objectively evaluate courses. These course evaluations often combine computer-graded multiple-choice items with open-ended items. Computer-graded multiple-choice items are easy to assess because the responses are quantifiable. However, open-ended items’ responses are text data, and objectively grasping the students’ general tendencies is challenging. Moreover, it is difficult to avoid risking arbitrary and subjective interpretations of the data by the analysts who summarize them. Therefore, to avoid these risks as much as possible, the so-called “text-mining” method or “quantitative content analysis” approach might be useful. This study shares our experiences using text mining to analyze students’ course evaluations through the visualization of their open-ended responses in a co-occurrence network, and we discuss the potential of this method.&nbsp

NEW ANALYSIS OF VISUALIZATION IN EDUINFORMATICS USING A NETWORK WITH PARAMETRIC AND NONPARAMETRIC CORRELATION COEFFICIENTS WITH THRESHOLD

Eduinformatics, a new term coined by us, is a field that combines education and informatics, and novel techniques will need to be developed for this field. Earlier, we developed a new visualization method to visualize the curriculum of Kobe Tokiwa University using multidimensional scaling (MDS) and a scatter plot. In this study, our focus is on methods to analyze the relationships between answers to questions in eduinformatics questionnaires. MDS methods are very useful, but have limitations in that their results are difficult to interpret. To facilitate the interpretation of these results, we develop a new visualization method using a network with both parametric and non-parametric correlation coefficients with a threshold (VNCC). VNCC has nine steps. We apply the VNCC method to research on nursing education, and provide an example of the visualization of the result. VNCC methods will be useful in dealing with qualitative research in eduinformatics