Application of Rasch model in validating the content of measurement instrument for blog quality

Research in blog quality is very crucial nowadays in order to have a good quality blog in the blogosphere. The blog quality criteria have been derived from a rigorous metadata analysis. Yet, these criteria have not been reviewed and their significance has not been proven systematically. In this paper, Rasch Model is applied to produce an empirical evidence of content validity of the blog quality criteria. This study confirms that the definitions of 11 families and the 49 criteria assigned have content validity by mean of online survey. These criteria will then be used as a basis of constructing the instrument to measure the acceptability of the criteria for blog quality

Validity of the Persian Blog Attitude Questionnaire: An Evidence-Based Approach

Validity evidence is provided for a Persian blog attitude questionnaire (P-BAQ). P-BAQ was administered to 565 Iranians and factor analysis and rating scale model identified affective, behavioral, and perseverance, and confidence dimensions underlying the data. P-BAQ’s validity argument was supported by the theoretical and psychometric evidence, although adding a few items to the instrument would improve its construct representativeness

Development of a Digital Problem Solving Skills Test Instrument: Model Rasch Analysis

Biodiversity is one of the subjects in Biology class X. Learning about the concept of biodiversity should be done by analyzing real natural phenomena, especially regarding biodiversity that exists in daily life. The government through the Regulation of the Minister of National Education Number 23 of 2006 has set competency standards for high school graduates that correlate with 21st century skills. One of the 21st century competencies is problem solving skills. A total of 26 questions have been developed in measuring Digital Problem Solving skills on biodiversity material for high school students. The questions were declared valid and reliable based on the results of analysis using the Rasch model through the Winsteps program in a polytomous manner, including the criteria of item validity, construct validity, and content validity, reliability, and DIF. As an implication from this study, the questions that have been developed in this study can be used to measure Digital Problem Solving skills in Biology lessons in high school, especially in the subject of biodiversit

University lecturers’ preparedness to use technology in teacher training of mathematics during COVID-19: The case of Ethiopia

The study aimed to investigate Ethiopian university lecturers’ readiness to use technology for teaching mathematics at the tertiary level during the COVID-19 pandemic when they were compelled to adapt to distant education. Using Google Forms, online questionnaires were distributed to 41 lecturers in three Ethiopian universities, of whom eighteen participated. Before the research, the questionnaire was piloted with eight lecturer participants to categorise questions and validate the instrument using the Rasch measurement model. The questionnaire was locally developed based on guidelines from the literature. It purposed to investigate university lecturers’ individual preparedness for technological instruction in terms of their knowledge, beliefs and current, and historical exposure to this mode of instruction. As a counterbalance, some circumstantial factors influencing their readiness were investigated too. Lecturers’ optimistic beliefs about using educational technologies have been found to contrast with some disabling circumstantial factors. This study revealed that the lecturers were generally able and interested in integrating technology into the teaching process but that barriers, primarily at the institutional level, hindered them from doing so. In addition to the technologies suggested in the questionnaire, participants enriched the research findings by adding more possible technologies that lecturers may use for educational purposes. The data was analysed using WINSTEPS (Student Version of WINSTEPS 4.7.0.0) and SPSS version 20. The results showed the reliability of using the instrument was 0.77 based on Cronbach’s alpha. The PT-measure correlation value determined the construct validity (PMC), ranging from 0.23 to 0.71 except item PUT15’s infit and outfit MNSQ between 0.1 to 1.86 and ZSTD range -1.05 to 1.61, which was acceptable. The fit statistics showed that the person separation index, 1.97, was considered good and that the item separation index, 0.63 was within an acceptable range. Person and item reliability were at 0.8 and 0.28, respectively. The result indicated that the new instrument with five items after eliminating unfit items (such as items FAT19, PTT10, KDT1, PTT8 and PTT 12) was reliable and valid to measure the use of technology in the teaching and learning process of the university lecturers

The KIDSCREEN-27 scale: translation and validation study of the Slovenian version

Background: There are many methods available for measuring social support and quality of life (QoL) of adolescents, of these, the KIDSCREEN tools are most widely used. Thus, we aimed to translate and validate the KIDSCREEN-27 scale for the usage among adolescents aged between 10 and 19 years old in Slovenia. Methods: A cross-sectional study was conducted among 2852 adolescents in primary and secondary school from November 2019 to January 2020 in Slovenia. 6-steps method of validation was used to test psychometric properties of the KIDSCREEN-27 scale. We checked descriptive statistics, performed a Mokken scale analysis, parametric item response theory, factor analysis, classical test theory and total (sub)scale scores. Results: All five subscales of the KIDSCREEN-27 formed a unidimensional scale with good homogeneity and reliability. The confirmatory factor analysis showed poor fit in user model versus baseline model metrics (CFI = 0.847TLI = 0.862) and good fit in root mean square error (RMSEA = 0.072p(χ2) < 0.001). A scale reliability was calculated using Cronbach\u27s α (0.93), beta (0.86), G6 (0.95) and omega (0.93). Conclusions: The questionnaire showed average psychometric properties and can be used among adolescents in Slovenia to find out about their quality of life. Further research is needed to explore why fit in user model metrics is poor

A Study of the Association of Teacher Efficacy and Students\u27 Achievement in a North Texas School District

The field of education places great importance on measurable student achievement and the role of teaching competencies for maximizing success. Whereas much focus has been given to professional development and improving teachers’ skills, this quantitative study examined whether teachers\u27 self-efficacy and judgment about their skills and capabilities were significantly associated with improving student achievement in third grade reading. This dissertation involved a study of teacher self-efficacy using a sample group of third grade English/Language Arts teachers from one Texas urban school district. Targeted teachers were invited to complete a valid and reliable teacher efficacy survey, the Tschannen-Moran and Woolfolk Hoy Teachers\u27 Sense of Efficacy Scale. The results of each teacher\u27s self-efficacy score were correlated with their students\u27 reading scores on two assessments: the Measures of Academic Progress and the Fountas & Pinnell Oral Reading Assessment. The study results indicated that there was not a statistically significant association between teacher self-efficacy and the reading scores of their students on the two assessments. The results were unexpected because the outcomes were different than those reported in the literature. The Covid pandemic created challenges. One was that student reading scores were from the 2018-2019 school year; the teacher self-efficacy scores were collected during 2020-2021. The 2019-2020 and 2020-2021 school years were significantly disrupted by the pandemic, and students were asked to complete the two assessments virtually from their homes with no supervision. Many students did not complete the assessments, including a high percentage of minority and at-risk students. In addition, the research design did not allow an exploration of how teachers\u27 overall emotional and professional feelings of efficacy may have been different because of the pandemi

PENGEMBANGAN INSTRUMEN PENGUKURAN IKLIM EMOSI DAN SIKAP PADA LINGKUNGAN PEMBELAJARAN IPA

ABSTRAK Penelitian ini bertujuan untuk menambah kajian bidang pengembangan instrumen sebagai bentuk refleksi siswa tentang lingkungan belajarnya, dan juga sebagai evaluasi oleh peneliti, guru dan kepala sekolah untuk meningkatkan prestasi belajar siswa. Jenis penelitian ini yaitu kuantitatif dengan motode pengembangan instrumen menurut Recker dan Rosemann (2010) melalui lima tahapan yaitu tahap item creation (pembuatan item), substrata identification (identifikasi substrat atau dimensi), item identification (identifikasi item), item revision (revisi item) dan instrument validation (validasi instrumen). Penelitian ini dilakukan pada 1.067 siswa SMPN di Jakarta. Pengumpulan data dilakukan melalui melalui online dengan menggunakan Google Form berupa instrumen pengukuran iklim emosi dan sikap pada lingkungan pembelajaran IPA. Hasil analisis menunjukkan bahwa (1) instrumen memiliki delapan dimensi yaitu kolaborasi, motivasi, peduli, kejelasan, sikap penyelidikan ilmiah, kesenangan dalam pembelajaran, adopsi sikap ilmiah dan implikasi sosial sains. (2) Uji confirmatory factor analysis dengan loading factor > 0.30 menghasilkan 37 item. (3) Analisis lanjutan dengan pemodelan rasch menunjukkan nila raw variance explain by measures 36,6% dan nilai Unexplned variance 1-5 dibawah 15%, Fit item menghasilkan 36 item dan 589 person dengan lima tingkat kesukaran item yaitu sangat sukar, sukar, sedang, mudah dan sangat mudah. Kemampuan person juga dikelompokkan menjadi lima tingkat yaitu sangat tinggi, tinggi, sedang, rendah dan sangat rendah. Selain itu cronbach alpha 0,92 Person reliability 0,88 dan item reliability 0,99. (4) Dimensi yang paling dominan jika dilihat dari banyaknya jumlah item dalam dimensi instrumen pengukuran iklim emosi dan sikap pada lingkungan pembelajaran IPA berada pada dimensi motivasi (motivation). Dapat disimpulkan bahwa instrumen valid dan reliabel sehingga dapat digunakan untuk mengukur iklim emosi dan sikap pada lingkungan pembelajaran IPA. Keywords: Lingkungan Belajar, Iklim Emosi, Sikap dan Ilmu Pengetahuan Alam. ************ ABSTRACT The purpose of this research is to enrich the discussion of the component of developing instrument as a realization of student reflection on their learning environment, and as an evaluation by researchers, teachers, and principals to increase students’ motivation and achievement. The approach of the research is a quantitative research with developing the instrument method according to Recker and Rosemann (2010) through five stages, namely the item creation stage, substrata identification (identification of substrates or dimensions), item identification (item identification), item revision (item revision) and instrument validation (instrument validation). The research is conduct on 1.067 students of public middle school in Jakarta. Data collection was Conducted online using google forms as a instrument emotional climate and attitude mesurement on science learning enviroment. The analysis of the results showed that (1) the instrument consists of eight dimensions are, collaboration, motivation, care, clarity, attitude to scientific inquiry, enjoyment of science lessons, adoption of scientific attitudes, and sosial implications of science. (2) Confirmatory Factor Analysis with a loading factor > 0,30 are 37 items. (3) the further analysis use Rasch modelling demonstrate the raw variance explained by measures value is 36,6%, and the unexplained variance value 1-5 is under 15%, Fit Item value is 36 items and 589 person have five items level of difficulty are, very hard, hard, medium, easy and very easy. Person abilities are also clustered into five levels are very high, high, moderate, low and very low. Besides that, Cronbach's alpha value is 0,92 Person reliability is 0,88 and item reliability is 0,99. (4) when looking at the number of items in each dimension of instrument emotional climate and attitude mesurement on science learning enviroment, the most dominant dimension is motivation. It can be concluded that the instrument is valid and reliable, thus it can be used to measure the emotional climate and attitudes in the science learning environment. Keywords: Learning Environment, Emotional Climate, Attitude, and Science

An instrument to measure information and communication technology user-skills ability for engineering learning

Some of the most important skills for engineering education in today’s digital world are information and communication technology (ICT) user-skills. This research concerned two main issues regarding ICT user-skills of engineering students. The first issue was the lack of a reliable and valid instrument to measure ICT user-skills ability for engineering learning. The second issue was the lack of profile information on students’ existing ICT user-skills, such as what their ICT skills level were, how they acquired the skills, their conception of ICT user-skills, to what extent ICT user-skills support engineering learning, as well as the difficulties faced in acquiring those skills. This information would provide the basis for student ICT skills improvement strategies. Thus, this research sought to address these issues by developing an instrument to measure students’ ICT user-skills and subsequently establishing the ICT user-skills profile. This study adopted an across-stage mixed method design, combining quantitative and qualitative approaches. The research process comprised eight major phases: problem identification, literature review, determining problem statement and research objectives, instrument design and development, sample selection, data collection, data analysis, discussion and conclusion. Instrument development and validation were performed in five phases: determining what to measure, a review and assessment of major existing instruments, drafting a new instrument, getting expert reviews and student feedback, pilot testing the instrument, checking the internal consistency and refining the instrument, testing the modified instrument, and finally conducting the main study using a stratified random sample. Reliability and validity of the instrument were established using a Rasch model. Quantitative data analyses were performed using the PASW and WINSTEPS software. Thematic analysis of interview transcriptions was conducted to corroborate quantitative findings. The outcomes of this study were a new survey instrument to measure ICT user-skills within context of the study population, and a profile of engineering students’ ICT user-skills