Quality of the Skill Skill Instrument Install Brick Based on the Indonesian National Qualification Framework (KKNI)

In the development of the current world of construction, mastery of competencies is themost important thing that every artisan must have in carrying out his work. To measure thecompetence of the artisan, an instrument is needed. The instrument used can be used as a reference forevaluating competencies that are by the standards and conditions in the field which later can helpcreate artisan who is qualified and able to keep abreast of the developments that occur in the world ofconstruction today. This study aims to determine whether the existing brick installation workinstruments are by the standards and needs in the field. This study uses qualitative methods with datacollection techniques in the form of questionnaires aimed at experts to obtain expert recognition orjustification of existing instruments. From the results of the study, according to the expert justification,it was found that there were several items of masonry work that were rarely used and not includingcore work or more precisely the work was only done if there were orders from owners and valuationmethods that were considered less objective in assessing job competencies brick couple. The benefitsof this research can be a reference in developing brick installation work instruments

An innovative approach for automatically grading spelling in essays using rubric-based scoring

Automated Essay Grading (AEG) is defined as a computer technology that evaluates and scores written prose. A number of AEG systems have been developed since the 1960s and in most of them, an ad-hoc or generalized approach is used to grade spelling even though it is an important element of an essay-scoring rubric. Existing approaches do not therefore give an accurate representation or measure of spelling in essays. According to the rubric-based scoring method used in the National Assessment Program "Literacy and Numeracy (NAPLAN) in Australia, spelling is marked in three steps" first, by identifying the correct and incorrect words in the essay; second, by categorizing each word based on the difficulty level into one of four classes: Simple, Common, Difficult or Challenging, and counting the number of correct and incorrect words in each category; finally, by using the pre-defined NAPLAN rubric scale to assign the mark. Only a small number of existing AEG systems can be used for rubric-based scoring, and none can be used to grade spelling according to the NAPLAN rubric. In this paper, we address this shortcoming in the existing literature and present an innovative approach to automatically mark spelling using rubric based scoring. We develop two algorithms based on the rules and heuristics of the English language to formulize the rubric for spelling and then implement these algorithms in Java language and perform a series of evaluations of our system using an essay dataset. Our results are very promising, even though it is the first system of this kind

Avaliações em larga escala com itens de respostas construídas no contexto do modelo multifacetas de Rasch

Tese (doutorado) - Universidade Federal de Santa Catarina, Centro Tecnológico, Programa de Pós-Graduação em Engenharia de Produção, Florianópolis, 2015.Esta tese apresenta um estudo sobre as avaliações com itens de respostas construídas em larga escala no contexto do modelo multifacetas de Rasch (LINACRE, 1989 apud LINACRE, 1994). Essas avaliações necessitam de avaliadores para julgar o desempenho das pessoas quanto à habilidade que está sendo medida por meio do teste. Entre as avaliações com itens de respostas construídas mais utilizadas no âmbito educacional e de seleção estão as provas das diversas disciplinas do Ensino Médio, as provas de redação do ENEM e dos concursos vestibulares e as provas com itens abertos de concursos para provimento de vagas de trabalho. Os resultados das avaliações com itens de respostas construídas não dependem apenas do nível de habilidade dos examinandos quanto ao construto avaliado e da dificuldade das tarefas, dependem também da severidade dos avaliadores que julgam os desempenhos e da estrutura da escala de classificação. Um dos principais problemas nessas avaliações é a pontuação de um mesmo desempenho com graus diferentes de severidade. Quando existem vários avaliadores, o ideal é que todos atribuam exatamente a mesma pontuação para os mesmos desempenhos observados, essa é a condição principal para se ter confiabilidade de pontuação. Entretanto, são muitos os fatores que podem causar variabilidade nessas pontuações. O modelo multifacetas de Rasch vem sendo cada vez mais utilizado para aferir a qualidade das avaliações com itens de respostas construídas, por permitir a inclusão de outras variáveis aos sistemas avaliativos, além da capacidade dos indivíduos e da dificuldade das tarefas. Algumas dessas variáveis consistem em importantes fontes geradoras de vieses nos processos avaliativos. Como exemplos têm-se as características pessoais dos avaliadores, as diferenças entre a severidade dos avaliadores, as tendências dos avaliadores em julgamentos sistemáticos, as diferenças entre as dificuldades de tarefas distintas e a variação quanto ao entendimento e utilização das categorias da escala de classificação por parte dos avaliadores. O modelo multifacetas de Rasch permite a inclusão de cada variável que pode interferir na avaliação, além de possibilitar análises para os efeitos causados por cada elemento que faz parte da avaliação individualmente, o que torna a utilização desse modelo muito vantajosa. O objetivo deste estudo é estabelecer como o modelo multifacetas de Rasch pode contribuir para a determinação da qualidade das avaliações com itens de respostas construídas. A abordagem utilizada pelo modelo multifacetas de Rasch proporciona análises sobre a qualidade das medidas relacionadas aos examinandos, aos avaliadores, às tarefas, aos itens e às escalas de classificação utilizadas para a pontuação das tarefas. Este trabalho também apresenta uma aplicação do modelo multifacetas de Rasch aos dados provenientes de uma avaliação real, na qual estabelece as principais análises sobre a qualidade dessa avaliação.Abstract : This thesis presents a study about the large-scale construct-response item evaluations in the context of the many-facet Rasch model (LINACRE, 1989 apud LINACRE 1994). These evaluations require raters in order to judge the performance of the people regarding the ability that is being measured through test. Among the evaluations with constructed-responses items most frequently used in the educational and hiring ambit are those with open questions of the disciplines of the High School, the writing test of the Brazilian High School National Exam and of the university entrance exams and the tests with open questions of contests. The results of the construct-response item evaluations do not depend only on the ability level of the examinants regarding the evaluated construct and the difficulty of the tasks; they depend also on the severity of the raters that judge the performance and the structure of the classification scale. One of the main problems of these evaluations is the rating of a same performance with different severity degrees. When there are many raters, it would be the ideal if all would give exactly the same rating for the same performances observed, this is the main condition in order to have reliability of rating. However, many are the factors that can cause variability in these ratings. The many-facet Rach model have been even more used to check the quality of the construct-response item evaluations, since it allows the inclusion of other variables to the evaluating systems, besides the capabilities of the individuals and the difficulty of the tasks. Some of these variables consists of important sources generator of biases in the evaluating processes. As examples are the personal characteristics of the raters, the differences between the severity of the raters, the tendencies of the raters in systematic judgements, the differences between the difficulties of the distinct tasks and the variation regarding the understanding and use of the categories of the classification scale by the raters. The many-facet Rach model allows the inclusion of each variable that can interfere in the evaluation besides allowing analyzes for the effects caused by each element that is individually part of the evaluation, which makes the use of the many-facet Rach model very advantageous. The objective of this study is to establish how the many-facet Rach model can contribute to the determination of the quality of the evaluations with construct-response items. The approach used by the many-facet Rach model provides analyzes on the quality of the measure related to the examinees, to the the raters, to the tasks, to the questions and to the classification scales used for the rating of the tasks. This work also presents an application of multi-faceted Rasch model to data from a real assessment, which establishes the main analyzes of the quality of the evaluation