A comparison of Cohen's kappa and agreement coefficients by Corrado Gini

Multivariate analysis of psychological dat

La calidad temporal de registros en balonmano

Este trabajo es un estudio exploratorio para incrementar el control de la calidad del dato, mediante el empleo de coeficientes de correlación temporal. La consolidación de la metodología observacional (Anguera, 2017) ha venido precedida por mejoras de distinta índole. Se partió de una herramienta de registro observacional aplicada al balonmano y compuesta por criterios y categorías. El diseño del estudio se correspondió con el cuadrante III, a) nomotético, puntual, multidimensional. Dos observadores registraron partidos de balonmano (N=4) con ayuda del software Lince. Se emplearon los paquetes estadísticos SPSS v. 24 y JASP v. 0820 para así asociar los tiempos de cada registro inter observador y calcular los coeficientes de correlación entre otros análisis. La aplicación práctica del trabajo tendría en el análisis del comportamiento motor un campo de aplicación pero no el único. Abordar la sincronización de los observadores al registrar es un procedimiento susceptible de avanzar en fiabilidad y precisión.This work is an exploratory study to increase data quality control, by using temporal correlation coefficients. The consolidation of the observational methodology (Anguera, 2017) has been preceded by improvements of a different nature. It was based on an observational registration tool applied to handball and made up of criteria and categories. The study design corresponded to quadrant III, as a) nomothetic, isolated, and multidimensional. Two observers recorded handball matches (N=4) with the help of Lince software. Statistical packages, SPSS v. 24 and JASP v. 0820, were used to associate the times of each inter-observer register and calculate the correlation coefficients among other analysis. The practical application of the work would have in the analysis of motor behavior an application but that is not the only one. Addressing the synchronization of observers when registering is a procedure open to improvement both in reliability and accuracy.Ministerio de Economía y Competitividad, con financiación de FEDER. Proyectos: “La actividad física y el deporte como potenciadores del estilo de vida saludable: evaluación del comportamiento deportivo desde metodologías no intrusivas” (DEP2015-66069-P) y “Avances metodológicos y tecnológicos en el estudio observacional del comportamiento deportivo” (PSI2015-71947-REDP)peerReviewe

A Quantitative Methodology for Vetting Dark Network Intelligence Sources for Social Network Analysis

Social network analysis (SNA) is used by the DoD to describe and analyze social networks, leading to recommendations for operational decisions. However, social network models are constructed from various information sources of indeterminate reliability. Inclusion of unreliable information can lead to incorrect models resulting in flawed analysis and decisions. This research develops a methodology to assist the analyst by quantitatively identifying and categorizing information sources so that determinations on including or excluding provided data can be made. This research pursued three main thrusts. It consolidated binary similarity measures to determine social network information sources\u27 concordance and developed a methodology to select suitable measures dependent upon application considerations. A methodology was developed to assess the validity of individual sources of social network data. This methodology utilized source pairwise comparisons to measure information sources\u27 concordance and a weighting schema to account for sources\u27 unique perspectives of the underlying social network. Finally, the developed methodology was tested over a variety of generated networks with varying parameters in a design of experiments paradigm (DOE). Various factors relevant to conditions faced by SNA analysts potentially employing this methodology were examined. The DOE was comprised of a 24 full factorial design augmented with a nearly orthogonal Latin hypercube. A linear model was constructed using quantile regression to mitigate the non-normality of the error terms

Visual lameness assessment in comparison to quantitative gait analysis data in horses

Background Quantitative gait analysis offers objective information to support clinical decision-making during lameness workups including advantages in terms of documentation, communication, education, and avoidance of expectation bias. Nevertheless, hardly any data exist comparing outcome of subjective scoring with the output of objective gait analysis systems. Objectives To investigate between- and within-veterinarian agreement on primary lame limb and lameness grade, and to determine relationships between subjective lameness grade and quantitative data, focusing on differences between (1) veterinarians, (2) live vs video assessment, (3) baseline assessment vs assessment following diagnostic analgesia. Study design Clinical observational study. Methods Kinematic data were compared to subjective lameness assessment by clinicians with >= 8 years of orthopaedic experience. Subjective assessments and kinematic data for baseline trot-ups and response to 48 diagnostic analgesia interventions in 23 cases were included. Between and within-veterinarian agreement was investigated using Cohen's Kappa (kappa). Asymmetry parameters for kinematic data ('forelimb lame pattern', 'hindlimb lame pattern', 'overall symmetry', 'vector sum head', 'pelvic sum') were determined, and used as outcome variables in mixed models; explanatory variables were subjective lameness grade and its interaction with (1) veterinarian, (2) live or video evaluation and (3) baseline or diagnostic analgesia assessment. Results Agreement on lame limb between live and video assessment was 'good' between and within veterinarians (median kappa = 0.64 and kappa = 0.53). There was a positive correlation between subjective scoring and measured asymmetry. The relationship between lameness grade and objective asymmetry differed slightly between (1) veterinarians (for all combined parameters, p-values between P < .001 and 0.04), (2) between live and video assessments ('forelimb lame pattern', 'overall symmetry', both P <= .001), and (3) between baseline and diagnostic analgesia assessment (all combined parameters, between P < .001 and .007). Main limitations Limited number of veterinarians (n = 4) and cases (n = 23), only straight-line soft surface data, different number of subjective assessments live vs from video. Conclusions Overall, between- and within-veterinarian agreement on lame limb was 'good', whereas agreement on lameness grade was 'acceptable' to 'poor'. Quantitative data and subjective assessments correlated well, with minor though significant differences in the number of millimetres, equivalent to one lameness grade between veterinarians, and between assessment conditions. Differences between baseline assessment vs assessment following diagnostic analgesia suggest that addition of objective data can be beneficial to reduce expectation bias. The small differences between live and video assessments support the use of high-quality videos for documentation, communication, and education, thus, complementing objective gait analysis data

Comparison of classification ability of hyperball algorithms to neural network and k-nearest neighbour algorithms

The main focus of this thesis is to evaluate and compare Hyperbalilearning algorithm (HBL) to other learning algorithms. In this work HBL is compared to feed forward artificial neural networks using back propagation learning, K-nearest neighbor and 103 algorithms. In order to evaluate the similarity of these algorithms, we carried out three experiments using nine benchmark data sets from UCI machine learning repository. The first experiment compares HBL to other algorithms when sample size of dataset is changing. The second experiment compares HBL to other algorithms when dimensionality of data changes. The last experiment compares HBL to other algorithms according to the level of agreement to data target values. Our observations in general showed, considering classification accuracy as a measure, HBL is performing as good as most ANn variants. Additionally, we also deduced that HBL.:s classification accuracy outperforms 103's and K-nearest neighbour's for the selected data sets

Development of a standardized histopathology scoring system using machine learning algorithms for intervertebral disc degeneration in the mouse model—An ORS spine section initiative

Mice have been increasingly used as preclinical model to elucidate mechanisms and test therapeutics for treating intervertebral disc degeneration (IDD). Several intervertebral disc (IVD) histological scoring systems have been proposed, but none exists that reliably quantitate mouse disc pathologies. Here, we report a new robust quantitative mouse IVD histopathological scoring system developed by building consensus from the spine community analyses of previous scoring systems and features noted on different mouse models of IDD. The new scoring system analyzes 14 key histopathological features from nucleus pulposus (NP), annulus fibrosus (AF), endplate (EP), and AF/NP/EP interface regions. Each feature is categorized and scored; hence, the weight for quantifying the disc histopathology is equally distributed and not driven by only a few features. We tested the new histopathological scoring criteria using images of lumbar and coccygeal discs from different IDD models of both sexes, including genetic, needle-punctured, static compressive models, and natural aging mice spanning neonatal to old age stages. Moreover, disc sections from common histological preparation techniques and stains including H&E, SafraninO/Fast green, and FAST were analyzed to enable better cross-study comparisons. Fleiss\u27s multi-rater agreement test shows significant agreement by both experienced and novice multiple raters for all 14 features on several mouse models and sections prepared using various histological techniques. The sensitivity and specificity of the new scoring system was validated using artificial intelligence and supervised and unsupervised machine learning algorithms, including artificial neural networks, k-means clustering, and principal component analysis. Finally, we applied the new scoring system on established disc degeneration models and demonstrated high sensitivity and specificity of histopathological scoring changes. Overall, the new histopathological scoring system offers the ability to quantify histological changes in mouse models of disc degeneration and regeneration with high sensitivity and specificity

Task-oriented evaluation of electronic medical records systems: development and validation of a questionnaire for physicians

BACKGROUND: Evaluation is a challenging but necessary part of the development cycle of clinical information systems like the electronic medical records (EMR) system. It is believed that such evaluations should include multiple perspectives, be comparative and employ both qualitative and quantitative methods. Self-administered questionnaires are frequently used as a quantitative evaluation method in medical informatics, but very few validated questionnaires address clinical use of EMR systems. METHODS: We have developed a task-oriented questionnaire for evaluating EMR systems from the clinician's perspective. The key feature of the questionnaire is a list of 24 general clinical tasks. It is applicable to physicians of most specialties and covers essential parts of their information-oriented work. The task list appears in two separate sections, about EMR use and task performance using the EMR, respectively. By combining these sections, the evaluator may estimate the potential impact of the EMR system on health care delivery. The results may also be compared across time, site or vendor. This paper describes the development, performance and validation of the questionnaire. Its performance is shown in two demonstration studies (n = 219 and 80). Its content is validated in an interview study (n = 10), and its reliability is investigated in a test-retest study (n = 37) and a scaling study (n = 31). RESULTS: In the interviews, the physicians found the general clinical tasks in the questionnaire relevant and comprehensible. The tasks were interpreted concordant to their definitions. However, the physicians found questions about tasks not explicitly or only partially supported by the EMR systems difficult to answer. The two demonstration studies provided unambiguous results and low percentages of missing responses. In addition, criterion validity was demonstrated for a majority of task-oriented questions. Their test-retest reliability was generally high, and the non-standard scale was found symmetric and ordinal. CONCLUSION: This questionnaire is relevant for clinical work and EMR systems, provides reliable and interpretable results, and may be used as part of any evaluation effort involving the clinician's perspective of an EMR system