ROSY application for selecting R packages that perform ROC analysis

The empirical ROC curve is a powerful statistical tool to evaluate the precision of tests in several fields of study. This is a two-dimensional plot where the horizontal and vertical axis represent false positive and true positive fraction respectively, also referred to as 1-specificity and sensitivity, where precision is evaluated through a summary index, the area under the curve (AUC). Several computer tools are used to perform this analysis one of which is the R environment, this is an open source and free to use environment that allows the creation of different packages designed to perform the same tasks in distinct ways often resulting in different customization and features often providing similar results. There is a need to explore these different packages to provide an experienced user with the simplest and most robust execution of a needed analysis. This work catalogued the different R packages capable of ROC analysis exploring their performance. A shiny web application is presented that serves as a repository allowing for efficient use of all of these packages.This work has been supported by FCT - Fundação para a Ciência e Tecnologia within the RD Units Project Scope: UIDB/00319/2020

Comparing empirical ROC curves using a Java application: CERCUS

Receiver Operating Characteristic (ROC) analysis is a methodology that has gained much popularity in our days, especially in Medicine, since through the ROC curves, it provides a useful tool to evaluate and specify problems in the performance of a diagnostic indicator. The area under empirical ROC curve (AUC) it’s an indicator that can be used to compare two or more ROC curves. This work arose from the necessity of the existence of software that allows the calculation of the necessary measures to compare systems based on ROC curves. Several software, commercial and non-commercial, are available to perform the calculation of the measures associated to the ROC analysis. However, they present some flaws, especially when there is a need to compare independent samples with different dimensions, or also to compare two ROC curves that intersect. In this paper is presented a new application called CERCUS (Comparison of Empirical ROC Curves). This was developed using a programming language (Java) and stands out for the possibility of comparing two or more ROC curves that cross each other. The main objective of CERCUS is the calculation of several ROC estimates using different methods and make the ROC curves comparison, even if there is an intersection, either for independent or paired samples. It also allows the graph representation of the ROC curve in a unitary plan as well the graph of the area between curves in comparison. This paper presents the program’s versatility in data entry, test menus and visualization of graphs and results.This work has been supported by FCT – Fundação para a Ciência e Tecnologia within the Project Scope: UID/CEC/00319/201

RealROC: a shiny based application for ROC curve study with covariate adjustment

Dissertação de mestrado em BioinformaticsA curva ROC (Receiver operating characteristic) é uma ferramenta analítica eficaz para testes clínicos. A análise permite visualizar a variação de sensibilidade e especificidade para uma dada região de corte através de um simples, mas robusto gráfico bidimensional. Num contexto biológico, testes podem ser influenciados por múltiplas variáveis externas e como tal a análise ROC pode não ser a ideal ou gerar resultados incompletos. É então necessário saber que variáveis afetam determinado teste clínico de forma a determinar os melhores parâmetros para determinado teste ou até descartar determinada metodologia mediante a situação. O ajuste da curva ROC a covariáveis permite a normalização do efeito das mesmas ou diretamente ajustar a curva para os seus efeitos. Software direcionado ao ajuste da curva ROC é, infelizmente, escasso e muitas vezes difícil de manusear por utilizadores não especializados. Recentemente o pacote AROC foi lançando para R que disponibiliza vários recursos para estes ajustamentos, no entanto a dificuldade de utilização mantém-se. A combinação deste pacote com a estrutura Shiny, um pacote que permite o desenvolvimento de aplicações interativas, tem por objetivo a criação de um programa grátis e acessível que permita uma análise mais aprofundada disponível para todos os investigadores. RealROC foi capaz de replicar resultados de um caso de estudo que analisou a influência do sexo no sistema de pontuação CRIB e respetiva previsão de mortalidade, demonstrando a usabilidade e acessibilidade do programa que será disponibilizado online e potencialmente contribuir para novos desenvolvimentos na área.Receiver operating characteristic (ROC) curves are a powerful analytical tool for clinical tests. The analysis allows the visualization of varying sensitivity and specificity for a given threshold through a simple, yet robust, two-dimensional plot. In a biological framework, tests can be influenced by multiple external variables, as such, standard ROC analysis may not be suitable or may provide incomplete data. It is then necessary to know which variables influence clinical test results to determine optimal conditions for trials or even to disregard a given method of evaluation in certain contexts. Adjusting for covariates allows ROC analysis to normalize the effects of the variable in question or to directly adjust the curve for its effects. Unfortunately ROC software that is able to conduct such an adjustment is sparse and proven difficult to use for non technical users. Recently, the AROC package for R was released and provides a robust resource for such adjustments however with he same usability problems previously stated. By combining this package with the Shiny framework, an R package that allows the creation of interactive applications, we hope to provide an accessible and free software that allows this extra depth of analysis to be available for all researchers. RealROC was able to mimic the results of a case study analysing the affects of sex to the CRIB score and resulting mortality rates that proving its practicality and will be made available online and hopefully contribute to the advancement of software in this field

Desenvolvimento de um programa para comparação de curvas ROC: para amostras independentes e amostras relacionadas

Dissertação de mestrado em BioinformáticaA análise ROC (Receiver Operating Characteristic) tem vindo a ganhar muita popularidade, principalmente na área da medicina, dado que é uma ferramenta útil para avaliar e especificar problemas no desempenho de um indicador de diagnóstico. A área abaixo da curva ROC (AUC) é um indicador que pode ser utilizado para comparação de duas ou mais curvas ROC. Este trabalho, surgiu da necessidade de existência de softwares que permitem o cálculo das medidas necessárias para comparação de sistemas com base nas curvas ROC. Existem vários softwares que efetuam o cálculo de medidas associadas à análise ROC, no entanto apresentam algumas lacunas, nomeadamente no que diz respeito à comparação para amostras independentes com diferentes dimensões e na comparação de duas curvas ROC quando estas se intersetam. Neste trabalho é apresentado uma nova aplicação que se designa por CERCUS. Esta foi desenvolvida usando a linguagem de programação JAVA e destaca-se pela possibilidade de comparar duas ou mais curvas ROC. Este programa tem como principal intuito o cálculo de várias estimativas ROC, usando os diferentes métodos sugeridos no desenrolar do trabalho e fazer a comparação de curvas ROC, mesmo que haja interseção, quer para amostras independentes ou amostras emparelhadas. Permite ainda, a representação no plano unitário da curva ROC empírica e a área entre as curvas.Receiver Operating Characteristic (ROC) analysis has gained much popularity, especially in the medical field, as it is a useful tool to assess and specify problems in the performance of a diagnostic indicator. The area below the ROC curve (AUC) is an indicator that can be used to compare two or more ROC curves. This work emerged from the need for software to allow the calculation of the necessary measurements to compare systems based on ROC curves. There are several software that perform the calculation of measures related to ROC analysis, however they present some gaps, particularly as regards the comparison for independent samples with different dimensions and in comparing two ROC curves where they intersect. In this work a new application is presented that is denominated by CERCUS. This was developed using the programming language JAVA and stands out by the possibility of comparing two or more ROC curves. The main purpose of this program is the calculation of several ROC estimates, using the different methods suggested along in the dissertation and comparing ROC curves, even if there is an intersection, for independent samples or paired samples. It also allows the representation in the unit plane of the empirical ROC curve and the area between the curves