A latent trait look at pretest-posttest validation of criterion-referenced test items

Since Cox and Vargas (1966) introduced their pretest-posttest validity index for criterion-referenced test items, a great number of additions and modifications have followed. All are based on the idea of gain scoring; that is, they are computed from the differences between proportions of pretest and posttest item responses. Although the method is simple and generally considered as the prototype of criterion-referenced item analysis, it has many and serious disadvantages. Some of these go back to the fact that it leads to indices based on a dual test administration- and population-dependent item p values. Others have to do with the global information about the discriminating power that these indices provide, the implicit weighting they suppose, and the meaningless maximization of posttest scores they lead to. Analyzing the pretest-posttest method from a latent trait point of view, it is proposed to replace indices like Cox and Vargas’ Dpp by an evaluation of the item information function for the mastery score. An empirical study was conducted to compare the differences in item selection between both methods

The relationship between the presence of anti-cyclic citrullinated peptide antibodies and clinical phenotype in very early rheumatoid arthritis

<p>Abstract</p> <p>Background</p> <p>Anti-cyclic citrullinated peptide (anti-CCP) antibodies are highly specific for RA, but are not detectable in all RA patients. The aim of this study was to establish whether the clinical phenotypes of anti-CCP positive and negative disease are distinct at the earliest clinically apparent phase of disease.</p> <p>Methods</p> <p>Patients were recruited from the Birmingham early inflammatory arthritis clinic. Participants were included in the current study if they presented within 3 months of symptom onset and fulfilled 1987 ACR criteria for RA at some point during an 18 month follow-up. Data were collected on demographic variables, joint symptoms and tender (n = 68) and swollen (n = 66) joint counts. CRP, ESR, rheumatoid factor and anti-CCP2 status were measured.</p> <p>Results</p> <p>92 patients were included (48 anti-CCP positive). The anti-CCP positive and negative groups were comparable in terms of demographic variables, inflammatory markers, joint counts and 1987 ACR classification criteria, except that more anti-CCP positive patients were rheumatoid factor positive (83.3% vs. 11.4%, p < 0.01). There was no significant difference in the pattern of joint involvement, except for an increased prevalence of knee joint swelling in anti-CCP positive patients (42.9% vs. 22.2%, p = 0.03).</p> <p>Conclusions</p> <p>Patients with and without anti-CCP antibodies present in a similar way, even within three months of clinically apparent disease that eventually develops into RA.</p

Informational needs of general practitioners regarding discharge medication: content, timing and pharmacotherapeutic advice

textabstractObjective: To investigate the needs of Dutch general practitioners on discharge medication, both regarding content, timing and the appreciation of pharma-cotherapeutic advices from clinical pharmacists. Setting: A general teaching hospital in Amsterdam, the Netherlands. Method: A prospective observational study was performed. A questionnaire with regard to the content, optimal timing (including way of information transfer) and appreciation of pharmacotherapeutic advices was posted to 464 general practitioners. One reminder was sent. Main outcome measure: Description of the needs of general practitioners was assessed. For each question and categories of comments frequency tables were made. The Fisher-exact test was used to study associations between the answers to the questions. Results: In total, 149 general practitioners (32%) responded. Most general practitioners (75%) experienced a delay in receiving discharge medication information and preferred to receive this on the day of discharge. GPs wished to receive this information mainly through e-mail (44%). There was a significant correlation (P = 0.002) between general practitioners who wanted to know whether and why medication had been stopped (87%) and changed (88%) during hospital admission. The general practitioners (88%) appreciated pharmacotherapeutic advices from clinical pharmacists. Conclusion: This study indicates how information transfer on discharge medication to GPs can be optimised in the Netherlands. The information arrives late and GPs want to be informed on the day of discharge mainly by e-mail. GPs wish to know why medication is changed or discontinued and appreciate pharmacotherapeutic advices from clinical pharmacists

Are physical symptoms among survivors of a disaster presented to the general practitioner? A comparison between self-reports and GP data

<p>Abstract</p> <p>Background</p> <p>Most studies examining medically unexplained symptoms (MUS) have been performed in primary or secondary care and have examined symptoms for which patients sought medical attention. Disasters are often described as precipitating factors for MUS. However, health consequences of disasters are typically measured by means of questionnaires, and it is not known whether these self-reported physical symptoms are presented to the GP. It is also not known if the self-reported symptoms are related to a medical disorder or if they remain medically unexplained. In the present study, three research questions were addressed. Firstly, were self-reported symptoms among survivors presented to the GP? Secondly, were the symptoms presented to the GP associated with a high level of functional impairment and distress? Thirdly, what was the GP's clinical judgment of the presented symptoms, i.e. were the symptoms related to a medical diagnosis or could they be labeled MUS?</p> <p>Methods</p> <p>Survivors of a man-made disaster (N = 887) completed a questionnaire 3 weeks (T1) and 18 months (T2) post-disaster. This longitudinal health survey was combined with an ongoing surveillance program of health problems registered by GPs.</p> <p>Results</p> <p>The majority of self-reported symptoms was not presented to the GP and survivors were most likely to present persistent symptoms to the GP. For example, survivors with stomachache at both T1 and T2 were more likely to report stomachache to their GP (28%) than survivors with stomachache at only T1 (6%) or only T2 (13%). Presentation of individual symptoms to the GP was not consistently associated with functional impairment and distress. 56 – 91% of symptoms were labeled as MUS after clinical examination.</p> <p>Conclusion</p> <p>These results indicate that the majority of self-reported symptoms among survivors of a disaster are not presented to the GP and that the decision to consult with a GP for an individual symptom is not dependent on the level of impairment and distress. Also, self-reported physical symptoms such as headache, back pain and shortness of breath are likely to remain medically unexplained after the clinical judgment of a GP.</p

Item response analysis of the Positive and Negative Syndrome Scale

<p>Abstract</p> <p>Background</p> <p>Statistical models based on item response theory were used to examine (a) the performance of individual Positive and Negative Syndrome Scale (PANSS) items and their options, (b) the effectiveness of various subscales to discriminate among individual differences in symptom severity, and (c) the appropriateness of cutoff scores recently recommended by Andreasen and her colleagues (2005) to establish symptom remission.</p> <p>Methods</p> <p>Option characteristic curves were estimated using a nonparametric item response model to examine the probability of endorsing each of 7 options within each of 30 PANSS items as a function of standardized, overall symptom severity. Our data were baseline PANSS scores from 9205 patients with schizophrenia or schizoaffective disorder who were enrolled between 1995 and 2003 in either a large, naturalistic, observational study or else in 1 of 12 randomized, double-blind, clinical trials comparing olanzapine to other antipsychotic drugs.</p> <p>Results</p> <p>Our analyses show that the majority of items forming the Positive and Negative subscales of the PANSS perform very well. We also identified key areas for improvement or revision in items and options within the General Psychopathology subscale. The Positive and Negative subscale scores are not only more discriminating of individual differences in symptom severity than the General Psychopathology subscale score, but are also more efficient on average than the 30-item total score. Of the 8 items recently recommended to establish symptom remission, 1 performed markedly different from the 7 others and should either be deleted or rescored requiring that patients achieve a lower score of 2 (rather than 3) to signal remission.</p> <p>Conclusion</p> <p>This first item response analysis of the PANSS supports its sound psychometric properties; most PANSS items were either very good or good at assessing overall severity of illness. These analyses did identify some items which might be further improved for measuring individual severity differences or for defining remission thresholds. Findings also suggest that the Positive and Negative subscales are more sensitive to change than the PANSS total score and, thus, may constitute a "mini PANSS" that may be more reliable, require shorter administration and training time, and possibly reduce sample sizes needed for future research.</p

Cost-effectiveness of collaborative care for chronically ill patients with comorbid depressive disorder in the general hospital setting, a randomised controlled trial

Background. Depressive disorder is one of the most common disorders, and is highly prevalent in chronically ill patients. The presence of comorbid depression has a negative influence on quality of life, health care costs, self-care, morbidity, and mortality. Early diagnosis and well-organized treatment of depression has a positive influence on these aspects. Earlier research in the USA has reported good results with regard to the treatment of depression with a collaborative care approach and an antidepressant algorithm. In the UK 'Problem Solving Treatment' has proved to be feasible. However, in the general hospital setting this approach has not yet been evaluated. Methods/Design. CC: DIM (Collaborative Care: Depression Initiative in the Medical setting) is a two-armed randomised controlled trial with randomisation at patient level. The aim of the trial is to evaluate the treatment of depressive disorder in general hospitals in the Netherlands based on a collaborative care framework, including contracting, 'Problem Solving Treatment', antidepressant algorithm, and manual-guided self-help. 126 outpatients with diabetes mellitus, chronic obstructive pulmonary disease, or cardiovascular diseases will be randomised to either the intervention group or the control group. Patients will be included if they have been diagnosed with moderate to severe depression, based on the DSM-IV criteria in a two-step screening method. The intervention group will receive treatment based on the collaborative care approach; the control group will receive 'care as usual'. Baseline and follow-up measurements (after 3, 6, 9, and 12 months) will be performed by means of questionnaires. The primary outcome measure is severity of depressive symptoms, as measured with the PHQ-9. The secondary outcome measure is the cost-effectiveness of these treatments according to the TiC-P, the EuroQol and the SF-36. Discussion. Earlier research has indicated that depressive disorder is a chronic, mostly recurrent illness, which tends to cluster with physical comorbidity. Even though the treatment of depressive disorder based on the guidelines for depression is proven effective, these guidelines are often insufficiently adhered to. Collaborative care and 'Problem Solving Treatment' will be specifically tailored to patients with depressive disorders and evaluated in a general hospital setting in the Netherlands