Software-based decision-support : a basis for the development of a predictive system for sustainable management of haemonchosis in small ruminants

Data generated by five years of FAMACHA© clinical evaluation trials on one farm, and two years of trials on a second farm in South Africa, where targeted selective treatment was applied to treat haemonchosis in sheep, was used as a basis to explore new computational epidemiological methods to analyse the results of the trials. The research flowed from the earlier work of Dr. J.A. van Wyk and co-workers at the Faculty of Veterinary Science, University of Pretoria, who did much to develop, introduce, and validate the FAMACHA©system in South Africa and elsewhere in the world. Clinical haemonchosis was common during the summer rainfall season, and was found to increase in severity during January and February of each year. Sensitivity analysis of FAMACHA© data indicated that on the first farm (Farm 1) investigated, many of the animals that were clinically non-diseased were in fact anaemic, but due to misclassification, these animals were not detected. This was not the case on the second farm (Farm 2), where most animals that were clinically diseased according to FAMACHA© were found to be truly anaemic. The high prevalence of misclassification on Farm 1 has important implications for monitoring and chemotherapy of haemonchosis. The results indicated that under the conditions where the data were generated, the FAMACHA© system is sensitive enough, and adequately specific, to detect anaemic sheep despite misclassification. The application of Receiver Operating Characteristic curve analysis to the FAMACHA© method to select FAMACHA© categories for treatment, was in agreement with the findings that misclassification on Farm 1 would of necessity require that different treatment thresholds would need to be implemented to achieve the same test sensitivity as on Farm 2.Although the use of the Receiver Operating Characteristic method requires the use of dedicated software to generate results, especially if large data sets are analysed, it was found to be an accurate and valid way of indicating FAMACHA© threshold categories for treatment on both farms, for a desired sensitivity. A previously published multiple regression model was modified to incorporate stochasticity in the FAMACHA© proportions and the body mass of sheep, in order to simulate probable worm count. The fluctuations in simulated worm count adequately reflected the changing epidemiological situation of haemonchosis as indicated by temporal histograms of differential FAMACHA© proportions in flocks. The model was most sensitive to changes in FAMACHA© proportions in the sample, followed by increasing variability in body mass as a worm season progressed. Furthermore, for a given class of animal, a range of probable haemoglobin values could be associated with a preselected threshold worm burden. Themodel was sensitive to blanket drenching events, as a lower intensity of infection was predicted immediately after blanket drenching in all samples. It followed that model indications could be used probabilistically, to indicate minimum haemoglobin levels that would need to be sustained in order to prevent overwhelming worm burdens in a given classof animal. The penultimate chapter of the thesis is concerned with alternative methods of evaluation of rainfall as a risk factor for haemonchosis. Three different periods of rainfall, in relation to FAMACHA© sampling events, were evaluated in terms of entropy, or spread, and tested for strength of association with simulated flock haemoglobin values by regression analysis. Shannon’s entropy was used as an indicator of rainfall variability. Findings indicated a negative, and significant, correlation between rainfall entropy and flock haemoglobin level. On the strength of the association, a simulation model was proposed, which could theoretically indicate a probable range for expected flock haemoglobin level in a subsequent two-week period following FAMACHA© evaluation, provided that rainfall entropy is known. This work attempts to bridge the gap between implementation of the FAMACHA© system, and the investigation of several vital issues that would need to be addressed in the development of a wider ranging anthelmintic treatment decision-support system to delay anthelmintic resistance. The application of important quantitative methods, such as two-graph Receiver Operating Characteristic analysis, Monte Carlo simulation, and Shannon’s entropy to the FAMACHA© system, have provided new perspectives from which to develop an integrated computerized decision-support system. The thesis strongly supports the continued use of the FAMACHA© system in its present form, but the work has emphasised several key issues, such as misclassification, the need to develop decision-support systems that are useable in real time at farm level as opposed to regional level, and that the FAMACHA© system can and should be used as a basis for further development of decision-support software.Thesis (PhD (Veterinary Tropical Diseases))--University of Pretoria, 2007.Veterinary Tropical Diseasesunrestricte

Blueprint for an automated specific decision support system for countering anthelmintic resistance in Haemonchus spp. at farm level

This article is the first of a series aimed at developing specific decision support software for on-farm optimisation of sustainable integrated management of haemonchosis. It contains a concept framework for such a system for use by farmers and/or their advisors but, as reported in the series, only the first steps have been taken on the road to achieve this goal. Anthelmintic resistance has reached such levels of prevalence and intensity that recently it evoked the comment that for small ruminants the final phase of resistance was being entered, without effective chemotherapeutic agents on some farms with which to control worms at a level commensurate with profitable animal production. In addition, in the case of cattle, a recent survey in New Zealand showed 92% of worm populations to be resistant to at least one anthelmintic group. Ironically, new technology, such as the FAMACHA© system which was devised for sustainable management of haemonchosis, is at present being adopted relatively slowly by the majority of farmers and it is suggested that an important reason for this is the complexity of integration of new methods with epidemiological factors. The alternatives to the simple drenching programmes of the past are not only more difficult to manage, but are also more labour-intensive. The problem is further complicated by a progressive global shortage of persons with the necessary experience to train farmers in the new methods. The opinion is advanced that only computerised, automated decision support software can optimise the integration of the range of factors (such as rainfall, temperature, host age and reproductive status, pasture type, history of host and pasture infection, and anthelmintic formulation) for more sustainable worm management than is obtainable with present methods. Other than the conventional method (in which prospective analysis of laboratory and other data is mainly used to suggest when strategic prophylactic drenching of all animals for preventing excessive helminthosis should be conducted during the relevant worm season), the computer model being proposed is to be based on targeted selective treatment, supported by progressive periodic retrospective analysis of clinical data of a given worm season. It is emphasised that, in order not to repeat the mistakes of the past, such an automated support system should ideally be developed urgently in a attempt to engineer greater sustainability of any unrelated new anthelmintics which may reach the market.This work was done within the Framework Agreement between the Directorate General for Development Cooperation, Belgium, and the Institute of Tropical Medicine, Antwerpen (Belgium, under University of Pretoria Belgian Grant No. AG534), and with financial support from the EU “PARASOL” project (Food-CT-2005-022851).http://www.elsevier.com/locate/vetpa

Validation of the FAMACHA© eye colour chart using sensitivity/specificity analysis on two South African sheep farms

A validation study of the FAMACHA© system for clinical evaluation of anaemia due to Haemonchuscontortus was conducted on two commercial sheep farms in the summer rain fall region of South Africa. In this region, the Haemonchus season lasts from October to April. On Farm 1 the system was tested over a period of five successive years in consecutive sets of young stud Merino replacement rams and ewes examined at intervals of 3–5weeks over each Haemonchus season,under routine farming conditions. When FAMACHA© scores of 3, 4, and 5 and haematocrit values of ≤22%, ≤19%, and ≤15% were separately considered to be anaemic, sensitivity on Farm 1 ranged from a maximum of 83% for a haematocrit cut-off of ≤15%, to40% for a haematocrit cut-off of≤22%. Sensitivity increased to 93%when FAMACHA© scores of 2, 3, 4, and 5 were considered anaemic at a cut-off value of≤19%, but the positive predictive value decreased to 0.43, indicating thatmany non-anaemic animals would be treated. The analysis indicated a high level of classification bias on Farm 1, with the animals consistently being classified one FAMACHA© category lower (i.e. less anaemic) than reality. On Farm 2 the test was conducted over two successive years in yearling rams evaluated at weekly to fortnightly intervals during each worm season. Every ram judged to be in FAMACHA© category 4 or 5 was bled for haematocrit determination, and it was only dewormed with effective anthelmintics if the haematocrit was 15% or lower. When FAMACHA© scores of 3, 4, and 5 and haematocrit values of≤22% and ≤19% were separately considered to be anaemic on Farm 2, sensitivity ranged from 64% for a haematocrit cut-off of ≤22%, to 80% for a cut-off of ≤19%. For identical haematocrit cut-off values and proportions of the sampled flock considered to be diseased as for Farm 1, sensitivity was always higher for Farm 2. On the other hand, further analysis of the data indicated that the magnitude of the error on Farm 1 was very consistent on average over the entire trial period. The results of this study indicate that (i) persons introduced to the system should not only be trained, but also be evaluated for accuracy of application; (ii) the sensitivity of the FAMACH© diagnostic system should ideally be evaluated at shorter intervals to avoid production losses due to failure to detect anaemic animals which may be at risk of death; (iii) that calibration of the FAMACHA© scoring is essential per individual evaluator, and (iv) that animals should be examined at weekly intervals during periods of the highest worm challenge

Application of ROC curve analysis to FAMACHA© evaluation of haemonchosis on two sheep farms in South Africa

Test sensitivity and specificity for the FAMACHA© clinical test for anaemia due to haemonchosis have previously been shown to be adequate in differentiating between heavily/less heavily infected sheep, but these properties give no objective guidance for setting the optimum threshold at which anthelmintic treatment should occur. The aim of this work was to use Receiver Operating Characteristic curves (ROC) to evaluate the diagnostic accuracy of FAMACHA© testing by estimating the area under the ROC curve, and to use Two-graph ROC curves to decrease subjectivity in selecting treatment thresholds on two farms with contrasting management. Test diagnostic accuracy, and thus discriminating power as determined by the area under the ROC curves, ranged from “moderate to good” on the first farm, and from “moderate to high” on the second farm for haematocrit (the Gold Standard for the test) cut-offs of ≤22 % and ≤19 % on both farms respectively. Accuracy of classification between haematocrit cut-offs was not significantly different within farms, but did differ significantly between farms, with test accuracy being highest on the second farm at both haematocrit cut-offs (p< 0.05). The results also showed the suitability of the two-graph ROC curve approach for discriminating not only between different levels of accuracy of evaluators, but also to give an indication of the so-called ROC cut point (i.e. the desired threshold level) at which animals should be treated for a given level of risk of loss. The approach appears to have the potential not only to validate the diagnostic accuracy of the test across the complete testing range (i.e. all FAMACHA© categories from 1 – 5), but also to compensate for such inaccuracy by allowing objective adjustment of the threshold treatment level according to the output of the two-graph ROC method.This work was carried out within the Framework Agreement between the Directorate General for Development Cooperation, Belgium and the Institute of Tropical Medicine, Antwerpen (Belgium – Grant No AG534) with the Department of Veterinary Tropical Diseases of the University of Pretoria Faculty of Veterinary Science, and with the support of the EU “PARASOL” FOOD-CT-2005-022851 FP6 project.http://www.elsevier.com/locate/vetpa

A stochastic model accommodating the FAMACHA© system for estimating worm burdens and associated risk factors in sheep naturally infected with Haemonchus contortus

A previously developed multiple regression algorithm was used as the basis of a stochastic model to simulate worm burdens in sheep naturally infected with Haemonchus contortus over five consecutive Haemonchus seasons (November to January/February) on a farm in the summer rainfall region in South Africa, although only one season is discussed. The algorithm associates haemoglobin levels with worm counts in individual animals. Variables were represented by distributions based on FAMACHA© scores and body weights of sheep, and Monte Carlo sampling was used to simulate worm burdens. Under conditions of high disease risk, defined as the sampling event during the worm season with the lowest relative mean haemoglobin level for a class of sheep, the model provided a distribution function for mean class H. contortus burdens and the probability of these occurring.\ud \ud A mean H. contortus burden for ewes (n = 130 per sample) of approximately 1000 (range 51–28,768) and 2933 (range 78–44,175) for rams (n = 120 per sample) was predicted under these conditions. At the beginning of the worm season when the risk of disease was lowest (i.e. when both classes had their highest estimated mean haemoglobin levels), a mean worm burden of 525 (range 39–4910) for ewes and 651 (range 37–17,260) for rams was predicted. Model indications were that despite being selectively drenched according to FAMACHA© evaluation, 72% of the ewes would maintain their mean worm burden below an arbitrarily selected threshold of 1000 even when risk of disease was at its highest. In contrast, far fewer rams (27%) remained below this threshold, especially towards the end of the worm season.\ud \ud The model was most sensitive to changes in haemoglobin value, and thus by extrapolation, the haematocrit, which is used as the gold standard for validating the FAMACHA© system. The mean class haemoglobin level at which there was a 50% probability of worm burdens being ≤1000 worms was 7.05 g/dl in ewes and 7.92 g/dl in rams

Validation of the FAMACHA© eye colour chart using sensitivity/specificity analysis on two South African sheep farms

A validation study of the FAMACHA© system for clinical evaluation of anaemia due to Haemonchus contortus was conducted on two commercial sheep farms in the summer rainfall region of South Africa. In this region, the Haemonchus season lasts from October to April. On Farm 1 the system was tested over a period of five successive years in consecutive sets of young stud Merino replacement rams and ewes examined at intervals of 3–5 weeks over each Haemonchus season, under routine farming conditions. When FAMACHA© scores of 3, 4, and 5 and haematocrit values of ≤22%, ≤19%, and ≤15% were separately considered to be anaemic, sensitivity on Farm 1 ranged from a maximum of 83% for a haematocrit cut-off of ≤15%, to 40% for a haematocrit cut-off of ≤22%. Sensitivity increased to 93% when FAMACHA© scores of 2, 3, 4, and 5 were considered anaemic at a cut-off value of ≤19%, but the positive predictive value decreased to 0.43, indicating that many non-anaemic animals would be treated. The analysis indicated a high level of classification bias on Farm 1, with the animals consistently being classified one FAMACHA© category lower (i.e. less anaemic) than reality.\ud \ud On Farm 2 the test was conducted over two successive years in yearling rams evaluated at weekly to fortnightly intervals during each worm season. Every ram judged to be in FAMACHA© category 4 or 5 was bled for haematocrit determination, and it was only dewormed with effective anthelmintics if the haematocrit was 15% or lower. When FAMACHA© scores of 3, 4, and 5 and haematocrit values of ≤22% and ≤19% were separately considered to be anaemic on Farm 2, sensitivity ranged from 64% for a haematocrit cut-off of ≤22%, to 80% for a cut-off of ≤19%.\ud \ud For identical haematocrit cut-off values and proportions of the sampled flock considered to be diseased as for Farm 1, sensitivity was always higher for Farm 2. On the other hand, further analysis of the data indicated that the magnitude of the error on Farm 1 was very consistent on average over the entire trial period.\ud \ud The results of this study indicate that (i) persons introduced to the system should not only be trained, but also be evaluated for accuracy of application; (ii) the sensitivity of the FAMACHA© diagnostic system should ideally be evaluated at shorter intervals to avoid production losses due to failure to detect anaemic animals which may be at risk of death; (iii) that calibration of the FAMACHA© scoring is essential per individual evaluator, and (iv) that animals should be examined at weekly intervals during periods of the highest worm challenge