Wilkins, Leslie T.

English

Northwestern University Illinois, School of Law: Scholarly Commons

Journal of Criminal Law and CriminologyVolume 71Issue 2 Summer Article 5Summer 1980Problems with Existing Prediction Studies andFuture Research NeedsLeslie T. WilkinsFollow this and additional works at: https://scholarlycommons.law.northwestern.edu/jclcPart of the Criminal Law Commons, Criminology Commons, and the Criminology and CriminalJustice CommonsThis Symposium is brought to you for free and open access by Northwestern University School of Law Scholarly Commons. It has been accepted forinclusion in Journal of Criminal Law and Criminology by an authorized editor of Northwestern University School of Law Scholarly Commons.Recommended CitationLeslie T. Wilkins, Problems with Existing Prediction Studies and Future Research Needs, 71 J. Crim. L. & Criminology 98 (1980)9901-4169/80/7 102-0098S02.00/0THE JOURNAL OF CRIMINAL LAw & CRIMINOLOGY Vol. 71, No. 2Copyright © 1980 by Northwestern University School of Law Printed in U.S.A.PROBLEMS WITH EXISTING PREDICTION STUDIES AND FUTURERESEARCH NEEDSLESLIE T. WILKINS*Most decisions involve risk and are thus subjectto two kinds of error. Prognoses, estimates of futureconditions, and all probability statements are sub-ject to the same two kinds of error. In criminolog-ical prognoses, the decisionmaker can be in errorin that: (1) the individual who is predicted to failmay succeed, or (2) the individual who is predictedto succeed may fail. These two kinds of error applyto all decisions or estimates irrespective of themeans by which such decisions or estimates arederived. In criminology, neither clinical nor statis-tical methods of prognosis and prediction can avoidthe two classes of error.1 In addition, these twokinds of error will be present no matter how successor failure are defined. The first kind of error isusually termed a "false positive," or "overpredic-tion." Overprediction will tend to increase in pro-portion to the decrease in the number of individ-uals who fit the "fail" category. It is important tostress that false positives cannot be avoided.Clinical methods usually are not able to estimatethe magnitude of the errors, whereas statisticalmethods do so as an integral part of the process ofcalculation of the likelihood of success or failure.But clinical ignorance of the size of the errors doesnot mean that those studies produce a smallernumber of errors than studies applying statisticalprediction. Where comparisons have been made ofthe false positive rates, the clinical rates have beenlarger in almost all instances.2THE USE OF PREDICTIVE STATEMENTSAt present, statistical predictions, and also, byinference, all clinical predictions, produce a largeproportion of false positives. The proportion, to* Professor, Graduate School of Criminal Justice,SUNY Albany; currently, Visiting Professor, SimonFraser University.IIn industrial decisions, the two classes of error areoften termed "producer" and "consumer" risks, and es-timates of the magnitude of the two classes are frequentlywritten into contracts.2 P. MEEHL, STATISTICAL V. CLINICAL PREDICTION: ATHEORETICAL ANALYSIS AND A REVIEW OF THE EVIDENCE(1954).some degree, depends on the frequency of thephenomenon predicted. Violent crimes, for exam-pIe, occur less frequently than property crimes andthus tend to show lower levels of predictability.Some law and criminal justice writers have sug-gested that because of the presence of false positivesin predictive statements, statements and decisionshaving a predictive basis should be avoided in alldealings with offenders. In other words, thesewriters assert that reference should be made onlyto the past; it is improper to give consideration tothe possible outcome of a decision. This is theposition taken by advocates of the "just deserts"theory. It must be realized, though, that despitethe fact that prediction is believed to have beenavoided if it is not directly invoked in the interpre-tation of previous events, it may, nonetheless be-come involved (in some ways not completely un-derstood) in the definitions of "culpability."Whether prediction really is avoided is not, ofcourse, the major issue of concern. What is ofconcern are the qualities of the decisions, whetheror not they make use of "predictions." Humansseem to be anticipating creatures, and there isdifficulty with the "just deserts" proposition thatthis faculty, for ethical reasons, should be inhibited.EVENTS AND PERSONSIndividuals cannot validly be classified as "dan-gerous" or "not dangerous," but their crimes canbe. The commission of a crime is an historical eventthat can be analyzed with precision. On this basis,one might argue that thought should be concen-trated upon definitions of those kinds of behaviors(not persons) that require restraint. Of course, theperson (who-prediction or not-will continue intothe unknown future) will be involved in any suchrestraining situation. But the logic underlying thedisposition of the case would be independent ofjudgments about personality.Although a crime may be contemplated, legallydefined, and discussed independently of the of-fender, in operational terms the actor cannot beseparated from the act. However, one can discrim-inate between micro and macro models in termsCAREER CRIMINAL SYMPOSIUMclosely analogous to those of economic theory.' Onecould propose methods of dealing with the issuesof crime that do not involve a clear concept of theindividual offender. The focus instead would beplaced upon the features of situations common tomany crimes. In this manner, it might be possibleto take action about crime (macro models) inde-pendently of action attempting to deal with theindividual offender (micro models). However, thistype of approach remains relatively unexplored.MEASURES OF THE SERIOUSNESS OF CRIMESResearchers who have attempted to providescales of the seriousness of crimes have relied uponratings by samples of assessors.4 A single dimensionof "seriousness" has been postulated in which theact is described without any reference to an actor.A nebulous generalized actor must be assumed.However, it is doubtful that the people who arerecruited as assessors find it possible to imagine adisembodied act. Raters probably fit a stereotypedactor to each instance of crime presented in thesample of acts. This produces an unknown sampleof actors (offenders) who have been supplied bythe imagination of the assessors making the ratingsof the crimes. For example, the crime stimulus"theft of $I.00" poses the image of the kind ofperson who could or would steal $1.00; and thestimulus, "theft of $10,000" poses the image of thekind of person who could or would steal $10,000.The two offenders "supplied" to match the stimu-lus are not likely to be the same kind of person.Thus, assessments of the seriousness of acts maywell be confounded with attributes of offenders asascribed by raters. The implications of this may beof considerable significance for all seriousnessscales.3The distinction between micro and macro modelsmay be illustrated by noting that the skill necessary tooperate a village grocery store is not an adequate quali-fication for a director of the budget. The economic theorythat leads to success in a small business is not the same asthe theory that is required to guide a national policy. Thedomestic detail of individual purchases (so much appre-ciated in a village community) does not rise to the levelof the kind of sophisticated abstraction necessary to formthe basis of a national marketing policy.Perhaps too much debate in jurisprudence is at thelevel of the village grocery store theory. Consider, forexample, the fondness for references to "little old ladies"and to the man who rides the local omnibus. This sort ofhomeliness characterizes many opinions on crime policyand legal philosophy.4 T. SELLIN & M. WOLFGANG, MEASUREMENT OF DELI-QUENCY (1964).Theories of "just deserts" propose that the sever-ity of the penalty should match the seriousness ofthe crime, but even according to these theories, themeasurement of seriousness must include some con-cept of culpability, an offender-related character-istic. Indeed, a crime ceases to be defined as suchif the culpability of the offender is reduced to zero.But such reduction of culpability does not, or doesnot greatly, modify the seriousness of the harmdone. The transition from crime to harm and therelationship between the concepts of culpabilityand seriousness need greater consideration and re-search.IMPROVEMENTS IN PREDICTIONThere are many reasons why the strategy ofavoiding predictive statements may not be realisticor even desirable. If one accepts this position, it isnecessary to confront the problem of false positivesand consider what is ethical under conditions ofuncertainty. The replacement of probability bysome subjective certainty is not a satisfactory ap-proach.First, one should estimate the magnitude andprobable impact of the false positive prognosis,both upon individuals and upon the social andlegal system. The present position is that for everyperson correctly identified as dangerous (likely tocommit another crime against the person), six oth-ers will possess the same predictive profile. Thiswas the conclusion of a very thorough and intensivetesting involving a large sample of young offend-ers.5 Given this background, one can considerwhether this level of precision can be increasedand, in addition, what other modifications can bemade to deal with the difficulties arising fromimprecise judgments.It is quite probable that the precision of predic-tion methods can be improved significantly. Thereare three areas where it would be necessary toinvest effort: (a) the basic data, (b) methods ofinput of basic data to analytical systems, and (c)the analytical systems.THE BASIC DATAThe data used in making predictions can giverise to several problems. A first problem arises fromthe fact that existing case papers, the basic sourcedocuments for information used in predictive stud-ies, do not seem to have the accuracy necessary to5 Wenk & Emrich, Assaultive Youth, 9 J. RESEARCHCRIME & DELINQUENCY 171 (1972).1980]LESLIE T. WILKINSwithstand analyses by means of the more powerfulmethods. The information may be mixed with"noise" as well as redundancy. It therefore may beworthwhile to explore the common kinds of record-ing errors and how they influence predictions basedon case papers.A second problem arises from the fact that someitems of information may be predictive, but "un-desirable" for inclusion in an analysis. Race, forexample, may be such an item. Any characteristicthat the individual cannot change voluntarily maybe regarded as suspect on ethical grounds. Thecorrelations may reflect the position of the individ-ual in the "real world" and include factors quiteindependent of the criminal justice processes. Dataobtained in criminal justice research may revealinjustice that cannot be remedied by any actionwithin the criminal justice system. The criminaljustice system is not operating in a perfect society.Where data problems arise from these sources, onemust ensure that legal and ethical considerationsare permitted to outweigh considerations of effi-ciency.A third problem arises from the fact that someimprovements in predictive power may require theuse of information that causes an unjustifiableintrusion upon the personal privacy of an individ-ual. But it is also possible that where offenses havebeen proven against an individual (prediction ofrecidivism), some of the rights safeguarding theprivacy of personal information are diminished.This is a matter that jurisprudence must decide.METHODS OF INPUT (CODING)Methods of input raise two problems for predic-tion. First, all prediction systems have used datawith a fixed time base: the files have been searchedfor information in one operation, and these datahave been coded as information input for analyses.Dynamic procedures of data recording may havepotential for more efficient prediction. Where in-dividuals are incarcerated (or in mental hospitals),it is possible to obtain data on transitional states.The problem with this is that the cost of such datacollection, as well as probable ethical objections,raises concerns that may outweigh the probableincrements in predictability.A second problem stems from the fact that someimportant information may be lost by coding proc-esses which tend to make implicit assumptions (e.g.,A + B = B + A, where A precedes B in time).Coding stage assumptions such as additivity andtransitivity are not removed by analytical methodsthat avoid such assumptions.ANALYTICAL SYSTEMSResearch has been directed towards refining thestatistical methods of analysis, and work in otherfields of application has provided new techniques.The results, however, have been disappointing.Little or no difference can be observed between thepower of quite sophisticated methods (e.g., logodds, discriminant function) and very simple sys-tems (e.g., point allocation as in the Guidelines ofthe United States Parole Commission and unitweights as used fifty years ago). It is possible thatthis result is due, in part, to the quality of the basicdata; simple methods are more resistant to "noise"in the data base. Clearly, this problem needs in-vestigation, which can be done rather simply andinexpensively through the use of simulation meth-ods.Although it is known that all methods of predic-tion provide estimates of approximately equal pre-dictive power, the relationship between the meth-ods as they apply in individual cases has not beeninvestigated. The different methods of predictioncorrelate quite highly with each other, but notperfectly. Thus, there must be some proportion ofcases predicted as "failures" by one method but as"successes" by another. It is possible to guess at thenature of these differences through the use of math-ematical theory, but a thorough practical exami-nation of the matter seems long overdue.SOME ADDITIONAL PROBLEMS IN PREDICTION ANDDECISIONThe prediction of recidivism of offenders hasbeen subject to a wide range of error. But a muchgreater accuracy has been obtained in the predic-tion of the decisions of authorities in the criminaljustice system (e.g., judges, parole officers, proba-tion officers). Why decisions about offenders (suchas sentencing and parole revocations) have provedmore predictable than decisions by offenders (tocommit crime) is unknown. A cross-analysis of datainvolving samples of these two classes of predictionshas not been undertaken. Indeed, there is littleresearch specifically directed at methods of or issuesin prediction. What is known about prediction hasarisen mainly from studies in which predictionmethods were incidental to some other focus. Thus,an investigation devoted solely to methods of pre-diction might prove worthwhile.Many authorities assume that if outcome couldbe predicted with reasonable accuracy, it would bea simple matter to determine the best course ofaction. But it is questionable whether improvedprediction techniques would provide useful infor-[Vol. 71CAREER CRIMINAL SYMPOSIUMmation in many circumstances. In the early daysof prediction in criminology, it was thought that ifparole boards had prediction tables, this wouldassist their decisions. But parole boards did notmake use of the available methods. The decisionsmade by parole boards were concerned with issuesother than "mere prediction of recidivism." Thisbecame clear in the work that led to the develop-ment of Guidelines for the United States ParoleCommission. While the guidelines include a pre-dictive element, it is the lesser of the two majordimensions considered; the main one being theseriousness of the instant crime.Modern decision theory and related practicalmethods can assist in decisionmaking where theobjectives are clearly stated. Prediction methodsmay be useful or even essential as a subset of otheranalytical techniques. In this way, it may be pos-sible to develop more efficient and ethical rules ofdecision and procedure. The main issue today isnot how to make predictions (the techniques ofprediction are known, and there already are goodideas for improving present methods), but ratherwhy to predict and when. An answer to the ques-tion of why to predict will help answer the questionof what to predict.It also may be desirable to simulate the condi-tions of decisionmaking where decisionmakers arerequired to respond to information controlled fortype, probable error, and situation. Predictiontechnique should not be seen as something thatwill stand up on its own. The decision environmentin which the methods of prediction are to beembedded is also an essential element of the pro-gram of research and assessment that is now re-quired.One final question seems to be of outstandingimportance. The question is whether predictionshould be considered in the disposition of offenders.This question is related to many subquestions thatmight challenge the relationship between ethicalconcerns and probability or degrees of belief. Butin the final analysis, the theory of "just deserts"does not appear, either practically or ethically, tohave disposed of the necessity of prediction.

Problems with Existing Prediction Studies and Future Research Needs

https://scholarlycommons.law.northwestern.edu/cgi/viewcontent.cgi?article=6167&context=jclc

Problems with Existing Prediction Studies and Future Research Needs

Abstract

Similar works

Full text

Available Versions

Northwestern University Illinois, School of Law: Scholarly Commons