Experiments involving sequential recognition techniques and feature ordering schemes were performed on 23 feature samples of vowel spectra and 12 feature samples of remotely sensed agricultural crop data. Since each experiment dealt with two pattern classes, Wald\u27s sequential probability ratio test was used. The test was implemented with both fixed and time-varying stopping boundaries. Feature ordering was accomplished by both dispersion analysis and the divergence criterion

Newman, R.

Reisine, B.

English

Embry-Riddle Aeronautical University

The Space Congress® Proceedings 1970 (7th) Technology Today and Tomorrow Apr 1st, 8:00 AM Practical Applications of Sequential Pattern Recognition Techniques R. Newman School of Electrical Engineering, Purdue University, Lafayette, Indiana B. Reisine School of Electrical Engineering, Purdue University, Lafayette, Indiana Follow this and additional works at: https://commons.erau.edu/space-congress-proceedings Scholarly Commons Citation Newman, R. and Reisine, B., "Practical Applications of Sequential Pattern Recognition Techniques" (1970). The Space Congress® Proceedings. 2. https://commons.erau.edu/space-congress-proceedings/proceedings-1970-7th/session-2/2 This Event is brought to you for free and open access by the Conferences at Scholarly Commons. It has been accepted for inclusion in The Space Congress® Proceedings by an authorized administrator of Scholarly Commons. For more information, please contact commons@erau.edu. PRACTICAL APPLICATIONS OF SEQUENTIAL PATTEK1 lECOGHIflOI TECHNIQUESR. NevmanSchool of Electrical Engineering Purdue University Lafayette, IndianaB* 'lei sineSchool of Electrical Engineering Purdue University Lafayette, IndianaABSTRACTExperiments involving sequential recognition tech­ niques and feature ordering schemes were performed on 23 feature samples of vowel spectra and 12 fea­ ture samples of remotely sensed agricultural crop data. Since each experiment dealt with two pattern classes, Wald's sequential probability ratio test was used. The test was implemented with both fixed and time-varying stopping boundaries. Feature ordering was accomplished by both dispersion analy­ sis and the divergence criterion.p(x|«,.)=[(2.)N/2 |K.| 1/2 ]-1 axp[-| (X - M.)^1 (X - It}] , i = 1,2 (2)then the above discriminant function yieldsD^X) - log P(BI ) - i log | K4 | - |(X - MjfINTRODUCTIONA pattern recognition system consists of a feature extractor and a classifier (see Figure l). The feature extractor makes measurements of salient characteristics of the input patterns. These are called feature measurements and based on them, the classifier assigns each input pattern to one of the possible pattern classes. We are concerned with those classifiers that are sequential in nature. That is, those that utilize the feature measurements one at a time in performing the classification. The advantages of sequential techniques are realized when the cost of taking feature measurements is high or the speed of classification is important.TECHNIQUESThe problem of classifying patterns from two classes is formulated here as a statistical decision pro­ blem. N feature measurements, denoted by X^X^,*-*, Xjj, are given for each pattern. The two pattern classes are called u>j_ and u£. For each pattern class u>j, j = 1,2, it is assumed that the probabili­ ty density function of this feature vector X,p(XJ u>.)> is known* A /^•?<-.«•«•?»-?v»«*%4- isiv*«%4-•?*%*•> *JA discriminant function,Di (X) = log 1,2 (1)is now defined which can easily be implemented by a Bayes classifyer. When DI(X) >Dj(X), i,j = 1,2, then X is said to be in class o^.When p(X | u^) i a 1,2, is a nmltivariate Gaussian density function with mean vector % and covariance matrix B, i.e.,(5)This is the discriminant function used as the samples to be classified are assumed to be Gaussian in nature. In all recognition schemes used in this paper, the training procedure has: been to compute M^ and K. from the first 75 sannleB of each clsss. xFor each sample to be classified, B^ and Bg were computed. If D^X) - B2: (X) was positive, tte sample was placed in class. 1 and, if" negjative in class 2.In the above procedure it is necessaxy to me •' measurements from, each, pattern to be classified. Quite often this is inconvenient (because of «r time consumption) and it becomes desirable to a scheme using less feature measurements* Mien there are only two pattern classes to be recognised* Wald's sequential probability ratio "best (SRf) can be applied. Here the feature measurements can De taken one at a time* At the nth stage- of the sequential process, that is, after the nth ftefcure measurement is taken, the classifier computes the sequential probability ratioAn -where p(X | ittj), i = 1,2 is the (moltivariate BI- dimenslonal)' conditional probability density frac­ tion of X for pattern class «^» lm is then com­ pared with two stopping boundaries" A and 1* If 1B > A, then the decision is; that X is in class u^j and if "kn < B, then the decision is X is In *2-1class u£. If B < Xn < A then an additional fea­ ture measurement will be taken and the process will proceed to the (n+l)st stage (see Figure 2). The two stopping boundaries are related to the error (misrecognition) probabilities by the follow­ ing expressions1 - e21 and B =C12"21 1 - e.12(5)where eij is the probability of deciding X is in class w±, when actually X is in class oug is true, i,j, = 1,2.The above sequential probability ratio test can be generalized to the case where the stopping bound­ aries become time varying instead of remaining constant. Let g-^(n) and g2(n) be either.constants or monotonically nonincreasing and nondecreasing functions of n, respectively. The classifier continuously takes measurements as long as the sequential probability ratio Xn lies between e&iv ; an^ e&2v / y that is, the sequential process continues by taking additional feature measurements as long aseg2 (n) <x <egi<n)^ n = 1,2) ... (6)If Xn > ebj- v *', then the decision is that X is in class a>L and if Xn < egg(a), then the decision is that X is in class u£. If gi(n) and gp(n) are constants it is easily seen that the standard Wald's SPRT can be considered as a special case of this modified SPRT. The fact that, in general, gl(n) and g2 (n) can be made functions of n enables the sequential classifier to be designed such that the expected number of features measurements in reaching a terminal decision and the probability of misrecognition may be controlled in advance.Here, the following two functions have been used:(7)DIVERGENCE CRITERION FOR FEATURE ORDERINGThe divergence between two pattern classes, whose samples are assumed to be Miltivariate Gaussian Distributed, with equal covariance matrices, can be expressed as t 1 ':(9)M. and ^ are the mean vectors inverse of the covariance matrix.— Tand K" is theIn the present study, the covariance matrices for the two classes were unequal, so the average was used.The above relationship was used to find optimal feature subsets of 1,2, ...,n-l features from feature sets of 2,3>«««>n features respectively, in the following way (see Figure k):1. The divergence between classes, for all n possible n-1 feature subsets of the original fea­ ture set, was computed.2. The subsets corresponding to the largest value of divergence was chosen as optimal and the feature, which was deleted from the original set to form this subset, was placed at the end of the list of ordered features.3- The divergence between classes, for all n-1 possible n-2 feature subsets of the new feature set, was computed.I*. The subset corresponding to the largest value of divergence was chosen as optimal and the feature, which was deleted to form this subset, was placed next to last on the ordered feature list.5. This procedure was continued until finally the best subset consisting of one feature was selected and the ordered feature list was completed.It should be noted that this technique is restric­ tive in that any feature discarded at a given level cannot be a member of a smaller optimal subset. To overcome this limitation, the divergence between classes would have to be computed for nCn-1 possible subsets to find optimal subsets of 1,2, ...,n-l features.where 0 < r < 1, a' > 0, and N is the prespecified number of feature measurements where the truncation occurs and the classifier is forced to reach a terminal decision (see Figure 3). The value of a' was obtained from the relation= e"a ! 1ra(8)where E-^(n) is the expected member of features to be used for the modified SPRT when X is in class 1. E'(n) was obtained by using the average number of features found in the truncated case.DISPERSION ANALYSIS FOR FEATURE ORDERING 'Vowel data was obtained (see Acknowledgement) in which the features had been ordered via a linear transformation. The technique utilized can be described as follows:1. A sample covariance matrix was computed as an approximation to the true covariance matrix of the data .2* The eigenvalues and corresponding eigenvectors of this matrix were determined and the eigenvectors were subsequently normalized and arranged according to the descending order of their associated eigen­ values. The resulting set of vectors constituted a generalized Kahunen-Loeve coordinate system.2-23« Each input feature vector was transformed by this coordinate system, and the first fifteen com­ ponents were retained.EXPERIMENTAL RESULTS AND DISCUSSIONThe data employed in the described recognition schemes consisted of vowel samples and remotely sensed agricultural crop data.Vowel samples were obtained by band-pass filtering recorded utterances of the form /h^CVC/, of each vowel, in each of the 23 consonantal environments, at 35 frequencies covering the range of the speech spectrum (250 to 10,000 hz). Filter outputs were rectified, smoothed, sampled and subsequently passed through an analog-to-digital converter. The data for each vowel in its final form consisted of a deck of IBM cards, each one containing the ampli­ tude, in decimal form, of each filter output during a particular sample interval. In all instances of the use of untransformed vowel data, 23 features were selected, corresponding to the first 23 filter outputs, These features were chosen because they cover the range of possible values for the 1st and 2nd formant frequencies of the vowels under con­ sideration (see Table k).Agricultural crop data was obtained by the de­ tection and recording of both reflected and emitted electromagnetic radiation energy, from specific earth surface areas. Spatially scanning radio­ meters were used to obtain relative measurements of energy from, the ground in 12 discrete spectral bands. The first ten bands encompassed visible wave lengths and the last two covered the reflect­ ive infrared portion of the spectrum. The data, first recorded on a 12-channel magnetic tape, underwent analog-to-digital conversion, and was then formatted and recorded on a data storage tape. IBM cards were punched from this tape such that each crop was characterized by a deck of cards, each one containing the relative energy in each spectral band, for a particular region of ground surface.It can be seen (Table l) that for the data under consideration, high recognition accuracies are achieved with the Bayes fixed sample size classi­ fier. However, the Bayes classifier necessitates the use of all feature measurements, which is quite inefficient.Table 2 shows the results of applying a sequential recognition technique and feature ordering. In the case of £ and & the number of features required for classification has decreased 8oA$ from the fixed sample size classifier without ordering and 83*6$ with ordering by divergence while recognition accuracy has dropped no more than 1.2#.The results for faf and /z>/ in Table 2 are a con­ clusive demonstration of the value of feature ordering. In the unordered sequential case, the number of features used is 81$ less than in the fixed sample size case. However, along with this is a decrease in accuracy of 27-3^» This startling decrease is understandable if one considers the rank of the various features (filter outputs)after ordering* !fbe sad fea­ ture* placed %'ftij.the first four features on the classifier was trained and hosed decisions were found to be poor and/or odsleadiog* The classification accuracy using' ordered feature* Is seen to he comparable to that of the fiicsd saoQiLe the average nturiher of 1* for the divergence criterion and 90*1^ analysis*In the case of the agrictd&ural 2)there is again, a substantial reduction in the num­ ber of features without erf" This result also indicates of in conjunction with sequential teehniqpie* 10 powerful than the of sequential alone *Through considerations of 3 5* Itis seen that the use ofboundaries causes a trade-off hetween theof recognition and the average ofused to reach, a decision* tothe average nuriber of f mtqr be reducedonly a slight drop in, accuracy*COHGMISIOifIn reviewing the experimental inthis study, it is reasonable touse of sequential recognition tectalqpe*feature ordering schenesefficient 'tit.lll.zat/ij0iii of featureUse of the time-varying imequation (7) is not, optisni* Is a forfurther1 research to devise selectionfor a', r, .and, functional of"boundaries.We 'wish, to thank both thethe Laboratory of AgriciO.tural atPurdue, for the vowel datatively.We would also like to Br» K» S* Fta, for' ing the foundation uisieii. thiaThis work was sugportedGK-1970.RECTHEHJCES(1) T* Iferill and 1« II. Green, ""la the Effective­ ness of Receptors in Recognition Syatess," |EEE T gains * JM j||gj: * _ jjjjjgy j 2* 1-965*(2) G* 1. Bodies ami. J» F* Bestial, "Speeeh Analy­ sis," Qniireraity Report, 5.5-9* 1965*(3) i. W. :Utagjh:ts f A* S» and I- P. Li, "Re­ search on, Word Shotting," Clearinghouse, Dept* of Cowserce, .»* 2SO-9^» 1969*2-3(I*-) K. S. Fu, D. A. Landgrebe, and T. L. Phillips, "Information Processing of Remotely Sensed Agricul­ tural Data," Proceedings IEEE, Vol. 57, April 1969 > pp. 639-653-BIBLIOGRAPHY(1) K. S. Fu, Sequential Methods in Pattern Re­ cognition and Machine Learning, Academic Press, 1955^(2) J. L. Flanagan, Speech Analysis, Synthesis and Perception, Springer-Verlag, 196*5• •INPUTPATTERNFEATURE EXTRACTORFEATURECLASSIFIER DECISIONMEASUREMENTSFIGURE I. A PATTERN RECOGNITION SYSTEM.2-4Xn EXCEEDS BOUNDARIESCOMPUTE SAMPLE MEANS AND COVARIANCESTRAINING PROCEDURE— »— •*CALL IN AFEATURE MEASUREMENT— »COMPUTE Xn— ^COMPARE *nSTOPPING BOUNDARIESr—X ,DECIDE PATTERN CLASS——— »STOPJnBOUNDARIESFIGURE 2 FUOW DIAGRAM OF A SEQUENTIAL PATTIRN RICO@N!TION SYSTfM,g,(n)Log ALog BTIME-VARYING BOUNDARIESNUMBER OF FEATURES, nTRUNCATED BOUNDARYFIGURE 3. DIAGRAM OF TRUNCATED AND TIME-VARYING STOPPING BOUNDARIES.COMPUTE VERGENCEOFn,n-l FEATURE SUBSETSSELECT SUBSETCORRESPONDINGTO LARGESTVALUE OFDIVERGENCEPLACE DISCARDED FEATURE IN UOWEST VACANTPOSITION ON ORDERED FEATURE1 IQTL.IW 1— •»DOESn-l«l ? *YES .—— i» NO»>LETn«ii-iFIGURE 4. FLOW DIAGRAM OF ORDERING BY THI DIVIRSINCI CRITfRION,TABLE I. BAYES FIXED SAMPLE SIZE RESULTSCLASSES|£|.|Q|.X0CORN SOYBEANSNUMBER OF SAMPLES171166370% ACCURACY96.599.087.5NUMBER OF FEATURES USED232312TABLE 2.SEQUENTIAL RESULTS WITH TRUNCATED STOPPING BOUNDARIESCLASSES\c\.lei.\alW.lol.aex\000CORN SOY BEANSCORN SOY BEANSFEATURE ORDERINGUNORDEREDDIVERGENCE CRITERIONUNORDEREDDIVERGENCE CRITERIONDISPERSION ANALYSISUNORDEREDDIVERGENCE CRITERIONNUMBER OF SAMPLES171171166166132370370% ACCURACY97.095.371.798.698.590.085.1AVERAGE NUMBER OF FEATURES4.593.784.321.002.2710.3133.432-8TABLE 3.SEQUENTIAL RESULTS WITH TIME VARYIU6 STOPPING BOUNDARIESr1.000.500.251.000.500.251.000.500.25CLASSES\a\. oIfl|.|3|l#l, 3||0|. 0|#|,|0|l#|.|3CORN SOYBEANSCORN SOYBEANSCORN SOYBEANSFEATURE ORDERINGUNORDEREDUNORDEREDUNORDEREDDISPERSION ANALDISPERSION ANAL.DISPERSION ANAL.UNORDEREDUNORDEREDUNORDEREDNUMBER OF SAMPLES1666616613232132370370370% ACCURACY71.772.972.996.296.996.979.288.489.7AVERAGE NUMBER Of' FEATURES4.20 .4.364.431.66; 1.711728.9810.281 1 .OOTABLE 4.FEATURES SELECTED181FILTER NUMBERi2345678910112CENTER FREQUENCY OF FILTER286317368428473526585643707780864966FILTER NUMBER1314151617181920212223CENTER FREQUENCY OF FILTER107011571290142515601713190120872316255028142-9ro i

Practical Applications of Sequential Pattern Recognition Techniques

https://commons.erau.edu/cgi/viewcontent.cgi?article=3132&amp;context=space-congress-proceedings

Practical Applications of Sequential Pattern Recognition Techniques

Abstract

Similar works

Full text

Available Versions

Embry-Riddle Aeronautical University