A dynamic programming based matching method for motion estimation, that optimises a Bayesian maximum likelihood function in a 3-D optimisation space, is presented. The Bayesian function consists of a matching cost and an object based 2-D regularisation cost. The method gives results more accurate than block-based matching since the motion boundaries are close to the actual object boundaries

Cosmas, J

Panis, S

Electronics Letters

English

Brunel University Research Archive

using the original MBE pitch analyser, but these errors are effec- tively removed by using the improved pitch analyser (unbroken). We also found that the improved pitch detection algorithm per- formed very well, even without pitch tracking, therefore the algo- rithmic coding delay was shortened to 32ms. A full-duplex MBELP coder employing the improved pitch detection algorithm was successfully implemented in real time on a single C31 DSP, and pitch estimation requires only 16% of the processor resource. Conclusion: We propose a new error measure for spectrum match- ing in MBE pitch analysis of speech. By applying a corrective measure to  the original error measure for pitch estimation. gross pitch errors can be effectively removed. The corrective measure is based on a sum-of-product formula which facilitates fast searching of the optimum pitch period by using partial-sum comparisons. 0 IEE 1996 Electronics Letteu Online No: 19960593 C.-F. Chan and E.W.M. Yu (Department of Electronic Eiigiizeering, City University of Hong Kong, 83, Tut Chee Aveiiue, Koivlooii, Horig 5 M L I I . ~ ~  1996 K o w )  References 1 GRIFFIN,  D.w., and LIM, J s.: ‘Multi-band excitation vocoder‘. IEEE Trans. Acoust., Speech Signal Process., 1988, ASSP-36. (8 ) .  pp, 1223.- 1235 2 YU, E.W.M , and CHAN, c.F.: ‘Efficient multiband excited linear predictive coding of speech at 1.6kbps’. Eurospeech-95. 1995. pp. 685-688 Motion estimation with object based regularisation S. Panis and J.P. Cosmas Indexiizg terms: Motion estimation, Dynamic progrninniiii:: A dynamic programming based matching method for motion estimation, that optimises a Bayesian maximum likelihood function in a 3-D optimisation space, is presented. The Bayesian function consists of a matching cost and an object based 2-D regularisation cost. The method gives results more accurate than block-based matching since the motion boundaries are close to the actual object boundaries. Introduction: A dynamic programming-based matching method for motion estimation is presented which optimises a Bayesian maxi- mum likelihood function that consists of two parts: a matching cost and a 2-D regularisation cost that enforces the motion vectors to be a monotonic function only within objects, thereby allowing an. edge preserving transition between objects and hence object- based regularisation. A 3-D optimisation space is used to accom- modate the two degrees of motion freedom. Fig. 1 Quick and rundorn motion of ohjects Three-dimensional optirnisation: If matching of a raster line from one image to the next is attempted, and one of the objects moves as shown in Fig. 1, then dynamic programming cannot correctly match raster line n in frames 1 and 2. This is apparent in Fig. 2u where the raster lines n from the two consecutive frames is plotted against each other. The dynamic programming will give no-match- ing results in this case. T o  prevent this, the raster line of frame 2 is composed of a collection of segments from other raster lines as in Fig. 2h. In Fig. 2b, a segment from raster line n-1 (Fig. 1) was used to compose the matching raster line of frame 2, and the ver- tical difference per section was recorded. Composition was achieved by having a 3-D optimisation space for dynamic pro- gramming. One axis of the optimisation space i s  the raster line and the other two axes are the possible horizontal and vertical motion vector axes. A frame 2 raster l inen n-I n frame 2 raster line segments - -~ . ,  Fig. 2 ]\‘on march~ihle and nintchable patterns ii Non matchable. h Matchable Cost fzmction: The total cost function for dynamic programming is a combination of a normalised matching cost and a regularisation cost. The normalised matching cost is determined by the quality of matching. The regularisation cost consists of two parts. One part enforces the vectors across a surface to be a monotonic function (monotonicity constrain [ 11) and the other part controls the strength of the monotonicity function so that its effect is dependent on the luminance variations and hence acts strongly inside objects but lightly at the edges. Cox et al. [2] claim that enforcement of regularisation may harm the results at the edges where they are most accurate owing to the rich texture. Geiger et d. [ 13 state that regularisation is necessary in order to obtain accu- rate results even where matching i s  poor, e.g. within objects with poor texture. The cost function presented here takes account of both points of view. By reducing the effect of the monotonicity cost at the edges. the dynamic programming can use the matching cost C ,  ,< as the main decision cost. Since matching near edges is usually better than within a surface, the C,,,, has very high confi- deuce at the edges and this leads the program to a correct decision without regularisation influence. A typical problem occurs at two consecutive pixels that belong to  two different objects with large motion vector difference. If no consideration for edges was there, the monotonicity constraint alone would impose vector uniform- ity. shooting the total cost high and resulting in very large punish- ment. The area would have been interpreted as unmatched. The regularisation cost function is defined as where s is the amount of motion vector jump between consecutive pixels of the raster line and s is the normalised luminance gradient extracted using a Sobel edge detector. The plot of the cost func- tion is shown in Fig. 3. The weights p, E, p are empirically deter- mined constants. For s I 1 i.e. when there is practically no edge, the function is punishing only motion vector jumps and hence act- ing purely as a monotonicity constraint [l]. For  s > 1, i.e. when there is a strong edge, the effectiveness of the motion vector jump function is reduced to allow a jump without strong punishment. According to Fig. 3, as s increases the smoothness criterion is pulled down to allow jumps between consecutive pixels. The total cost TJj, i) is calculated with the recursive function where Y , ~  is the horizontal motion jump between the current candi- date horizontal motion vector and the horizontal motion vector of the previous column, x, is the respective vertical motion jump, n is the current column number, j is the candidate horizontal motion 1-ector and i is the candidate vertical motion vector. (j,,ei,rihesi) s the best predecessor vector. 872 ELECTRONICS LETTERS 9th May 1996 Vol. 32 No. IO of the motion vectors, block artefacts and the mosquito effect are not present The proposed algorithm “a’, used in the object based coder of [4] and experiments showed that errors owing to motion compensation were considerably lower than when block based matching was used, with the performance of the coder being better than that of MPEG-2 Acknowledgments This work was done within the framework of RACE (Research on Advanced Communications in Europe) under contract R2045 ‘DISTIMA’ (DIgital Slrereoscopic IMaging and Applications). 0 1EE 1996 Electr onits Letten Online No 19960598 S Panis (Semen, AG, Z F E  T SN 2, 81730 Munich, Germuny) J.P. Cosmas (Queen Mary and Westfield College, Electronic Engineering Depur tnwnt, Mile End Rmd, London El 4NS, United Kingdom) References 7 Mrrrcli 1996 Fig. 3 Cost function for motion estinwtion n b Fig. 4 ‘Manege’ sequence u First frame, h Second (edited) frame Results: In Fig. 4a and Fig. 4b, frames 1 and 2, respectively, of the sequence ‘Manege’ [3] are shown where, after editing in the second frame, the bus and the bus stop were swapped. The motion esti- mation program was first run without edge consideration (p 0.0) i.e. with the monotonicity constraint applied uniformly on the entire scanline. Motion results were obtained for the bus, whereas the bus stop was classified as an area with no match owing to  punishment of the large motion vector difference at the boundary between the bus and the bus stop. Hence a cheaper alternative path was to ignore the pixels that belonged to the bus stop, and jump until pixels with motion vectors comparable to those of the bus were met. 5 Fig. 5 Motion vector , f Z d  with edge considerution and reconstructed imuge The program with edge consideration p = 0.1 was run on the two frames, and the horizontal motion vector field is shown in Fig. 50. For the bus stop, there is a calculated motion value (arrows pointing left) because the regularisation cost at the bound- ary between the bus and the bus stop was pulled down by the edge factor, therefore the cost was reduced such that it could allow transition from a bus pixel to a bus stop pixel without a jump, and without shooting the total cost up so that it was very high. Fig. 512 shows a reconstruction of frame 1 from frame 2 using the motion vector field. An advantage of this method over the block-based matching is that the motion edges are closer to  the actual luminance edges, which is a Valuable feature for segmentation and motion compen- sation in object based coding. Owing to  the object base accuracy Trellis decoding of combined diversity- coding scheme (MLSD) for fading channels K. Mohamed-Pour and P.G. Farre11 Indexing terms: Trellis coded modulution, Fudiizn, Diversity reception ~ The performance of a receiver using a combination of soft- decision decoding and diversity reception is investigated Tor nonsclective multipath Rayleigh fading channels. A new scheme for soft-diversity, soft-decision detection, maximum likelihood selection and decoding (MLSD), is introduced, in which decisions on the diversity channels and decoding are carried out simultaneously by using a trellis and the Viterbi algorithm. Introduction: Diversity reception is one way to improve the relia- bility of communication without increasing either the transmitted power or the bandwidth. Coding is mother advantageous way which can be used for power-limited and/or bandlimited channels [l]. In this Letter a combination of diversity detection and decod- ing, especially in the soft-decision decoding case, is introduced which improves the performance of the transmission of data over a multipath fading channel, compared with separate diversity detection and decoding. Diversity with sojt decision decodinE: Supposing an L-diversity scheme, the system is modelled as L, independent multipath Rayleigh channels corrupted by additive white Gaussian noise n(t) .  The L receivers are assumed to employ matched filters to  get the maximum signalinoise ratio at sampling times. Values {ri,), i = 1,2 ,___, L ; j = 1,2 ,___, n, are the unquantised samples of the outputs. These samples are then quantised to  Q levels and then accumu- lated in an L x iz buffer matrix as {y) , )  values which can be used in a diversity combination scheme, involving a code of length n. ‘Soft-diversity decoding’ refers to any decoding process on the soft values of the outputs of the channels in a diversity system which may involve different combination techniques. Two such techniques are: ELECTRONICS LETTERS 9th May  1996 Vol. 32 No. IO 873 

Motion estimation with object based regularisation

Abstract

Similar works

Full text

Available Versions

Brunel University Research Archive