The technology of moving objects detection has become an important research subject for its extensive application prospect. In this paper, it is presented that interframe difference algorithm and background difference algorithm are combined to update the background. The algorithm can deal with the flaw of background difference algorithm. The mathematical morphology method is employed to denoise the image, which may be helpful to improve the accuracy of the detection. The Pyramid algorithm is used to compress each frame data of video sequence. Then, the detecting and tracking of moving objects are tested on the hardware platform (DM643) and the software frame (RF5). The running speed is about 3 times faster than before. The result shows that the accuracy demanded by the detection is met. This method can provide a useful reference for similar application.</jats:p