In this work, we study the problem of anomaly detection of the trajectories of objects in a visual scene. For this purpose, we propose a novel representation for trajectories utilizing covariance features. Representing trajectories via covariance features enables us to calculate the distance between the trajectories of different lengths. After setting this proposed representation and calculation of distances, anomaly detection is achieved by sparse representations on nearest neighbours. Conducted experiments on both synthetic and real datasets show that the proposed method yields results which are outperforming or comparable with state of the art
