Collisions at intersections are common and their consequences are often severe. This paper addresses the need for information on accident causation; a knowledge that can be used to obtain more effective countermeasures. A novel method that can be applied to data recorded in a groundbased observation system or similar is proposedfor classifying vehicle interactions into a set ofpredefined traffic scenarios. The classification isbased on possible combinations of trajectories of two interacting vehicles that have passed through an intersection. Additionally, the authors present an incident detection algorithm that uses the classified vehicle interactions. This algorithm constitutes the core of a video-based automatic incident detection at intersections (AIDI) system. The performance of the AIDI system was successfullyverified both in a driving simulator and in real traffic conditions