Tracking movement of insects in a social group (such as ants) is challenging, because the individuals are not only similar in appearance but also likely to perform intensive body contact and sudden movement adjustment (start/stop, direction changes). To address this challenge, we introduce an online multi-object tracking framework that combines both the motion and appearance information of ants. We obtain the appearance descriptors by using the ResNet model for offline training on a small (N=50) sample dataset. For online association, a cosine similarity metric computes the matching degree between historical appearance sequences of the trajectory and the current detection. We validate our method in both indoor (lab setup) and outdoor video sequences. The results show that our model obtains 99.3% ± 0.5% MOTA and 91.9% ± 2.1% MOTP across 24,050 testing samples in five indoor sequences, with real-time tracking performance. In an outdoor sequence, we achieve 99.3% MOTA and 92.9% MOTP across 22,041 testing samples. The datasets and code are made publicly available for future research in relevant domains
