We investigate the potential of combined features of aerial images and high-resolution interferometric SAR (InSAR) data for building detection in urban areas. It is shown that completeness and correctness may be increased if we integrate both InSAR double-bounce lines and 3D lines of stereo data in addition to building hints of a single optical orthophoto. In order to exploit context information, which is crucial for object detection in urban areas, we use a Conditional Random Field approach. It proves to be a valuable method for context-based building detection with multi-sensor features
