We describe the development of a system that automates data collection, metadata extraction and analysis of spatio-temporal multi-modal data, combining data management and data analysis to provide an efficient resource for clinicians. Though the system is extensible to many applications, the current focus is on managing Multiple Sclerosis (MS) lesion data, which are disparate streams of image, numeric, and text data. In order to discover patterns of MS pathology and plan early and effective treatment, multispectral magnetic resonance (MR) image streams collected over time need to be correlated efficiently with each other and with patient performance and clinical data streams. 1