We aim to perform medical note comprehension and medical image processing, with an ultimate goal of cross-domain aggregation into a cooperative disease management system. The dissertation focuses on the initial technical development of each domain utilizing both physical modeling and deep learning methods. Medical notes and images taken from patients during their clinical visits are essential for patient care management. In this thesis, natural language processing techniques are developed for patient private information removal from medical reports, and image processing techniques are developed for semantic segmentation on different imaging modalities to achieve higher accuracy and enhanced structural integrity of the segmentation. Moreover, we demonstrate the importance of the manual labels used as the ground truth for supervised learning and assessment in the biomedical applications, and further propose a refinement scheme to improve label quality. Future directions would be integrating the complementary text and image information into a single robust system
