Real-time food intake classification and energy expenditure estimation on a mobile device

© 2015 IEEE.Assessment of food intake has a wide range of applications in public health and life-style related chronic disease management. In this paper, we propose a real-time food recognition platform combined with daily activity and energy expenditure estimation. In the proposed method, food recognition is based on hierarchical classification using multiple visual cues, supported by efficient software implementation suitable for realtime mobile device execution. A Fischer Vector representation together with a set of linear classifiers are used to categorize food intake. Daily energy expenditure estimation is achieved by using the built-in inertial motion sensors of the mobile device. The performance of the vision-based food recognition algorithm is compared to the current state-of-the-art, showing improved accuracy and high computational efficiency suitable for realtime feedback. Detailed user studies have also been performed to demonstrate the practical value of the software environment

Content-based Image Understanding with Applications to Affective Computing and Person Recognition in Natural Settings

Understanding the visual content of images is one of the most important topics in computer vision. Many researchers have tried to teach the machine to see and perceive like human. In this dissertation, we develop several new approaches for image understanding with applications to affective computing, and person detection and recognition. Our proposed method applied to fashion photo analysis can understand the aesthetic quality of photos. Further, a bilinear model that takes into account the relative confidence of region proposals and the mutual relationship between multiple labels is developed to boost multi-label classification. It is evaluated both on object recognition and aesthetic attributes learning. We also develop a person detection and recognition system in natural settings that can robustly handle various pose, viewpoints, and lighting conditions. The system is then put into several real scenarios that has different amount of labelled data. Our algorithm that utilizes unlabelled data reduces the effort needed for data annotation while achieving similar results as with labelled data