Automatic Prediction of Building Age from Photographs

We present a first method for the automated age estimation of buildings from unconstrained photographs. To this end, we propose a two-stage approach that firstly learns characteristic visual patterns for different building epochs at patch-level and then globally aggregates patch-level age estimates over the building. We compile evaluation datasets from different sources and perform an detailed evaluation of our approach, its sensitivity to parameters, and the capabilities of the employed deep networks to learn characteristic visual age-related patterns. Results show that our approach is able to estimate building age at a surprisingly high level that even outperforms human evaluators and thereby sets a new performance baseline. This work represents a first step towards the automated assessment of building parameters for automated price prediction.Comment: Preprint of paper to appear in ACM International Conference on Multimedia Retrieval (ICMR) 2018 Conferenc

Where and Who? Automatic Semantic-Aware Person Composition

Image compositing is a method used to generate realistic yet fake imagery by inserting contents from one image to another. Previous work in compositing has focused on improving appearance compatibility of a user selected foreground segment and a background image (i.e. color and illumination consistency). In this work, we instead develop a fully automated compositing model that additionally learns to select and transform compatible foreground segments from a large collection given only an input image background. To simplify the task, we restrict our problem by focusing on human instance composition, because human segments exhibit strong correlations with their background and because of the availability of large annotated data. We develop a novel branching Convolutional Neural Network (CNN) that jointly predicts candidate person locations given a background image. We then use pre-trained deep feature representations to retrieve person instances from a large segment database. Experimental results show that our model can generate composite images that look visually convincing. We also develop a user interface to demonstrate the potential application of our method.Comment: 10 pages, 9 figure

Visual ageing of human faces in three dimensions using morphable models and projection to latent structures

We present an approach to synthesising the effects of ageing on human face images using three-dimensional modelling. We extract a set of three dimensional face models from a set of two-dimensional face images by fitting a Morphable Model. We propose a method to age these face models using Partial Least Squares to extract from the data-set those factors most related to ageing. These ageing related factors are used to train an individually weighted linear model. We show that this is an effective means of producing an aged face image and compare this method to two other linear ageing methods for ageing face models. This is demonstrated both quantitatively and with perceptual evaluation using human raters.Postprin

Retinal Nerve Fiber Layer Features Identified by Unsupervised Machine Learning on Optical Coherence Tomography Scans Predict Glaucoma Progression.

Purpose:To apply computational techniques to wide-angle swept-source optical coherence tomography (SS-OCT) images to identify novel, glaucoma-related structural features and improve detection of glaucoma and prediction of future glaucomatous progression. Methods:Wide-angle SS-OCT, OCT circumpapillary retinal nerve fiber layer (cpRNFL) circle scans spectral-domain (SD)-OCT, standard automated perimetry (SAP), and frequency doubling technology (FDT) visual field tests were completed every 3 months for 2 years from a cohort of 28 healthy participants (56 eyes) and 93 glaucoma participants (179 eyes). RNFL thickness maps were extracted from segmented SS-OCT images and an unsupervised machine learning approach based on principal component analysis (PCA) was used to identify novel structural features. Area under the receiver operating characteristic curve (AUC) was used to assess diagnostic accuracy of RNFL PCA for detecting glaucoma and progression compared to SAP, FDT, and cpRNFL measures. Results:The RNFL PCA features were significantly associated with mean deviation (MD) in both SAP (R2 = 0.49, P < 0.0001) and FDT visual field testing (R2 = 0.48, P < 0.0001), and with mean circumpapillary RNFL thickness (cpRNFLt) from SD-OCT (R2 = 0.58, P < 0.0001). The identified features outperformed each of these measures in detecting glaucoma with an AUC of 0.95 for RNFL PCA compared to an 0.90 for mean cpRNFLt (P = 0.09), 0.86 for SAP MD (P = 0.034), and 0.83 for FDT MD (P = 0.021). Accuracy in predicting progression was also significantly higher for RNFL PCA compared to SAP MD, FDT MD, and mean cpRNFLt (P = 0.046, P = 0.007, and P = 0.044, respectively). Conclusions:A computational approach can identify structural features that improve glaucoma detection and progression prediction

State-of-the-art on research and applications of machine learning in the building life cycle

Fueled by big data, powerful and affordable computing resources, and advanced algorithms, machine learning has been explored and applied to buildings research for the past decades and has demonstrated its potential to enhance building performance. This study systematically surveyed how machine learning has been applied at different stages of building life cycle. By conducting a literature search on the Web of Knowledge platform, we found 9579 papers in this field and selected 153 papers for an in-depth review. The number of published papers is increasing year by year, with a focus on building design, operation, and control. However, no study was found using machine learning in building commissioning. There are successful pilot studies on fault detection and diagnosis of HVAC equipment and systems, load prediction, energy baseline estimate, load shape clustering, occupancy prediction, and learning occupant behaviors and energy use patterns. None of the existing studies were adopted broadly by the building industry, due to common challenges including (1) lack of large scale labeled data to train and validate the model, (2) lack of model transferability, which limits a model trained with one data-rich building to be used in another building with limited data, (3) lack of strong justification of costs and benefits of deploying machine learning, and (4) the performance might not be reliable and robust for the stated goals, as the method might work for some buildings but could not be generalized to others. Findings from the study can inform future machine learning research to improve occupant comfort, energy efficiency, demand flexibility, and resilience of buildings, as well as to inspire young researchers in the field to explore multidisciplinary approaches that integrate building science, computing science, data science, and social science