Encoding remotely sensed time series data as two-dimensional images for urban change detection using convolution neural networks

Thesis (MSc)--Stellenbosch University, 2021.ENGLISH ABSTRACT: Urban expansion is the most pervasive form of land cover change in South Africa. A method that can effectively detect and indicate areas that have a higher probability of displaying urban change will therefore be a valuable asset to analysts. That is why it is critical to derive a rapid framework that can accurately map urban change. An alternative remote sensing approach that uses multi-temporal time series data and deep learning techniques has been proposed as a potential method for performing a successful urban change detection. The interdisciplinary scientific field of computer vision holds a framework for encoding time-series data as two-dimensional (2D) images for input to a convolution neural network (CNN). Traditional image classifications techniques and more recent studies that have deployed machine learning and deep learning classifiers (namely support vector machine (SVM), random forest (RF), k-nearest neighbour (kNN), long short-term memory (LSTM) and CNN) have been used for urban land cover classification. In this study, a unique framework proposed within computer vision that exploits Gramian angular fields (GAF) and Markov transition fields (MTF) as the transformations for encoding time series data as 2D imagery prior to deep learning classification is investigated for urban change detection. Two main experiments were carried out, both of which utilised the proposed framework for performing an effective urban change detection. The first experiment used coarse resolution data derived from Pretoria using MODIS 500m and 250m normalised difference vegetation index (NDVI). The proposed framework was then deployed, and Gramian angular summation field (GASF), Gramian angular difference field (GADF), and MTF transformations used to encode the time series data. A concatenated encoded image containing the information from all three transformations was formed and was run alongside the three individual transformations. Multiple pre-trained CNN architectures (namely ResNet, DenseNet, InceptionV3, InceptionResNetV2, VGG and MobileNet) were used, from which an urban change detection was derived. It was established that the concatenated images yielded the highest accuracy at 91% and 93% for the 500m and 250m resolution datasets, respectively. The proposed framework was compared to a current state-of-the-art time series classifier (LSTM) to illustrate the effectiveness of encoding and processing deep learning classifiers. The results also outperformed that of other urban change detections studies conducted in South Africa. The second experiment made use of higher resolution Sentinel-2 data derived from a resampled 30m resolution NDVI product of Pretoria. Several investigations were made into the influencing elements that affect the performance of the urban change detection. These were the spatial and temporal resolutions, training data size and different classification schemes. Using the proposed Stellenbosch University https://scholar.sun.ac.za iv framework from the first experiment, the spatial and temporal resolutions were tested. The results showed that an increase in spatial or temporal resolution will have a positive effect on the performance. The 30m resolution dataset yielded a 4% increase over the 250m resolution data tested in the first experiment. Altering the time-series length (TSL) from 32 to 82, the accuracy increased from 96% to 98%, respectively. It was also illustrated that by increasing the amount of training data, one could improve the performance of the change detection. Multiple classifications were performed, and the accuracy assessed using a confusion matrix. It was established that a 70%+ minimum pixel probability and the majority ensemble classifier performed the best. The frameworks generalisability was tested at three different locations (Durban, Gqeberha, and Khayelitsha), and was able to generalise using the Durban dataset. However, the models were unable to generalise using the Gqeberha, and Khayelitsha datasets due to the diverse ecological and climatic properties. The experiments showed that deploying a computer vision framework of encoding multi-temporal time series data as two-dimensional images for an urban change detection using CNN classifications is, in fact possible, and proved to be one of the most effective urban change detection methods performed in South Africa. However, it is recommended that further research deploys a signature extension approach for training the models in order to improve the generalisability. Additional research into using Landsat8 and increased TSL datasets is also recommended.AFRIKAANSE OPSOMMING: Stedelike uitbreiding is die heersende vorm van grondbedekkingsverandering in Suid-Afrika. 'n Metode om gebiede met 'n groter waarskynlikheid van stedelike veranderinge te toon of effektief te kan kan opspoor en aandui, sal 'n waardevolle bate vir ontleders wees. Daarom is dit van kritieke belang om 'n minder tydrowende raamwerk op te stel wat stedelike verandering akkuraat kan karteer. 'n Alternatiewe afstandswaarnemingsbenadering wat multi-temporale tydreeksdata en diepleertegnieke gebruik, word voorgestel as 'n moontlike metode vir suksesvolle opsporing van stedelike veranderinge. Die interdissiplinere wetenskaplike veld van rekenaarvisie bevat 'n raamwerk vir die kodering van tydreeksdata as tweedimensionele beelde wat as invoer dien vir 'n konvolusionele neurale netwerk (CNN). Tradisionele beeldklassifikasietegnieke en meer onlangse studies wat masjienleer- en diepleerklassifiseerders (naamlik ondersteuningsvektormasjien (SVM), ewekansige woud (RF), k-naaste buurtklassifiseerder (kNN), lang-kort-termyn-geheue (LSTM) en CNN) word dikwels gebruik vir klassifikasie van stedelike grondbedekkings. In hierdie studie word 'n unieke raamwerk voorgestel wat binne rekenaarvisie ontwikkel is wat Gramian-hoekvelde (GAF) en Markov-oorgangsvelde (MTF) benut as ‘n transformasie in die kodering van tydreeksdata as tweedimensionele beelde voordat diepleerklassifikasie ondersoek word vir die opsporing van stedelike veranderinge . Twee eksperimente is uitgevoer, wat beide die voorgestelde raamwerk gebruik het vir opsporing van stedelike veranderinge. Die eerste eksperiment het gegewens gebruik van growwe resolusie wat uit Pretoria verkry is, met behulp van MODIS 500m en 250m genormaliseerde verskil plantegroei-indeks (NDVI) data. Die voorgestelde raamwerk is daarna ontplooi deur Gramian hoeksomvelde (GASF), Gramian hoekverskilvelde (GADF) en MTF transformasies te gebruik om die tydreeksdata te kodeer. 'n Saamgevoegde gekodeerde beeld wat al drie transformasies bevat, is gemaak en saam met die drie individuele transformasies analiseer. Veelvuldige vooraf-opgeleide CNN-argitekture (naamlik ResNet, DenseNet, InceptionV3, InceptionResNetV2, VGG en MobileNet) is gebruik, waaruit die stedelike verandering afgelei is. Daar is vasgestel dat die saamgevoegde beelde die hoogste akkuraatheid gelewer het met 91% en 93% vir die datastelle van onderskeidelik 500m en 250m. Die voorgestelde raamwerk is vergelyk met 'n huidige moderne tydreeksklassifiseerder (LSTM) om die doeltreffendheid van kodering en verwerking van 'n diepleerklassifiseerder te illustreer. Die resultate was ook beter as die van ander stedelike veranderingstudies in Suid-Afrika. Die tweede eksperiment het gebruik gemaak van Sentinel-2-data met 'n hoer resolusie, ook afgelei van 'n NDVI-produk vir Pretoria, verwerk na 30m. Verskeie ondersoeke is gedoen om vas te stel wat die faktore is wat die akkuraatheid van die opsporing van stedelike verandering beinvloed, byvoorbeeld, die ruimtelike en temporale resolusies, die grootte van die opleidingsdata en verskillende klassifikasie skemas. Met behulp van die voorgestelde raamwerk van die eerste eksperiment, is die effek van ruimtelike en temporale resolusies getoets. Die resultate het getoon dat 'n toename in ruimtelike of temporale resolusie 'n positiewe uitwerking op die akkuraatheid sal hê. Die datastel met 'n resolusie van 30m het 'n toename van 4% opgelewer in vergelyking met die resolusiedata van 250m wat in die eerste eksperiment getoets is. Deur die tydreekslengte (TSL) van 32 na 82 te verander, het die akkuraatheid toegeneem van 96% tot 98%. Die studie het ook aangedui dat die akkuraatheid van veranderingopsporing sou verbeter kon word deur die hoeveelheid opleidingsdata te vermeerder. Veelvuldige klassifikasie skemas is uitgevoer en die akkuraatheid met behulp van 'n verwarringsmatriks getoets. Daar is vasgestel dat 'n 70%+ minimum pixelwaarskynlikheid en die meerderheidsensemble-klassifiseerder die beste gevaar het. Die veralgemeenbaarheid van die raamwerke is op drie verskillende plekke (Durban, Gqeberha en Khayelitsha) getoets, maar kon slegs in Durban veralgemeen word. Die modelle kon nie stedelike verandering met Gqeberha- en Khayelitsha -datastelle optel nie weens die uiteenlopende ekologiese en klimaatseienskappe. Die eksperimente het getoon dat die implementering van 'n rekenaarvisie raamwerk vir die kodering van multi-temporale tydreeksdata as tweedimensionele beelde vir die opsporing van stedelike veranderinge met behulp van CNN-klassifikasies in werklikheid moontlik is en een van die mees doeltreffende opsporingstegnieke vir stedelike veranderinge in Suid-Afrika kan wees. Dit word egter aanbeveel dat verdere navorsing 'n uitbreidingsbenadering gebruik vir die opleidingsdata vir die modelle om die veralgemenbaarheid te verbeter. Bykomende navorsing oor die gebruik van Landsat8 en verhoogde TSL-datastelle word ook aanbeveel.Master

The ABC of social learning:Affect, Behaviour and Cognition

Debates concerning social learning in the behavioral and the developmental cognitive sciences have largely ignored the literature on social influence in the affective sciences despite having arguably the same object of study. We argue that this is a mistake and that no complete model of social learning can exclude an affective aspect. In addition, we argue that including affect can advance the somewhat stagnant debates concerning the unique characteristics of social learning in humans compared to other animals. We first review the two major bodies of literature in nonhuman animals and human development, highlighting the fact that the former has adopted a behavioral approach while the latter has adopted a cognitive approach, leading to irreconcilable differences. We then introduce a novel framework, affective social learning (ASL), that studies the way we learn about value(s). We show that all three approaches are complementary and focus, respectively, on behavior toward; cognitions concerning; and feelings about objects, events, and people in our environment. All three thus contribute to an affective, behavioral, and cognitive (ABC) story of knowledge transmission: the ABC of social learning. In particular, ASL can provide the backbone of an integrative approach to social learning. We argue that this novel perspective on social learning can allow both evolutionary continuity and ontogenetic development by lowering the cognitive thresholds that appear often too complex for other species and nonverbal infants. Yet, it can also explain some of the major achievements only found in human cultures

Estimating long-term treatment effects in observational data: A comparison of the performance of different methods under real-world uncertainty.

In the presence of time-dependent confounding, there are several methods available to estimate treatment effects. With correctly specified models and appropriate structural assumptions, any of these methods could provide consistent effect estimates, but with real-world data, all models will be misspecified and it is difficult to know if assumptions are violated. In this paper, we investigate five methods: inverse probability weighting of marginal structural models, history-adjusted marginal structural models, sequential conditional mean models, g-computation formula, and g-estimation of structural nested models. This work is motivated by an investigation of the effects of treatments in cystic fibrosis using the UK Cystic Fibrosis Registry data focussing on two outcomes: lung function (continuous outcome) and annual number of days receiving intravenous antibiotics (count outcome). We identified five features of this data that may affect the performance of the methods: misspecification of the causal null, long-term treatment effects, effect modification by time-varying covariates, misspecification of the direction of causal pathways, and censoring. In simulation studies, under ideal settings, all five methods provide consistent estimates of the treatment effect with little difference between methods. However, all methods performed poorly under some settings, highlighting the importance of using appropriate methods based on the data available. Furthermore, with the count outcome, the issue of non-collapsibility makes comparison between methods delivering marginal and conditional effects difficult. In many situations, we would recommend using more than one of the available methods for analysis, as if the effect estimates are very different, this would indicate potential issues with the analyses

Anxiety in families of individuals with neurodevelopmental conditions in the early months of the COVID-19 pandemic in Switzerland

In the spring of 2020, the COVID-19 pandemic generated a health, social, political, and economic crisis that dramatically reduced the institutional support for families of individuals with neurodevelopmental conditions (NDCs). To understand how these families may have experienced and coped with the pandemic, we created an online questionnaire that reached more than 10,000 families in 78 countries. The current manuscript - framed within the International Classification of Functioning, Disability and Health (ICF-10) model - investigates the impact of specific health conditions and personal or environmental factors on the anxiety of families living in Switzerland during the early months of the pandemic. To assess how differences in anxiety over time were predicted by specific health conditions or personal and environmental factors, two separate multilevel analyses were conducted for parents and their children with NDCs (N = 256). First, results showed that only parents reported an increase in anxiety when the pandemic started. Second, concerns related to loss of institutional support and financial and economic problems were the most anxiety-provoking factors for parents, whereas parents reported that the most anxiety-provoking factor for children was their concern about becoming bored. Many parents may have struggled with economic problems and managed multiple extra roles and tasks in their daily lives because institutional support was no longer available. As reported by their parents, although individuals with NDCs did not show an increase in anxiety, they may have struggled with boredom. This result may represent the inability to engage in satisfactory activities in daily life associated with a partial unawareness of the pandemic and the respective protective measures. Further research should more thoroughly investigate the potential effects of the individual’s primary condition, presence and severity of intellectual disability and awareness of the pandemic on the anxiety of individuals with NDCs. Ultimately, we present a series of reflections and practical suggestions that could help guide policymakers in potential future periods of crisis, social estrangement, and distance learning

Charm2000: A >10^8-charm experiment for the turn of the millennium

I discuss the physics reach of a fixed-target charm experiment which can reconstruct >10^8 charm decays, three orders of magnitude beyond the largest extant sample. Such an experiment may run at Fermilab shortly after the Year 2000. In addition to "programmatic" charm physics such as spectroscopy, lifetimes, and tests of QCD, this "Charm2000" experiment will have significant sensitivity to new physics in the areas of CP violation, flavor-changing neutral-current and lepton-number-violating decays, and mixing, and could observe direct CP violation in Cabibbo-suppressed decays at the level predicted by the Standard Model.Comment: 10 pages, 3 PostScript figure

The impact of COVID-19 on individuals with ASD in the US: Parent perspectives on social and support concerns

The COVID-19 pandemic’s disruptions to daily routines and services have proven especially challenging for children with autism spectrum disorder (ASD) and their families. The current retrospective study aimed to determine the impact of the COVID-19 pandemic’s social environmental changes on parental ratings of personal and child concerns about family conflict, opportunities for social interaction, and loss of institutional support (school and therapy services). Analyses of responses from families with ASD in the US determined differences in concerns across three time points which were measured simultaneously: prior to COVID-19, at the start of COVID-19, and at the time of survey completion. From our sample of 246 school-aged children, parents retrospectively reported significantly increasing levels of concern for both themselves and their children over time, with parents’ personal concern levels rated consistently higher than their ratings of their child’s level of concern. Concerns about loss of institutional support were higher for parents of children reported as having co-occurring intellectual disability. Further, parents of younger children also reported more concerns about loss of services, as well as more social concerns. For parent ratings of child concerns, children who were reportedly aware of COVID-19 were determined to have higher levels of social concerns and concerns about loss of institutional support. Meanwhile, the child’s age and gender did not impact their parent ratings of child concerns. The increased level of parental and child-perceived concerns over the course of the pandemic suggests a need for improved service delivery and support for these families. The high levels of concerns observed in the current study provide support for the need to assess families’ priorities and tailor services to best meet families’ needs. This will potentially increase the quality of life of family members, and improve ASD services across the lifespan, and improve outcomes

Hoosiers’ Health in a Changing Climate: A Report from the Indiana Climate Change Impacts Assessment

In the coming decades, Indiana’s changing climate will bring with it higher temperatures, longer heat waves, more extremely hot days and more frequent extreme storm events. Those changes will affect the health of Hoosiers in every part of the state. This report from the Indiana Climate Change Impacts Assessment (IN CCIA) describes historical and future climate-related health impacts that affect Hoosiers