Natively probabilistic computation

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Brain and Cognitive Sciences, 2009.Includes bibliographical references (leaves 129-135).I introduce a new set of natively probabilistic computing abstractions, including probabilistic generalizations of Boolean circuits, backtracking search and pure Lisp. I show how these tools let one compactly specify probabilistic generative models, generalize and parallelize widely used sampling algorithms like rejection sampling and Markov chain Monte Carlo, and solve difficult Bayesian inference problems. I first introduce Church, a probabilistic programming language for describing probabilistic generative processes that induce distributions, which generalizes Lisp, a language for describing deterministic procedures that induce functions. I highlight the ways randomness meshes with the reflectiveness of Lisp to support the representation of structured, uncertain knowledge, including nonparametric Bayesian models from the current literature, programs for decision making under uncertainty, and programs that learn very simple programs from data. I then introduce systematic stochastic search, a recursive algorithm for exact and approximate sampling that generalizes a popular form of backtracking search to the broader setting of stochastic simulation and recovers widely used particle filters as a special case. I use it to solve probabilistic reasoning problems from statistical physics, causal reasoning and stereo vision. Finally, I introduce stochastic digital circuits that model the probability algebra just as traditional Boolean circuits model the Boolean algebra.(cont.) I show how these circuits can be used to build massively parallel, fault-tolerant machines for sampling and allow one to efficiently run Markov chain Monte Carlo methods on models with hundreds of thousands of variables in real time. I emphasize the ways in which these ideas fit together into a coherent software and hardware stack for natively probabilistic computing, organized around distributions and samplers rather than deterministic functions. I argue that by building uncertainty and randomness into the foundations of our programming languages and computing machines, we may arrive at ones that are more powerful, flexible and efficient than deterministic designs, and are in better alignment with the needs of computational science, statistics and artificial intelligence.by Vikash Kumar Mansinghka.Ph.D

Using Spatial Microsimulation techniques in the Aggregation of Environmental Benefit Values: An Application to Corncrake Conservation on Irish Farmland

This paper considers the use of spatial microsimulation techniques in the growing area of Benefits Transfer. Benefits transfer involves the results of existing environmental valuation studies being applied to different policy contexts such as alternative populations or environmental sites. The spatial microsimulation model developed in this paper uses a combinational optimatisation technique called simulated annealing to match the Irish Census of Agriculture data to a Contingent Valuation Survey that contains information on Irish farmers' willingness to pay (WTP) to have the corncrake restored as a common sight in the Irish countryside. We then use this matched farm survey and Census information to produce small area population environmental benefit microdata estimates for the year 2005. These figures are then aggregated to get a total value figure for the farming community of Corncrake conservation in Ireland and compared to figures derived using more standard approaches to calculating aggregate environment benefit values

Heuristic search methods and cellular automata modelling for layout design

This thesis was submitted for the degree of Doctor of Philosophy and awarded by Brunel University.Spatial layout design must consider not only ease of movement for pedestrians under normal conditions, but also their safety in panic situations, such as an emergency evacuation in a theatre, stadium or hospital. Using pedestrian simulation statistics, the movement of crowds can be used to study the consequences of different spatial layouts. Previous works either create an optimal spatial arrangement or an optimal pedestrian circulation. They do not automatically optimise both problems simultaneously. Thus, the idea behind the research in this thesis is to achieve a vital architectural design goal by automatically producing an optimal spatial layout that will enable smooth pedestrian flow. The automated process developed here allows the rapid identification of layouts for large, complex, spatial layout problems. This is achieved by using Cellular Automata (CA) to model pedestrian simulation so that pedestrian flow can be explored at a microscopic level and designing a fitness function for heuristic search that maximises these pedestrian flow statistics in the CA simulation. An analysis of pedestrian flow statistics generated from feasible novel design solutions generated using the heuristic search techniques (hill climbing, simulated annealing and genetic algorithm style operators) is conducted. The statistics that are obtained from the pedestrian simulation is used to measure and analyse pedestrian flow behaviour. The analysis from the statistical results also provides the indication of the quality of the spatial layout design generated. The technique has shown promising results in finding acceptable solutions to this problem when incorporated with the pedestrian simulator when demonstrated on simulated and real-world layouts with real pedestrian data.This study was funded by the University Science of Malaysia and Kementerian Pengajian Tinggi Malaysia

Thermodynamic AI and the fluctuation frontier

Many Artificial Intelligence (AI) algorithms are inspired by physics and employ stochastic fluctuations. We connect these physics-inspired AI algorithms by unifying them under a single mathematical framework that we call Thermodynamic AI. Seemingly disparate algorithmic classes can be described by this framework, for example, (1) Generative diffusion models, (2) Bayesian neural networks, (3) Monte Carlo sampling and (4) Simulated annealing. Such Thermodynamic AI algorithms are currently run on digital hardware, ultimately limiting their scalability and overall potential. Stochastic fluctuations naturally occur in physical thermodynamic systems, and such fluctuations can be viewed as a computational resource. Hence, we propose a novel computing paradigm, where software and hardware become inseparable. Our algorithmic unification allows us to identify a single full-stack paradigm, involving Thermodynamic AI hardware, that could accelerate such algorithms. We contrast Thermodynamic AI hardware with quantum computing where noise is a roadblock rather than a resource. Thermodynamic AI hardware can be viewed as a novel form of computing, since it uses a novel fundamental building block. We identify stochastic bits (s-bits) and stochastic modes (s-modes) as the respective building blocks for discrete and continuous Thermodynamic AI hardware. In addition to these stochastic units, Thermodynamic AI hardware employs a Maxwell's demon device that guides the system to produce non-trivial states. We provide a few simple physical architectures for building these devices and we develop a formalism for programming the hardware via gate sequences. We hope to stimulate discussion around this new computing paradigm. Beyond acceleration, we believe it will impact the design of both hardware and algorithms, while also deepening our understanding of the connection between physics and intelligence.Comment: 47 pages, 18 figures, Added relevant reference

A Systematic Investigation of Multi-Objective Evolutionary Algorithms Applied to the Water Distribution System Problem

Water distribution systems (WDSs) are one of society’s most important infrastructure assets. They consist of a great number of pumps, valves, junctions and a tremendous number of pipes that connect these nodes within the system, all of which induce a significant capital cost at the time of construction. However, there is no singular option for designing a WDS, and each potential design affects the cost and performance of the system differently (i.e., the pressure at each node and flow rates for each pipe). To identify solutions with a better trade-off between the cost and performance, multi-objective evolutionary algorithms (MOEAs) provide a robust optimisation tool to solve this type of problem. This PhD thesis focuses on improving and developing a more effective MOEA for WDS problems, and optimisation problems in general. The first stage of the research is to study the impact of select critical processes in MOEAs on algorithm performance and understand the reasons behind the performance observations. There are two chapters related to the first stage. The second stage is to develop a proposed General Multi-Objective Evolutionary Algorithm (GMOEA) and compare this with existing MOEAs for WDS problems. This is associated with the third content chapter. In the first paper, the impact of the operators on an algorithm’s performance has been studied. The operators are the key component for exchange of information between solutions in populations to produce offspring solutions, thereby exploring alternative regions of the search space. These have a significant impact on an algorithm’s search behaviour. However, the composition and number of operators that should be included in an MOEA is generally fixed, based on choices made by the developers of these algorithms. To explore this issue, an assessment was conducted via comprehensive numerical experiments that isolate the influence of the size of the operator set, as well as its composition. In addition, the relative influence of other search processes affecting search behaviour (e.g., the selection strategy and hyperheuristic) have been studied. It has been found that operator set size is a dominant factor affecting algorithm performance, having a greater influence than operator set composition and other search processes affecting algorithm search behaviour. Moreover, it was also found that an existing MOEAs’ performance can be improved by simply increasing the number of operators used within the algorithm. This finding can be applied to justify the usage of operators for designing a new MOEA in the future. In the second paper, a new convex hull contribution selection strategy for population-based MOEAs (termed CHCGen) has been proposed and compared with existing MOEAs in order to study the impact of the selection strategy on MOEA performance. It has been found that the CHCGen selection strategy is able to emphasise selection of the population of solutions on the convex hull of the non-dominated set of solutions. The CHCGen selection strategy has demonstrated that it can also improve an existing MOEAs’ performance. The finding suggests different selection strategies have an impact on MOEA performance. In addition, CHCGen can be used for developing a new MOEA in the future. In the third paper, a new multi-objective evolutionary algorithm, called GMOEA(CHCGen,12,T,A)1 has been proposed by conducting comprehensive numerical experiments to determine the optimised component configuration for each MOEA process. The components considered within the algorithm construction include: the selection strategy, hyperheuristic, and operator set size. The numerical experiments not only explore the impact of each process’s component on algorithm performance comprehensively, but also investigate the correlation of each pairwise combination of the process’s components. In addition, the optimal form of the algorithm GMOEA(CHCGen,12,T,A) was compared with seven other existing MOEAs with an extended computational budget for a range of WDS problems. From the results, GMOEA(CHCGen,12,T,A) was shown not only to have outperformed all other MOEAs considered, but also to find a greater number of new Pareto front solutions for intermediate and large scale problems.Thesis (Ph.D.) -- University of Adelaide, School of Civil, Environmental and Mining Engineering, 202

Multisource Remote Sensing based Impervious Surface Mapping

Impervious surface (IS) not only serves as a key indicator of urbanization, but also affects the micro-ecosystem. Therefore, it is essential to monitor IS distribution timely and accurately. Remote sensing is an effective approach as it can provide straightforward and consistent information over large area with low cost. This thesis integrates multi-source remote sensing data to interpretate urban patterns and provide more reliable IS mapping results. Registration of optical daytime and nighttime lights (NTL) data is developed in the first contribution. An impervious surface based optical-to-NTL image registration algorithm with iterative blooming effect reduction (IS_iBER) algorithm is proposed. This coarse-to-fine procedure investigates the correlation between optical and NTL features. The iterative registration and blooming effect reduction method obtains precise matching results and reduce the spatial extension of NTL. Considering the spatial transitional nature of urban-rural fringes (URF) areas, the second study proposed approach for URF delineation, namely optical and nighttime lights (NTL) data based multi-scale URF (msON_URF).The landscape heterogeneity and development vitality derived from optical and NTL features are analyzed at a series of scales to illustrate the urban-URF-rural pattern. Results illustrate that msON_URF is effective and practical for not only concentric, but also polycentric urban patterns. The third study proposes a nighttime light adjusted impervious surface index (NAISI) to detect IS area. Parallel to baseline subtraction approaches, NAISI takes advantage of features, rather than spectral band information to map IS. NAISI makes the most of independence between NTL-ISS and pervious surface to address the high spectral similarity between IS and bare soil in optical image. An optical and NTL based spectral mixture analysis (ON_SMA) is proposed to achieve sub-pixel IS mapping result in the fourth study. It integrates characteristics of optical and NTL imagery to adaptively select local endmembers. Results illustrate the proposed method yields effective improvement and highlight the potential of NTL data in IS mapping. In the fifth study, GA-SVM IS mapping algorithm is investigated with introduction of the achieved urban-URF-rural spatial structure. The combination of optical, NTL and SAR imagery is discussed. GA is implemented for feature selection and parameter optimization in each urban scenario

The ingenuity of common workmen: and the invention of the computer

Since World War II, state support for scientific research has been assumed crucial to technological and economic progress. Governments accordingly spent tremendous sums to that end. Nothing epitomizes the alleged fruits of that involvement better than the electronic digital computer. The first such computer has been widely reputed to be the ENIAC, financed by the U.S. Army for the war but finished afterwards. Vastly improved computers followed, initially paid for in good share by the Federal Government of the United States, but with the private sector then dominating, both in development and use, and computers are of major significance.;Despite the supposed success of public-supported science, evidence is that computers would have evolved much the same without it but at less expense. Indeed, the foundations of modern computer theory and technology were articulated before World War II, both as a tool of applied mathematics and for information processing, and the computer was itself on the cusp of reality. Contrary to popular understanding, the ENIAC actually represented a movement backwards and a dead end.;Rather, modern computation derived more directly, for example, from the prewar work of John Vincent Atanasoff and Clifford Berry, a physics professor and graduate student, respectively, at Iowa State College (now University) in Ames, Iowa. They built the Atanasoff Berry Computer (ABC), which, although special purpose and inexpensive, heralded the efficient and elegant design of modern computers. Moreover, while no one foresaw commercialization of computers based on the ungainly and costly ENIAC, the commercial possibilities of the ABC were immediately evident, although unrealized due to war. Evidence indicates, furthermore, that the private sector was willing and able to develop computers beyond the ABC and could have done so more effectively than government, to the most sophisticated machines.;A full and inclusive history of computers suggests that Adam Smith, the eighteenth century Scottish philosopher, had it right. He believed that minimal and aloof government best served society, and that the inherent genius of citizens was itself enough to ensure the general prosperity

Remote sensing of impervious surface area and its interaction with land surface temperature variability in Pretoria, South Africa

Includes summary for chapter 1-5Pretoria, City of Tshwane (COT), Gauteng Province, South Africa is one of the cities that continues to experience rapid urban sprawl as a result of population growth and various land use, leading to the change of natural vegetation lands into impervious surface area (ISA). These are associated with transportation (paved roads, streets, highways, parking lots and sidewalks) and cemented buildings and rooftops, made of completely or partly impermeable artificial materials (e.g., asphalt, concrete, and brick). These landscapes influence the micro-climate (e.g., land surface temperature, LST) of Pretoria City as evidenced by the recent heat waves characterized by high temperature. Therefore, understanding ISA changes will provide information for city planning and environmental management. Conventionally, deriving ISA information has been dependent on field surveys and manual digitizing from hard copy maps, which is laborious and time-consuming. Remote sensing provides an avenue for deriving spatially explicit and timely ISA information. Numerous methods have been developed to estimate and retrieve ISA and LST from satellite imagery. There are limited studies focusing on the extraction of ISA and its relationship with LST variability across major cities in Africa. The objectives of the study were: (i) to explore suitable spectral indices to improve the delineation of built-up impervious surface areas from very high resolution multispectral data (e.g., WorldView-2), (ii) to examine exposed rooftop impervious surface area based on different colours, and their interplay with surface temperature variability, (iii) to determine if the spatio-temporal built-up ISA distribution pattern in relation to elevation influences urban heat island (UHI) extent using an optimal analytical scale and (iv) to assess the spatio-temporal change characteristics of ISA expansion using the corresponding surface temperature (LST) at selected administrative subplace units (i.e., local region scale). The study objectives were investigated using remote sensing data such as WorldView-2 (a very high-resolution multispectral sensor), medium resolution Landsat-5 Thematic Mapper (TM) and Landsat-8 OLI (Operational Land Imager) and TIRS (Thermal Infrared Sensor) at multiple scales. The ISA mapping methods used in this study can be grouped into two major categories: (i) the classification-based approach consisting of an object-based multi-class classification with overall accuracy ~90.4% and a multitemporal pixel-based binary classification. The latter yielded an area under the receiver operating characteristic curve (AUROC) = 0.8572 for 1995, AUROC = 0.8709 for 2005, AUROC = 0.8949 for 2015. (ii) the spectral index-based approach such as a new built-up extraction index (NBEI) derived in this study which yielded a high AUROC = ~0.82 compared to Built-up Area Index (BAI) (AUROC = ~0.73), Built-up spectral index (BSI) (AUROC = ~0.78), Red edge / Green Index (RGI) (AUROC = ~0.71) and WorldView-Built-up Index (WV-BI) (AUROC = ~0.67). The multitemporal built-up Index (BUI) also estimated with AUROC = 0.8487 for 1993, AUROC = 0.8302 for 2003, AUROC = 0.8790 for 2013. This indicates that all these methods employed, mapped ISA with high predictive accuracy from remote sensing data. Furthermore, the single-channel algorithm (SCA) was employed to retrieve LST from the thermal infrared (TIR) band of the Landsat images. The LST overall retrieval error for the entire study generally was quite low (overall root mean square RMSE ≤ ~1.48OC), which signifies that the Landsat TIR used provided good results for further analysis. In conclusion, the study showed the potential of multispectral remote sensing data to quantify ISA and evaluate its interaction with surface temperature variability despite the complex urban landscape in Pretoria. Also, using impervious surface LST as a complementary metric in this research helped to reveal urban heat island distribution and improve understanding of the spatio-temporal developing trend of urban expansion at a local spatial scale.Rapid urbanization because of population growth has led to the conversion of natural lands into large man-made landscapes which affects the micro-climate. Rooftop reflectivity, material, colour, slope, height, aspect, elevation are factors that potentially contribute to temperature variability. Therefore, strategically designed rooftop impervious surfaces have the potential to translate into significant energy, long-term cost savings, and health benefits. In this experimental study, we used the semi-automated Environment for Visualizing Images (ENVI) Feature Extraction that uses an object-based image analysis approach to classify rooftop based on colours from WorldView-2 (WV-2) image with overall accuracy ~90.4% and kappa coefficient ~0.87 respectively. The daytime retrieved surface temperatures were derived from 15m pan-sharpened Landsat 8 TIRS with a range of ~14.6OC to ~65OC (retrieval error = 0.38OC) for the same month covering Lynwood Ridge a residential area in Pretoria. Thereafter, the relationship between the rooftops and surface temperature (LST) were examined using multivariate statistical analysis. The results of this research reveal that the interaction between the applicable rooftop explanatory features (i.e., reflectance, texture measures and topographical properties) can explain over 22.10% of the variation in daytime rooftop surface temperatures. Furthermore, analysis of spatial distribution between mean daytime surface temperature and the residential rooftop indicated that the red, brown and green roof surfaces show lower LST values due to high reflectivity, high emissivity and low heat capacity during the daytime. The study concludes that in any study related to the spatial distribution of rooftop impervious surface area surface temperature, effect of various explanatory variables must be considered. The results of this experimental study serve as a useful approach for further application in urban planning and sustainable development.Evaluating changes in built-up impervious surface area (ISA) to understand the urban heat island (UHI) extent is valuable for governments in major cities in developing countries experiencing rapid urbanization and industrialization. This work aims at assessing built-up ISA spatio-temporal and influence on land surface temperature (LST) variability in the context of urban sprawl. Landsat-5 Thematic Mapper (TM) and Landsat-8 OLI (Operational Land Imager) and TIRS (Thermal Infrared Sensor) were used to quantify ISA using built-up Index (BUI) and spatio-temporal dynamics from 1993-2013. Thereafter using a suitable analytical sampling scale that represents the estimated ISA-LST, we examined its distribution in relation to elevation using the Shuttle Radar Topography Mission (SRTM) and also create Getis-Ord Gi* statistics hotspot maps to display the UHI extent. The BUI ISA extraction results show a high predictive accuracy with area under the receiver operating characteristic curve, AUROC = 0.8487 for 1993, AUROC = 0.8302 for 2003, AUROC = 0.8790 for 2013. The ISA spatio-temporal changes within ten years interval time frame results revealed a 14% total growth rate during the study year. Based on a suitable analytical scale (90x90) for the hexagon polygon grid, the majority of ISA distribution across the years was at an elevation range of between >1200m – 1600m. Also, Getis-Ord Gi* statistics hotspot maps revealed that hotspot regions expanded through time with a total growth rate of 19% and coldspot regions decreased by 3%. Our findings can represent useful information for policymakers by providing a scientific basis for sustainable urban planning and management.Over the years, rapid urban growth has led to the conversion of natural lands into large man-made landscapes due to enhanced political and economic growth. This study assessed the spatio-temporal change characteristics of impervious surface area (ISA) expansion using its surface temperature (LST) at selected administrative subplace units (i.e., local region scale). ISA was estimated for 1995, 2005 and 2015 from Landsat-5 Thematic Mapper (TM) and Landsat-8 OLI (Operational Land Imager) and TIRS (Thermal Infrared Sensor) images using a Random Forest (RF) algorithm. The spatio-temporal trends of ISA were assessed using an optimal analytical scale to aggregate ISA LST coupled with weighted standard deviational ellipse (SDE) method. The ISA was quantified with high predictive accuracy (i.e., AUROC = 0.8572 for 1995, AUROC = 0.8709 for 2005, AUROC = 0.8949 for 2015) using RF classifier. More than 70% of the selected administrative subplaces in Pretoria experienced an increase in growth rate (415.59%) between 1995 and 2015. LST computations from the Landsat TIRS bands yielded good results (RMSE = ~1.44OC, 1.40OC, ~0.86OC) for 1995, 2005 and 2015 respectively. Based on the hexagon polygon grid (90x90), the aggregated ISA surface temperature weighted SDE analysis results indicated ISA expansion in different directions at the selected administrative subplace units. Our findings can represent useful information for policymakers in evaluating urban development trends in Pretoria, City of Tshwane (COT).Globally, the unprecedented increase in population in many cities has led to rapid changes in urban landscape, which requires timely assessments and monitoring. Accurate determination of built-up information is vital for urban planning and environmental management. Often, the determination of the built-up area information has been dependent on field surveys, which is laborious and time-consuming. Remote sensing data is the only option for deriving spatially explicit and timely built-up area information. There are few spectral indices for built-up areas and often not accurate as they are specific to impervious material, age, colour, and thickness, especially using higher resolution images. The objective of this study is to test the utility of a new built-up extraction index (NBEI) using WorldView-2 to improve built-up material mapping irrespective of material type, age and colour. The new index was derived from spectral bands such as Green, Red edge, NIR1 and NIR2 bands that profoundly explain the variation in built-up areas on WorldView-2 image (WV-2). The result showed that NBEI improves the extraction of built-up areas with high accuracy (area under the receiver operating characteristic curve, AUROC = ~0.82) compared to the existing indices such as Built-up Area Index (BAI) (AUROC = ~0.73), Built-up spectral index (BSI) (AUROC = ~0.78 ), Red edge / Green Index (RGI) (AUROC = ~0.71) and WorldView-Built-up Index (WV-BI) (AUROC = ~0.67). The study demonstrated that the new built-up index could extract built-up areas using high-resolution images. The performance of NBEI could be attributed to the fact that it is not material specific, and would be necessary for urban area mapping.Environmental SciencesD. Phil. (Environmental Sciences

A Hybrid Lehmer Code Genetic Algorithm and Its Application on Traveling Salesman Problems

Traveling Salesman Problems (TSP) is a widely studied combinatorial optimization problem. The goal of the TSP is to find a tour which begins in a specific city, visits each of the remaining cities once and returns to the initial cities such that the objective functions are optimized, typically involving minimizing functions like total distance traveled, total time used or total cost. Genetic algorithms were first proposed by John Holland (1975). It uses an iterative procedure to find the optimal solutions to optimization problems. This research proposed a hybrid Lehmer code Genetic Algorithm. To compensate for the weaknesses of traditional genetic algorithms in exploitation while not hampering its ability in exploration, this new genetic algorithm will combine genetic algorithm with 2-opt and non-sequential 3-opt heuristics. By using Lehmer code representation, the solutions created by crossover parent solutions are always feasible. The new algorithm was used to solve single objective and multi-objectives Traveling Salesman Problems. A non Pareto-based technique will be used to solve multi-objective TSPs. Specifically we will use the Target Vector Approach. In this research, we used the weighted Tchebycheff function with the ideal points as the reference points as the objective function to evaluate solutions, while the local search heuristics, the 2-opt and non-sequential 3-opt heuristics, were guided by a weighted sum function