Network-based ranking in social systems: three challenges

Ranking algorithms are pervasive in our increasingly digitized societies, with important real-world applications including recommender systems, search engines, and influencer marketing practices. From a network science perspective, network-based ranking algorithms solve fundamental problems related to the identification of vital nodes for the stability and dynamics of a complex system. Despite the ubiquitous and successful applications of these algorithms, we argue that our understanding of their performance and their applications to real-world problems face three fundamental challenges: (i) Rankings might be biased by various factors; (2) their effectiveness might be limited to specific problems; and (3) agents' decisions driven by rankings might result in potentially vicious feedback mechanisms and unhealthy systemic consequences. Methods rooted in network science and agent-based modeling can help us to understand and overcome these challenges.Comment: Perspective article. 9 pages, 3 figure

Genetic programming application in predicting fluid loss severity.

Numerous wells worldwide encounter significant, costly, and time-consuming lost circulation issues during drilling or while deploying tubulars across naturally fractured or induced fractured formations. This can potentially lead to formation damage, wellbore instability, and even blowouts. Effectively addressing this problem and restoring fluid circulation becomes crucial to curbing non-productive time and overall operational expenses. Although numerous methods have been introduced, a universally accepted industry solution for predicting lost circulation remains absent due to the complex interplay of various factors influencing its severity. Anticipating the onset of circulation loss is imperative to mitigate its impacts, minimise costs, and reduce risks to personnel and the environment. In this study, an innovative machine learning approach employing multigene genetic algorithms is utilised to analyse a dataset of 16,970 drilling datasets from 61 wells within the Marun oil field, located in Iran, where severe loss of circulation occurred. Geological characteristics, operational drilling parameters, and the properties of the drilling fluid were all considered. The dataset encompasses 19 parameters, of which seven are chosen as inputs for predicting lost circulation incidents. These inputs are then employed to construct a predictive model, employing an 85:15 training-to-test data ratio. To assess the model's performance, unseen datasets are utilised. The novelty of this study lies in the proposed model's consideration of a concise set of relevant input parameters, particularly real-time surface drilling parameters that are easily accessible for every well. The model attains a remarkable level of prediction accuracy for fluid loss, as indicated by various performance indices. The results indicate a mean absolute error of 1.33, a root mean square error of 2.58, and a coefficient of determination of 0.968. The suggested prediction model is optimised not only for data reduction but also for universal prediction and compatibility with other existing platforms. Moreover, it aids drilling engineers in implementing suitable mitigation strategies and designing optimal values for key operational surface parameters, both prior to and during drilling operations

Network Parameterisation and Activation Functions in Deep Learning

Deep learning, the study of multi-layered artificial neural networks, has received tremendous attention over the course of the last few years. Neural networks are now able to outperform humans in a growing variety of tasks and increasingly have an impact on our day-to-day lives. There is a wide range of potential directions to advance deep learning, two of which we investigate in this thesis:(1) One of the key components of a network are its activation functions. The activations have a big impact on the overall mathematical form of the network. The \textit{first paper} studies generalisation of neural networks with rectified linear activations units (“ReLUs”). Such networks partition the input space into so-called linear regions, which are the maximally connected subsets on which the network is affine. In contrast to previous work, which focused on obtaining estimates of the number of linear regions, we proposed a tropical algebra-based algorithm called TropEx to extract coefficients of the linear regions. Applied to fully-connected and convolutional neural networks, TropEx shows significant differences between the linear regions of these network types. The \textit{second paper} proposes a parametric rational activation function called ERA, which is learnable during network training. Although ERA only adds about ten parameters per layer, the activation significantly increases network expressivity and makes small architectures have a performance close to large ones. ERA outperforms previous activations when used in small architectures. This is relevant because neural networks keep growing larger and larger and the computational resources they require result in greater costs and electricity usage (which in turn increases the CO2 footprint).(2) For a given network architecture, each parameter configuration gives rise to a mathematical function. This functional realisation is far from unique and many different parameterisations can give rise to the same function. Changes to the parameterisation that do not change the function are called symmetries. The \textit{third paper} theoretically studies and classifies all the symmetries of 2-layer networks using the ReLU activation. Finally, the \textit{fourth paper} studies the effect of network parameterisation on network training. We provide a theoretical analysis of the effect that scaling layers have on the gradient updates. This provides a motivation for us to propose a Cooling method, which automatically scales the network parameters during training. Cooling reduces the reliance of the network on specific tricks, in particular the use of a learning rate schedule

Developing a Global Healthcare Innovation Index

Our understanding of medicine is being revolutionised by the pace of science. But not all the potential innovations in life sciences and medical technology are taken up into everyday practice in healthcare, even when they are shown to be beneficial. For the poorest people in the world, many innovations are not accessible because they are either unaffordable or unsuitable for their health systems. Tackling this gap requires the development of appropriate and affordable health technologies and novel business models. In the more advanced health systems there is a disconnection between the effort on research and development (R&D) and how much of this makes it into mainstream healthcare practice. Even the most evidence-based and affordable innovations can fail or are only taken up patchily, whether we compare across countries, or between localities or health organisations within countries. And technological innovation can be a problem for those responsible for paying for health systems. New technologies often increase costs because they allow us to treat more people for a longer part of their lives. Yet the general view amongst politicians, managers and others involved in healthcare is that health systems across the world need new thinking. They are increasingly facing escalating demand from an ageing population and the growing incidence of chronic disease. Healthcare is consuming an ever-increasing share of gross domestic product (GDP). The search is on for ways of providing the best quality healthcare as affordably as possible. The health technology industries – pharmaceutical and biotechnology, medical devices, information technology and the built environment (design, engineering and construction) – drive much of the innovation that takes place in healthcare. They are very big business. Collectively these companies have global revenues in the order of USD 2 trillion a year, about a quarter of overall global spending on healthcare. But they too are experiencing a changing landscape – an evolving market for their products, a changing balance of power across health systems as governments and payers seek to control costs, hence pressure on their business models. Innovation is regarded by economists and politicians as one of the main drivers of economic growth. It helps to explain why some companies, regions and countries perform better than others in terms of higher productivity and income. For companies involved in the health technology sector, and governments in countries where they are located, there is concern to ensure that their business models are sustainable and continue to successfully deliver new products to the market

Sparse MoEs meet Efficient Ensembles

Machine learning models based on the aggregated outputs of submodels, either at the activation or prediction levels, often exhibit strong performance compared to individual models. We study the interplay of two popular classes of such models: ensembles of neural networks and sparse mixture of experts (sparse MoEs). First, we show that the two approaches have complementary features whose combination is beneficial. This includes a comprehensive evaluation of sparse MoEs in uncertainty related benchmarks. Then, we present Efficient Ensemble of Experts (E$^3$), a scalable and simple ensemble of sparse MoEs that takes the best of both classes of models, while using up to 45% fewer FLOPs than a deep ensemble. Extensive experiments demonstrate the accuracy, log-likelihood, few-shot learning, robustness, and uncertainty improvements of E$^3$ over several challenging vision Transformer-based baselines. E$^3$ not only preserves its efficiency while scaling to models with up to 2.7B parameters, but also provides better predictive performance and uncertainty estimates for larger models.Comment: 59 pages, 26 figures, 36 tables. Accepted at TML

An investigation into weighted data fusion for content-based multimedia information retrieval

Content Based Multimedia Information Retrieval (CBMIR) is characterised by the combination of noisy sources of information which, in unison, are able to achieve strong performance. In this thesis we focus on the combination of ranked results from the independent retrieval experts which comprise a CBMIR system through linearly weighted data fusion. The independent retrieval experts are low-level multimedia features, each of which contains an indexing function and ranking algorithm. This thesis is comprised of two halves. In the first half, we perform a rigorous empirical investigation into the factors which impact upon performance in linearly weighted data fusion. In the second half, we leverage these finding to create a new class of weight generation algorithms for data fusion which are capable of determining weights at query-time, such that the weights are topic dependent

Data Augmentation based Cellular Traffic Prediction in Edge Computing Enabled Smart City

This is the author accepted manuscript; the final version is available from IEEE via the DOI in this record.With the massive deployment of 5G cellular infrastructures, traffic prediction has become an indispensable part of the cellular resource management system in order to provide reliable and fast communication services that can meet the increasing Quality-of-Service (QoS) requirements of smart city. A promising approach for handling this problem is to introduce intelligent methods to implement a highly effective and efficient cellular traffic prediction model. Meanwhile, integrating the multi-access edge computing framework in 5G cellular networks facilitates the application of intelligent traffic prediction models by enabling their implementation at the network edge. However, the data shortage and privacy issues may still be obstacles for training a robust and accurate prediction model at the edge. To address these issues, we propose a data augmentation based cellular traffic prediction model where a generative adversarial network-based data augmentation method is proposed to improve the prediction performance while protecting data privacy, and a long short-term memory based sequence-to-sequence model is used to achieve the flexible multi-step cellular traffic prediction. The experimental results on a real-world city-scale cellular traffic dataset reveal that our model achieves up to 48.49% improvement of the prediction accuracy compared to four typical reference models.National Key R&D Program of ChinaNational Natural Science Foundation of ChinaChina Scholarship Counci

Navigating the Environmental, Social, and Governance (ESG) landscape: constructing a robust and reliable scoring engine - insights into Data Source Selection, Indicator Determination, Weighting and Aggregation Techniques, and Validation Processes for Comprehensive ESG Scoring Systems

This white paper explores the construction of a reliable Environmental, Social, and Governance (ESG) scoring engine, with a focus on the importance of data sources and quality, selection of ESG indicators, weighting and aggregation methodologies, and the necessary validation and benchmarking procedures. The current challenges in ESG scoring and the importance of a robust ESG scoring system are addressed, citing its increasing relevance to stakeholders. Furthermore, different data types, namely self-reported data, third-party data, and alternative data, are critically evaluated for their respective merits and limitations. The paper further elucidates the complexities and implications involved in the choice of ESG indicators, illustrating the trade-offs between standardized and customized approaches. Various weighting methodologies including equal weighting, factor weighting, and multi-criteria decision analysis are dissected. The paper culminates in outlining processes for validating the ESG scoring engine, emphasizing the correlation with financial performance, and conducting robustness and sensitivity analyses. Practical examples through case studies exemplify the implementation of the discussed techniques. The white paper aims to provide insights and guidelines for practitioners, academics, and policy makers in designing and implementing robust ESG scoring systems. This ESG white paper explores the interplay between Environmental, Social, and Governance (ESG) factors and green finance. We begin by defining ESG and green finance, exploring their evolution, and discussing their importance in financial markets. The paper emphasises the role of green finance in driving sustainable development. Next, we delve into the ESG scoring landscape. We outline various methodologies, key players in ESG ratings, and present challenges and criticisms of current ESG scoring systems. In the third section, we propose a blueprint for a reliable ESG scoring engine. This includes discussion on various data sources and the selection of ESG indicators, highlighting the role of materiality assessment, and the balance between standardized and customized indicators. We then discuss different methodologies for weighting and aggregating these indicators. The paper concludes with the necessity of validation and benchmarking of ESG scores, particularly correlating them with financial performance and performing robustness and sensitivity analyses

Evaluating visually grounded language capabilities using microworlds

Deep learning has had a transformative impact on computer vision and natural language processing. As a result, recent years have seen the introduction of more ambitious holistic understanding tasks, comprising a broad set of reasoning abilities. Datasets in this context typically act not just as application-focused benchmark, but also as basis to examine higher-level model capabilities. This thesis argues that emerging issues related to dataset quality, experimental practice and learned model behaviour are symptoms of the inappropriate use of benchmark datasets for capability-focused assessment. To address this deficiency, a new evaluation methodology is proposed here, which specifically targets in-depth investigation of model performance based on configurable data simulators. This focus on analysing system behaviour is complementary to the use of monolithic datasets as application-focused comparative benchmarks. Visual question answering is an example of a modern holistic understanding task, unifying a range of abilities around visually grounded language understanding in a single problem statement. It has also been an early example for which some of the aforementioned issues were identified. To illustrate the new evaluation approach, this thesis introduces ShapeWorld, a diagnostic data generation framework. Its design is guided by the goal to provide a configurable and extensible testbed for the domain of visually grounded language understanding. Based on ShapeWorld data, the strengths and weaknesses of various state-of-the-art visual question answering models are analysed and compared in detail, with respect to their ability to correctly handle statements involving, for instance, spatial relations or numbers. Finally, three case studies illustrate the versatility of this approach and the ShapeWorld generation framework: an investigation of multi-task and curriculum learning, a replication of a psycholinguistic study for deep learning models, and an exploration of a new approach to assess generative tasks like image captioning.Qualcomm Award Premium Research Studentship, Engineering and Physical Sciences Research Council Doctoral Training Studentshi