291 research outputs found
Meta-learning algorithms and applications
Meta-learning in the broader context concerns how an agent learns about their own learning, allowing them to improve their learning process. Learning how to learn is not only beneficial for humans, but it has also shown vast benefits for improving how machines learn. In the context of machine learning, meta-learning enables models to improve their learning process by selecting suitable meta-parameters that influence the learning. For deep learning specifically, the meta-parameters typically describe details of the training of the model but can also include description of the model itself - the architecture. Meta-learning is usually done with specific goals in mind, for example trying to improve ability to generalize or learn new concepts from only a few examples.
Meta-learning can be powerful, but it comes with a key downside: it is often computationally costly. If the costs would be alleviated, meta-learning could be more accessible to developers of new artificial intelligence models, allowing them to achieve greater goals or save resources. As a result, one key focus of our research is on significantly improving the efficiency of meta-learning. We develop two approaches: EvoGrad and PASHA, both of which significantly improve meta-learning efficiency in two common scenarios. EvoGrad allows us to efficiently optimize the value of a large number of differentiable meta-parameters, while PASHA enables us to efficiently optimize any type of meta-parameters but fewer in number.
Meta-learning is a tool that can be applied to solve various problems. Most commonly it is applied for learning new concepts from only a small number of examples (few-shot learning), but other applications exist too. To showcase the practical impact that meta-learning can make in the context of neural networks, we use meta-learning as a novel solution for two selected problems: more accurate uncertainty quantification (calibration) and general-purpose few-shot learning. Both are practically important problems and using meta-learning approaches we can obtain better solutions than the ones obtained using existing approaches. Calibration is important for safety-critical applications of neural networks, while general-purpose few-shot learning tests model's ability to generalize few-shot learning abilities across diverse tasks such as recognition, segmentation and keypoint estimation.
More efficient algorithms as well as novel applications enable the field of meta-learning to make more significant impact on the broader area of deep learning and potentially solve problems that were too challenging before. Ultimately both of them allow us to better utilize the opportunities that artificial intelligence presents
An ensemble model for predictive energy performance:Closing the gap between actual and predicted energy use in residential buildings
The design stage of a building plays a pivotal role in influencing its life cycle and overall performance. Accurate predictions of a building's performance are crucial for informed decision-making, particularly in terms of energy performance, given the escalating global awareness of climate change and the imperative to enhance energy efficiency in buildings. However, a well-documented energy performance gap persists between actual and predicted energy consumption, primarily attributed to the unpredictable nature of occupant behavior.Existing methodologies for predicting and simulating occupant behavior in buildings frequently neglect or exclusively concentrate on particular behaviors, resulting in uncertainties in energy performance predictions. Machine learning approaches have exhibited increased accuracy in predicting occupant energy behavior, yet the majority of extant studies focus on specific behavior types rather than investigating the interactions among all contributing factors. This dissertation delves into the building energy performance gap, with a particular emphasis on the influence of occupants on energy performance. A comprehensive literature review scrutinizes machine learning models employed for predicting occupants' behavior in buildings and assesses their performance. The review uncovers knowledge gaps, as most studies are case-specific and lack a consolidated database to examine diverse behaviors across various building types.An ensemble model integrating occupant behavior parameters is devised to enhance the accuracy of energy performance predictions in residential buildings. Multiple algorithms are examined, with the selection of algorithms contingent upon evaluation metrics. The ensemble model is validated through a case study that compares actual energy consumption with the predictions of the ensemble model and an EnergyPlus simulation that takes occupant behavior factors into account.The findings demonstrate that the ensemble model provides considerably more accurate predictions of actual energy consumption compared to the EnergyPlus simulation. This dissertation also addresses the research limitations, including the reusability of the model and the requirement for additional datasets to bolster confidence in the model's applicability across diverse building types and occupant behavior patterns.In summary, this dissertation presents an ensemble model that endeavors to bridge the gap between actual and predicted energy usage in residential buildings by incorporating occupant behavior parameters, leading to more precise energy performance predictions and promoting superior energy management strategies
A bi-objective hybrid vibration damping optimization model for synchronous flow shop scheduling problems
Flow shop scheduling deals with the determination of the optimal sequence of jobs processing on machines in a fixed order with the main objective consisting of minimizing the completion time of all jobs (makespan). This type of scheduling problem appears in many industrial and production planning applications. This study proposes a new bi-objective mixed-integer programming model for solving the synchronous flow shop scheduling problems with completion time. The objective functions are the total makespan and the sum of tardiness and earliness cost of blocks. At the same time, jobs are moved among machines through a synchronous transportation system with synchronized processing cycles. In each cycle, the existing jobs begin simultaneously, each on one of the machines, and after completion, wait until the last job is completed. Subsequently, all the jobs are moved concurrently to the next machine. Four algorithms, including non-dominated sorting genetic algorithm (NSGA II), multi-objective simulated annealing (MOSA), multi-objective particle swarm optimization (MOPSO), and multi-objective hybrid vibration-damping optimization (MOHVDO), are used to find a near-optimal solution for this NP-hard problem. In particular, the proposed hybrid VDO algorithm is based on the imperialist competitive algorithm (ICA) and the integration of a neighborhood creation technique. MOHVDO and MOSA show the best performance among the other algorithms regarding objective functions and CPU Time, respectively. Thus, the results from running small-scale and medium-scale problems in MOHVDO and MOSA are compared with the solutions obtained from the epsilon-constraint method. In particular, the error percentage of MOHVDO’s objective functions is less than 2% compared to the epsilon-constraint method for all solved problems. Besides the specific results obtained in terms of performance and, hence, practical applicability, the proposed approach fills a considerable gap in the literature. Indeed, even though variants of the aforementioned meta-heuristic algorithms have been largely introduced in multi-objective environments, a simultaneous implementation of these algorithms as well as a compared study of their performance when solving flow shop scheduling problems has been so far overlooked
A GPT-Based Approach for Scientometric Analysis: Exploring the Landscape of Artificial Intelligence Research
This study presents a comprehensive approach that addresses the challenges of
scientometric analysis in the rapidly evolving field of Artificial Intelligence
(AI). By combining search terms related to AI with the advanced language
processing capabilities of generative pre-trained transformers (GPT), we
developed a highly accurate method for identifying and analyzing AI-related
articles in the Web of Science (WoS) database. Our multi-step approach included
filtering articles based on WoS citation topics, category, keyword screening,
and GPT classification. We evaluated the effectiveness of our method through
precision and recall calculations, finding that our combined approach captured
around 94% of AI-related articles in the entire WoS corpus with a precision of
90%. Following this, we analyzed the publication volume trends, revealing a
continuous growth pattern from 2013 to 2022 and an increasing degree of
interdisciplinarity. We conducted citation analysis on the top countries and
institutions and identified common research themes using keyword analysis and
GPT. This study demonstrates the potential of our approach to facilitate
accurate scientometric analysis, by providing insights into the growth,
interdisciplinary nature, and key players in the field.Comment: 29 pages, 10 figures, 5 table
Computational approaches to Explainable Artificial Intelligence: Advances in theory, applications and trends
Deep Learning (DL), a groundbreaking branch of Machine Learning (ML), has emerged as a driving force in both theoretical and applied Artificial Intelligence (AI). DL algorithms, rooted in complex and non-linear artificial neural systems, excel at extracting high-level features from data. DL has demonstrated human-level performance in real-world tasks, including clinical diagnostics, and has unlocked solutions to previously intractable problems in virtual agent design, robotics, genomics, neuroimaging, computer vision, and industrial automation. In this paper, the most relevant advances from the last few years in Artificial Intelligence (AI) and several applications to neuroscience, neuroimaging, computer vision, and robotics are presented, reviewed and discussed. In this way, we summarize the state-of-the-art in AI methods, models and applications within a collection of works presented at the 9
International Conference on the Interplay between Natural and Artificial Computation (IWINAC). The works presented in this paper are excellent examples of new scientific discoveries made in laboratories that have successfully transitioned to real-life applications
Reinforcement Learning-assisted Evolutionary Algorithm: A Survey and Research Opportunities
Evolutionary algorithms (EA), a class of stochastic search methods based on
the principles of natural evolution, have received widespread acclaim for their
exceptional performance in various real-world optimization problems. While
researchers worldwide have proposed a wide variety of EAs, certain limitations
remain, such as slow convergence speed and poor generalization capabilities.
Consequently, numerous scholars actively explore improvements to algorithmic
structures, operators, search patterns, etc., to enhance their optimization
performance. Reinforcement learning (RL) integrated as a component in the EA
framework has demonstrated superior performance in recent years. This paper
presents a comprehensive survey on integrating reinforcement learning into the
evolutionary algorithm, referred to as reinforcement learning-assisted
evolutionary algorithm (RL-EA). We begin with the conceptual outlines of
reinforcement learning and the evolutionary algorithm. We then provide a
taxonomy of RL-EA. Subsequently, we discuss the RL-EA integration method, the
RL-assisted strategy adopted by RL-EA, and its applications according to the
existing literature. The RL-assisted procedure is divided according to the
implemented functions including solution generation, learnable objective
function, algorithm/operator/sub-population selection, parameter adaptation,
and other strategies. Finally, we analyze potential directions for future
research. This survey serves as a rich resource for researchers interested in
RL-EA as it overviews the current state-of-the-art and highlights the
associated challenges. By leveraging this survey, readers can swiftly gain
insights into RL-EA to develop efficient algorithms, thereby fostering further
advancements in this emerging field.Comment: 26 pages, 16 figure
Harnessing Evolution in-Materio as an Unconventional Computing Resource
This thesis illustrates the use and development of physical conductive analogue systems for unconventional computing using the Evolution in-Materio (EiM) paradigm. EiM uses an Evolutionary Algorithm to configure and exploit a physical material (or medium) for computation. While EiM processors show promise, fundamental questions and scaling issues remain. Additionally, their development is hindered by slow manufacturing and physical experimentation. This work addressed these issues by implementing simulated models to speed up research efforts, followed by investigations of physically implemented novel in-materio devices.
Initial work leveraged simulated conductive networks as single substrate ‘monolithic’ EiM processors, performing classification by formulating the system as an optimisation problem, solved using Differential Evolution. Different material properties and algorithm parameters were isolated and investigated; which explained the capabilities of configurable parameters and showed ideal nanomaterial choice depended upon problem complexity. Subsequently, drawing from concepts in the wider Machine Learning field, several enhancements to monolithic EiM processors were proposed and investigated. These ensured more efficient use of training data, better classification decision boundary placement, an independently optimised readout layer, and a smoother search space. Finally, scalability and performance issues were addressed by constructing in-Materio Neural Networks (iM-NNs), where several EiM processors were stacked in parallel and operated as physical realisations of Hidden Layer neurons. Greater flexibility in system implementation was achieved by re-using a single physical substrate recursively as several virtual neurons, but this sacrificed faster parallelised execution. These novel iM-NNs were first implemented using Simulated in-Materio neurons, and trained for classification as Extreme Learning Machines, which were found to outperform artificial networks of a similar size. Physical iM-NN were then implemented using a Raspberry Pi, custom Hardware Interface and Lambda Diode based Physical in-Materio neurons, which were trained successfully with neuroevolution. A more complex AutoEncoder structure was then proposed and implemented physically to perform dimensionality reduction on a handwritten digits dataset, outperforming both Principal Component Analysis and artificial AutoEncoders.
This work presents an approach to exploit systems with interesting physical dynamics, and leverage them as a computational resource. Such systems could become low power, high speed, unconventional computing assets in the future
Machine Learning Approaches for Heart Disease Detection: A Comprehensive Review
This paper presents a comprehensive review of the application of machine learning algorithms in the early detection of heart disease. Heart disease remains a leading global health concern, necessitating efficient and accurate diagnostic methods. Machine learning has emerged as a promising approach, offering the potential to enhance diagnostic accuracy and reduce the time required for assessments. This review begins by elucidating the fundamentals of machine learning and provides concise explanations of the most prevalent algorithms employed in heart disease detection. It subsequently examines noteworthy research efforts that have harnessed machine learning techniques for heart disease diagnosis. A detailed tabular comparison of these studies is also presented, highlighting the strengths and weaknesses of various algorithms and methodologies. This survey underscores the significant strides made in leveraging machine learning for early heart disease detection and emphasizes the ongoing need for further research to enhance its clinical applicability and efficacy
Oil and Gas flow Anomaly Detection on offshore naturally flowing wells using Deep Neural Networks
Dissertation presented as the partial requirement for obtaining a Master's degree in Data Science and Advanced Analytics, specialization in Data ScienceThe Oil and Gas industry, as never before, faces multiple challenges. It is being impugned for being
dirty, a pollutant, and hence the more demand for green alternatives. Nevertheless, the world still has
to rely heavily on hydrocarbons, since it is the most traditional and stable source of energy, as opposed
to extensively promoted hydro, solar or wind power. Major operators are challenged to produce the
oil more efficiently, to counteract the newly arising energy sources, with less of a climate footprint,
more scrutinized expenditure, thus facing high skepticism regarding its future. It has to become
greener, and hence to act in a manner not required previously.
While most of the tools used by the Hydrocarbon E&P industry is expensive and has been used for
many years, it is paramount for the industry’s survival and prosperity to apply predictive maintenance
technologies, that would foresee potential failures, making production safer, lowering downtime,
increasing productivity and diminishing maintenance costs. Many efforts were applied in order to
define the most accurate and effective predictive methods, however data scarcity affects the speed
and capacity for further experimentations. Whilst it would be highly beneficial for the industry to invest
in Artificial Intelligence, this research aims at exploring, in depth, the subject of Anomaly Detection,
using the open public data from Petrobras, that was developed by experts.
For this research the Deep Learning Neural Networks, such as Recurrent Neural Networks with LSTM
and GRU backbones, were implemented for multi-class classification of undesirable events on naturally
flowing wells. Further, several hyperparameter optimization tools were explored, mainly focusing on
Genetic Algorithms as being the most advanced methods for such kind of tasks.
The research concluded with the best performing algorithm with 2 stacked GRU and the following
vector of hyperparameters weights: [1, 47, 40, 14], which stand for timestep 1, number of hidden units
47, number of epochs 40 and batch size 14, producing F1 equal to 0.97%.
As the world faces many issues, one of which is the detrimental effect of heavy industries to the
environment and as result adverse global climate change, this project is an attempt to contribute to
the field of applying Artificial Intelligence in the Oil and Gas industry, with the intention to make it
more efficient, transparent and sustainable
- …