7,747 research outputs found
Reinforcement learning in large state action spaces
Reinforcement learning (RL) is a promising framework for training intelligent agents which learn to optimize long term utility by directly interacting with the environment. Creating RL methods which scale to large state-action spaces is a critical problem towards ensuring real world deployment of RL systems. However, several challenges limit the applicability of RL to large scale settings. These include difficulties with exploration, low sample efficiency, computational intractability, task constraints like decentralization and lack of guarantees about important properties like performance, generalization and robustness in potentially unseen scenarios.
This thesis is motivated towards bridging the aforementioned gap. We propose several principled algorithms and frameworks for studying and addressing the above challenges RL. The proposed methods cover a wide range of RL settings (single and multi-agent systems (MAS) with all the variations in the latter, prediction and control, model-based and model-free methods, value-based and policy-based methods). In this work we propose the first results on several different problems: e.g. tensorization of the Bellman equation which allows exponential sample efficiency gains (Chapter 4), provable suboptimality arising from structural constraints in MAS(Chapter 3), combinatorial generalization results in cooperative MAS(Chapter 5), generalization results on observation shifts(Chapter 7), learning deterministic policies in a probabilistic RL framework(Chapter 6). Our algorithms exhibit provably enhanced performance and sample efficiency along with better scalability. Additionally, we also shed light on generalization aspects of the agents under different frameworks. These properties have been been driven by the use of several advanced tools (e.g. statistical machine learning, state abstraction, variational inference, tensor theory).
In summary, the contributions in this thesis significantly advance progress towards making RL agents ready for large scale, real world applications
The determinants of value addition: a crtitical analysis of global software engineering industry in Sri Lanka
It was evident through the literature that the perceived value delivery of the global software
engineering industry is low due to various facts. Therefore, this research concerns global
software product companies in Sri Lanka to explore the software engineering methods and
practices in increasing the value addition. The overall aim of the study is to identify the key
determinants for value addition in the global software engineering industry and critically
evaluate the impact of them for the software product companies to help maximise the value
addition to ultimately assure the sustainability of the industry.
An exploratory research approach was used initially since findings would emerge while the
study unfolds. Mixed method was employed as the literature itself was inadequate to
investigate the problem effectively to formulate the research framework. Twenty-three face-to-face online interviews were conducted with the subject matter experts covering all the
disciplines from the targeted organisations which was combined with the literature findings as
well as the outcomes of the market research outcomes conducted by both government and nongovernment institutes. Data from the interviews were analysed using NVivo 12. The findings
of the existing literature were verified through the exploratory study and the outcomes were
used to formulate the questionnaire for the public survey. 371 responses were considered after
cleansing the total responses received for the data analysis through SPSS 21 with alpha level
0.05. Internal consistency test was done before the descriptive analysis. After assuring the
reliability of the dataset, the correlation test, multiple regression test and analysis of variance
(ANOVA) test were carried out to fulfil the requirements of meeting the research objectives.
Five determinants for value addition were identified along with the key themes for each area.
They are staffing, delivery process, use of tools, governance, and technology infrastructure.
The cross-functional and self-organised teams built around the value streams, employing a
properly interconnected software delivery process with the right governance in the delivery
pipelines, selection of tools and providing the right infrastructure increases the value delivery.
Moreover, the constraints for value addition are poor interconnection in the internal processes,
rigid functional hierarchies, inaccurate selections and uses of tools, inflexible team
arrangements and inadequate focus for the technology infrastructure. The findings add to the
existing body of knowledge on increasing the value addition by employing effective processes,
practices and tools and the impacts of inaccurate applications the same in the global software
engineering industry
Recommended from our members
Machine Learning for Gravitational-Wave Astronomy: Methods and Applications for High-Dimensional Laser Interferometry Data
Gravitational-wave astronomy is an emerging field in observational astrophysics concerned with the study of gravitational signals proposed to exist nearly a century ago by Albert Einstein but only recently confirmed to exist. Such signals were theorized to result from astronomical events such as the collisions of black holes, but they were long thought to be too faint to measure on Earth. In recent years, the construction of extremely sensitive detectors—including the Laser Interferometer Gravitational-Wave Observatory (LIGO) project—has enabled the first direct detections of these gravitational waves, corroborating the theory of general relativity and heralding a new era of astrophysics research.
As a result of their extraordinary sensitivity, the instruments used to study gravitational waves are also subject to noise that can significantly limit their ability to detect the signals of interest with sufficient confidence. The detectors continuously record more than 200,000 time series of auxiliary data describing the state of a vast array of internal components and sensors, the environmental state in and around the detector, and so on. This data offers significant value for understanding the nearly innumerable potential sources of noise and ultimately reducing or eliminating them, but it is clearly impossible to monitor, let alone understand, so much information manually. The field of machine learning offers a variety of techniques well-suited to problems of this nature.
In this thesis, we develop and present several machine learning–based approaches to automate the process of extracting insights from the vast, complex collection of data recorded by LIGO detectors. We introduce a novel problem formulation for transient noise detection and show for the first time how an efficient and interpretable machine learning method can accurately identify detector noise using all of these auxiliary data channels but without observing the noise itself. We present further work employing more sophisticated neural network–based models, demonstrating how they can reduce error rates by over 60% while also providing LIGO scientists with interpretable insights into the detector’s behavior. We also illustrate the methods’ utility by demonstrating their application to a specific, recurring type of transient noise; we show how we can achieve a classification accuracy of over 97% while also independently corroborating the results of previous manual investigations into the origins of this type of noise.
The methods and results presented in the following chapters are applicable not only to the specific gravitational-wave data considered but also to a broader family of machine learning problems involving prediction from similarly complex, high-dimensional data containing only a few relevant components in a sea of irrelevant information. We hope this work proves useful to astrophysicists and other machine learning practitioners seeking to better understand gravitational waves, extremely complex and precise engineered systems, or any of the innumerable extraordinary phenomena of our civilization and universe
Command and Persuade
Why, when we have been largely socialized into good behavior, are there more laws that govern our behavior than ever before? Levels of violent crime have been in a steady decline for centuries—for millennia, even. Over the past five hundred years, homicide rates have decreased a hundred-fold. We live in a time that is more orderly and peaceful than ever before in human history. Why, then, does fear of crime dominate modern politics? Why, when we have been largely socialized into good behavior, are there more laws that govern our behavior than ever before? In Command and Persuade, Peter Baldwin examines the evolution of the state's role in crime and punishment over three thousand years. Baldwin explains that the involvement of the state in law enforcement and crime prevention is relatively recent. In ancient Greece, those struck by lightning were assumed to have been punished by Zeus. In the Hebrew Bible, God was judge, jury, and prosecutor when Cain killed Abel. As the state's power as lawgiver grew, more laws governed behavior than ever before; the sum total of prohibited behavior has grown continuously. At the same time, as family, community, and church exerted their influences, we have become better behaved and more law-abiding. Even as the state stands as the socializer of last resort, it also defines through law the terrain on which we are schooled into acceptable behavior. This title is also available in an Open Access edition
COVID-19 Outbreak and Beyond
The COVID-19 pandemic drastically changed our lifestyle when, on 30 January 2020, the World Health Organization declared the coronavirus disease outbreak a public health emergency of international concern. Since then, many governments have introduced unprecedented containment measures, hoping to slow the spread of the virus. International research suggests that both the pandemic and the related protective measures, such as lockdown, curfews, and social distancing, are having a profound impact on the mental health of the population. Among the most commonly observed psychological effects, there are high levels of stress, anxiety, depression, and post-traumatic symptoms, along with boredom and frustration. At the same time, the behavioral response of the population is of paramount importance to successfully contain the outbreak, creating a vicious circle in which the psychological distress impacts the willingness to comply with the protective measures, which, in turn, if prolonged, could exacerbate the population’s distress. This book includes: i) original studies on the worldwide psychological and behavioral impact of COVID-19 on targeted individuals (e.g., parents, social workers, patients affected by physical and mental disorders); ii) studies exploring the effect of COVID-19 using advanced statistical and methodological techniques (e.g., machine learning technologies); iii) research on practical applications that could help identify persons at risk, mitigate the negative effects of this situation, and offer insights to policymakers to manage the pandemic are also highly welcomed
COVID-19: Current Challenges and Future Perspectives
On March 11, 2020, the World Health Organization declared COVID-19 a global pandemic and the disease now affects nearly every country and region. Caused by SARS-CoV-2, COVID-19 presents significant challenges to health systems and public health in both hemispheres as well as to the economies of each country. The morbidity and mortality due to infections caused by SARS-CoV-2 have been significant despite the short duration since its discovery and initially overwhelmed many hospitals and clinics. It influences everyone, and countermeasures have been dramatic in their impact on employment, social systems, and mental health. This Special Issue provides an avenue for authors from various disciplines to provide feedback on our responses and preparedness to COVID-19 globally as well as to disseminate critical information about the SARS-CoV-2 virus and the associated COVID-19 pandemic. It consists of 22 peer-reviewed papers that cover worldwide perspectives encompasses the following: Original articles about COVID-19 (including epidemiology, modelling, clinical data, treatment, prevention, countermeasures, impacts on tropical regions, response, and preparedness);Original articles about SARS-CoV-2 (microbiology, virology, transmission, pathology, and vaccinology);Perspectives about COVID-19 and SARS-CoV-2 (comparisons with past coronavirus outbreaks, impactful local initiatives, novel responses, and commentaries);Reviews on COVID-19 (based on systematic and narrative reviews);and Innovations (vaccine development, drug trials, and original countermeasures)
Advanced Modeling, Control, and Optimization Methods in Power Hybrid Systems - 2021
The climate changes that are becoming visible today are a challenge for the global research community. In this context, renewable energy sources, fuel cell systems and other energy generating sources must be optimally combined and connected to the grid system using advanced energy transaction methods. As this reprint presents the latest solutions in the implementation of fuel cell and renewable energy in mobile and stationary applications such as hybrid and microgrid power systems based on the Energy Internet, blockchain technology and smart contracts, we hope that they will be of interest to readers working in the related fields mentioned above
Green Economy and Sustainable Development
Considering the importance of the challenges for sustainable development, this Book is intended to disseminate the results of cutting-edge research and broadcast the opinions of scientists from around the world, providing technological breakthroughs in green energy and urbanism, recycling and modernization of basic industries, conducting fundamental research on the economic problems of the transition to sustainable development
TOWARDS AN UNDERSTANDING OF EFFORTFUL FUNDRAISING EXPERIENCES: USING INTERPRETATIVE PHENOMENOLOGICAL ANALYSIS IN FUNDRAISING RESEARCH
Physical-activity oriented community fundraising has experienced an exponential growth in popularity over the past 15 years. The aim of this study was to explore the value of effortful fundraising experiences, from the point of view of participants, and explore the impact that these experiences have on people’s lives. This study used an IPA approach to interview 23 individuals, recognising the role of participants as proxy (nonprofessional) fundraisers for charitable organisations, and the unique organisation donor dynamic that this creates. It also bought together relevant psychological theory related to physical activity fundraising experiences (through a narrative literature review) and used primary interview data to substantiate these. Effortful fundraising experiences are examined in detail to understand their significance to participants, and how such experiences influence their connection with a charity or cause. This was done with an idiographic focus at first, before examining convergences and divergences across the sample. This study found that effortful fundraising experiences can have a profound positive impact upon community fundraisers in both the short and the long term. Additionally, it found that these experiences can be opportunities for charitable organisations to create lasting meaningful relationships with participants, and foster mutually beneficial lifetime relationships with them. Further research is needed to test specific psychological theory in this context, including self-esteem theory, self determination theory, and the martyrdom effect (among others)
- …