Multi-Agent Reinforcement Learning for Connected and Automated Vehicles Control: Recent Advancements and Future Prospects

Connected and automated vehicles (CAVs) have emerged as a potential solution to the future challenges of developing safe, efficient, and eco-friendly transportation systems. However, CAV control presents significant challenges, given the complexity of interconnectivity and coordination required among the vehicles. To address this, multi-agent reinforcement learning (MARL), with its notable advancements in addressing complex problems in autonomous driving, robotics, and human-vehicle interaction, has emerged as a promising tool for enhancing the capabilities of CAVs. However, there is a notable absence of current reviews on the state-of-the-art MARL algorithms in the context of CAVs. Therefore, this paper delivers a comprehensive review of the application of MARL techniques within the field of CAV control. The paper begins by introducing MARL, followed by a detailed explanation of its unique advantages in addressing complex mobility and traffic scenarios that involve multiple agents. It then presents a comprehensive survey of MARL applications on the extent of control dimensions for CAVs, covering critical and typical scenarios such as platooning control, lane-changing, and unsignalized intersections. In addition, the paper provides a comprehensive review of the prominent simulation platforms used to create reliable environments for training in MARL. Lastly, the paper examines the current challenges associated with deploying MARL within CAV control and outlines potential solutions that can effectively overcome these issues. Through this review, the study highlights the tremendous potential of MARL to enhance the performance and collaboration of CAV control in terms of safety, travel efficiency, and economy

The evolution of business analytics : based on case study research

While business analytics is becoming more significant and widely used by companies from increasing industries, for many the concept remains a complex illusion. The field of business analytics is considerably generic and fragmented, leaving managers confused and ultimately inhibited to make valuable decisions. This paper presents an evolutionary depiction of business analytics, using real-world case studies to illustrate a distinct overview that describes where the phenomenon was derived from, where it currently stands, and where it is heading towards. This paper provides eight case studies, representing three different eras: yesterday (1950s to 1990s), today (2000s to 2020s), and tomorrow (2030s to 2050s). Through cross-case analysis we have identified concluding patterns that lay as foundation for the discussion on future development within business analytics. We argue based on our findings that automatization of business processes will most likely continue to increase. AI is expanding in numerous areas, each specializing in a complex task, previously reserved by professionals. However, patterns show that new occupations linked to artificial intelligence will most probably be created. For the training of intelligent systems, data will most likely be requested more than ever. The increasing data will likely cause complications in current data infrastructures, causing the need for stronger networks and systems. The systems will need to process, store, and manage the great amount of various data types in real-time, while maintaining high security. Furthermore, data privacy concerns have become more significant in recent years, although, the case study research indicates that it has not limited corporations access to data. On the contrary, corporations, people, and devices will most likely become even more connected than ever before.nhhma

The potential of automated transport systems in the future of urban mobility

Global warming is endangering the earth as we know it, and CO2 levels are rising to amounts the world has never seen before and cannot handle. Additionally fossil fuels and natural resources are depleting very rapidly. For that reason the 1997 Kyoto protocol up to the 2018 Paris Agreement were signed, aiming towards a more sustainable and environmentally friendly re-source extraction and reduction of the CO2 footprint. Introducing Smart, emission free cities, is one of the solutions to this challenge. To develop such a city, all day to day activities must be considered. This includes but is not limited to: communication, electricity and mobility. These factors are in the bigger picture connected and will be introduced through analyzing future mobility in this thesis. Mobility plays a big role in maintaining the stability of the grid, expected to be supporting the unpredictable renewable energy sources through charging and discharging when needed. This will play a big factor for Demand Side Management and enable a better, higher quality of life. Future vehicles are expected to be electrical and autonomous, requiring no human interaction during the driving. They should be a part of a bigger system, allowing all the city’s residents to be able to get from A to B safely and efficiently. Allowing on the city to become more self sufficient and sustainable and on the other hand easier connection to neighbouring cities. The future resident will not have to worry about getting from a place to the other, even if its in a rural area. The beauty of having such a system enables worry free mobility for the citizen and a structured design for the governments. The potential of these autonomous vehicles is analysed on a techno-economical base and their implementation is simulated on a virtual residential quarter in Berlin, Germany. The technical simulation works intelligently, creating iterations to provide the best possible way and the amount of vehicles needed. Whereas the economical analysis shows the potential of the autonomous ve-hicle with regards to value and money. The potential of autonomous vehicles as means of mobility in Smart Cities will be conveyed in this master thesis, clearly showing the benefits if such a system were adapted

Modern computing: vision and challenges

Over the past six decades, the computing systems field has experienced significant transformations, profoundly impacting society with transformational developments, such as the Internet and the commodification of computing. Underpinned by technological advancements, computer systems, far from being static, have been continuously evolving and adapting to cover multifaceted societal niches. This has led to new paradigms such as cloud, fog, edge computing, and the Internet of Things (IoT), which offer fresh economic and creative opportunities. Nevertheless, this rapid change poses complex research challenges, especially in maximizing potential and enhancing functionality. As such, to maintain an economical level of performance that meets ever-tighter requirements, one must understand the drivers of new model emergence and expansion, and how contemporary challenges differ from past ones. To that end, this article investigates and assesses the factors influencing the evolution of computing systems, covering established systems and architectures as well as newer developments, such as serverless computing, quantum computing, and on-device AI on edge devices. Trends emerge when one traces technological trajectory, which includes the rapid obsolescence of frameworks due to business and technical constraints, a move towards specialized systems and models, and varying approaches to centralized and decentralized control. This comprehensive review of modern computing systems looks ahead to the future of research in the field, highlighting key challenges and emerging trends, and underscoring their importance in cost-effectively driving technological progress

Modern computing: Vision and challenges

Over the past six decades, the computing systems field has experienced significant transformations, profoundly impacting society with transformational developments, such as the Internet and the commodification of computing. Underpinned by technological advancements, computer systems, far from being static, have been continuously evolving and adapting to cover multifaceted societal niches. This has led to new paradigms such as cloud, fog, edge computing, and the Internet of Things (IoT), which offer fresh economic and creative opportunities. Nevertheless, this rapid change poses complex research challenges, especially in maximizing potential and enhancing functionality. As such, to maintain an economical level of performance that meets ever-tighter requirements, one must understand the drivers of new model emergence and expansion, and how contemporary challenges differ from past ones. To that end, this article investigates and assesses the factors influencing the evolution of computing systems, covering established systems and architectures as well as newer developments, such as serverless computing, quantum computing, and on-device AI on edge devices. Trends emerge when one traces technological trajectory, which includes the rapid obsolescence of frameworks due to business and technical constraints, a move towards specialized systems and models, and varying approaches to centralized and decentralized control. This comprehensive review of modern computing systems looks ahead to the future of research in the field, highlighting key challenges and emerging trends, and underscoring their importance in cost-effectively driving technological progress

Federated Learning for Iot/Edge/Fog Computing Systems

With the help of a new architecture called Edge/Fog (E/F) computing, cloud computing services can now be extended nearer to data generator devices. E/F computing in combination with Deep Learning (DL) is a promisedtechnique that is vastly applied in numerous fields. To train their models, data producers in conventional DL architectures with E/F computing enable them to repeatedly transmit and communicate data with third-party servers, like Edge/Fog or cloud servers. Due to the extensive bandwidth needs, legal issues, and privacy risks, this architecture is frequently impractical. Through a centralized server, the models can be co-trained by FL through distributed clients, including cars, hospitals, and mobile phones, while preserving data localization. As it facilitates group learning and model optimization, FL can therefore be seen as a motivating element in the E/F computing paradigm. Although FL applications in E/F computing environments have been considered in previous studies, FL execution and hurdles in the E/F computing framework have not been thoroughly covered. In order to identify advanced solutions, this chapter will provide a review of the application of FL in E/F computing systems. We think that by doing this chapter, researchers will learn more about how E/F computing and FL enable related concepts and technologies. Some case studies about the implementation of federated learning in E/F computing are being investigated. The open issues and future research directions are introduced.Comment: 21 pages, 4 figures, Book chapte

Evolving to Digital and Programmable Value Based Economy: General Prospect and Specific Applications over Sustainability

[eng] In the fields of economics, business and management, how could Digital Transformation (DT) advance value creation and reliably encourage value capture, exchange and distribution? This thesis aim to fill that gap with a novel framework to support policy-makers, countries, cities and businesses address the potential value that can be generated and captured by digitalization combining DT and Internet of Value theoretical perspectives and practical applications of them over concrete issues such as sustainability in cities, as an example. For this, it is proposed to make new contributions related to DT and Internet of Value in two main aspects: to explore DT countries’ mindsets when it relates to their value progress through Digital Ecosystems and to advance with the potential digital value applications through Programmable Economy advantages when it focus on concrete aspect such as sustainability in cities. Both perspectives, although it will be applied on different dimensions and on different purposes, have in common that they are focus on digital and programable value based economy and management and want to explore the best way to maximize and capture the DT potential in terms of value for organizations and society. Thus, first, it will be analysed the importance of knowing clearly the digital ecosystem in which the agents are operating in order to reinforce the value creation by promoting the inclusivity and connectivity of the endpoints involved in it. Secondly, it will be analysed how the digital value can be captured, exchanged and redistributed in a complex issues such as sustainability by deploying concrete digital applications that include human reinforcement aspects to, finally, closing the circle combining both perspectives in a single framework. To achieve these objectives in this thesis, own models are proposed, inspired by other theoretical models already contrasted, and some proven methodologies are used related to Conditional Probability, Forgotten Effects and Fuzzy Sets. As a main conclusion, Digital Transformation has the potential to generate immense value for economy and society. Although currently the capture of the vast majority of it is not guaranteed and its distribution between agents is no clear, new formulas are being explored supported by the Internet of Value. This thesis defends that if agents want to advance value creation and encourage value capture, they should consider to make their own Digital and Programmable Value Based Economy and Management framework through: - Allowing all functional agents work in a Digital Ecosystem embracing new relationships and ways of collaborating pursuing the same purpose. - Deploying Programmable Economy applications advantages, mixing digital's and analogue's world that can be interlinked and programmed by the blockchain allowing monetization and exploring new human and machine alliances. - Adopting strong and inclusive agents’ commitment in order to exploit the advantages that this smart economy system has from a human centric vision, discovering new forms of value, considering that, although tech can be everywhere, value not

Dynamics in Logistics

This open access book highlights the interdisciplinary aspects of logistics research. Featuring empirical, methodological, and practice-oriented articles, it addresses the modelling, planning, optimization and control of processes. Chiefly focusing on supply chains, logistics networks, production systems, and systems and facilities for material flows, the respective contributions combine research on classical supply chain management, digitalized business processes, production engineering, electrical engineering, computer science and mathematical optimization. To celebrate 25 years of interdisciplinary and collaborative research conducted at the Bremen Research Cluster for Dynamics in Logistics (LogDynamics), in this book hand-picked experts currently or formerly affiliated with the Cluster provide retrospectives, present cutting-edge research, and outline future research directions