Deep Reinforcement Learning for Resource Allocation in V2V Communications

In this article, we develop a decentralized resource allocation mechanism for vehicle-to-vehicle (V2V) communication systems based on deep reinforcement learning. Each V2V link is considered as an agent, making its own decisions to find optimal sub-band and power level for transmission. Since the proposed method is decentralized, the global information is not required for each agent to make its decisions, hence the transmission overhead is small. From the simulation results, each agent can learn how to satisfy the V2V constraints while minimizing the interference to vehicle-to-infrastructure (V2I) communications

Thirty Years of Machine Learning: The Road to Pareto-Optimal Wireless Networks

Future wireless networks have a substantial potential in terms of supporting a broad range of complex compelling applications both in military and civilian fields, where the users are able to enjoy high-rate, low-latency, low-cost and reliable information services. Achieving this ambitious goal requires new radio techniques for adaptive learning and intelligent decision making because of the complex heterogeneous nature of the network structures and wireless services. Machine learning (ML) algorithms have great success in supporting big data analytics, efficient parameter estimation and interactive decision making. Hence, in this article, we review the thirty-year history of ML by elaborating on supervised learning, unsupervised learning, reinforcement learning and deep learning. Furthermore, we investigate their employment in the compelling applications of wireless networks, including heterogeneous networks (HetNets), cognitive radios (CR), Internet of things (IoT), machine to machine networks (M2M), and so on. This article aims for assisting the readers in clarifying the motivation and methodology of the various ML algorithms, so as to invoke them for hitherto unexplored services as well as scenarios of future wireless networks.Comment: 46 pages, 22 fig

Model Creation and Equivalence Proofs of Cellular Automata and Artificial Neural Networks

Computational methods and mathematical models have invaded arguably every scientific discipline forming its own field of research called computational science. Mathematical models are the theoretical foundation of computational science. Since Newton's time, differential equations in mathematical models have been widely and successfully used to describe the macroscopic or global behaviour of systems. With spatially inhomogeneous, time-varying, local element-specific, and often non-linear interactions, the dynamics of complex systems is in contrast more efficiently described by local rules and thus in an algorithmic and local or microscopic manner. The theory of mathematical modelling taking into account these characteristics of complex systems has to be established still. We recently presented a so-called allagmatic method including a system metamodel to provide a framework for describing, modelling, simulating, and interpreting complex systems. Implementations of cellular automata and artificial neural networks were described and created with that method. Guidance from philosophy were helpful in these first studies focusing on programming and feasibility. A rigorous mathematical formalism, however, is still missing. This would not only more precisely describe and define the system metamodel, it would also further generalise it and with that extend its reach to formal treatment in applied mathematics and theoretical aspects of computational science as well as extend its applicability to other mathematical and computational models such as agent-based models. Here, a mathematical definition of the system metamodel is provided. Based on the presented formalism, model creation and equivalence of cellular automata and artificial neural networks are proved. It thus provides a formal approach for studying the creation of mathematical models as well as their structural and operational comparison.Comment: 13 pages, 1 tabl

Controlling a mobile robot with a biological brain

The intelligent controlling mechanism of a typical mobile robot is usually a computer system. Some recent research is ongoing in which biological neurons are being cultured and trained to act as the brain of an interactive real world robot�thereby either completely replacing, or operating in a cooperative fashion with, a computer system. Studying such hybrid systems can provide distinct insights into the operation of biological neural structures, and therefore, such research has immediate medical implications as well as enormous potential in robotics. The main aim of the research is to assess the computational and learning capacity of dissociated cultured neuronal networks. A hybrid system incorporating closed-loop control of a mobile robot by a dissociated culture of neurons has been created. The system is flexible and allows for closed-loop operation, either with hardware robot or its software simulation. The paper provides an overview of the problem area, gives an idea of the breadth of present ongoing research, establises a new system architecture and, as an example, reports on the results of conducted experiments with real-life robots