Mobility Management in beyond 3G-Environments

Beyond 3G-environments are typically defined as environments that integrate different wireless and fixed access network technologies. In this paper, we address IP based Mobility Management (MM) in beyond 3G-environments with a focus on wireless access networks, motivated by the current trend of WiFi, GPRS, and UMTS networks. The GPRS and UMTS networks provide countrywide network access, while the WiFi networks provide network access in local areas such as city centres and airports. As a result, mobile end-users can be always on-line and connected to their preferred network(s), these network preferences are typically stored in a user profile. For example, an end-user who wishes to be connected with highest bandwidth could be connected to a WiFi network when available and fall back to GPRS when moving outside the hotspot area.\ud In this paper, we consider a combination of MM for legacy services (like web browsing, telnet, etc.) using Mobile IP and multimedia services using SIP. We assume that the end-user makes use of multi-interface terminals with the capability of selecting one or more types of access networks\ud based on preferences. For multimedia sessions, like VoIP or streaming video, we distinguish between changes in network access when the end-user is in a session or not in a session. If the end-user is not in a session, he or she needs to be able to start new sessions and receive invitations for new sessions. If the end-user is in a session, the session needs to be handed over to the new access network as seamless as possible from the perspective of the end-user. We propose an integrated but flexible solution to these problems that facilitates MM with a customizable transparency to applications and end-users

Exploring Restart Distributions

We consider the generic approach of using an experience memory to help exploration by adapting a restart distribution. That is, given the capacity to reset the state with those corresponding to the agent's past observations, we help exploration by promoting faster state-space coverage via restarting the agent from a more diverse set of initial states, as well as allowing it to restart in states associated with significant past experiences. This approach is compatible with both on-policy and off-policy methods. However, a caveat is that altering the distribution of initial states could change the optimal policies when searching within a restricted class of policies. To reduce this unsought learning bias, we evaluate our approach in deep reinforcement learning which benefits from the high representational capacity of deep neural networks. We instantiate three variants of our approach, each inspired by an idea in the context of experience replay. Using these variants, we show that performance gains can be achieved, especially in hard exploration problems.Comment: RLDM 201

Multicast Mobility in Mobile IP Version 6 (MIPv6) : Problem Statement and Brief Survey

Publisher PD

Multi-Agent Deep Reinforcement Learning with Human Strategies

Deep learning has enabled traditional reinforcement learning methods to deal with high-dimensional problems. However, one of the disadvantages of deep reinforcement learning methods is the limited exploration capacity of learning agents. In this paper, we introduce an approach that integrates human strategies to increase the exploration capacity of multiple deep reinforcement learning agents. We also report the development of our own multi-agent environment called Multiple Tank Defence to simulate the proposed approach. The results show the significant performance improvement of multiple agents that have learned cooperatively with human strategies. This implies that there is a critical need for human intellect teamed with machines to solve complex problems. In addition, the success of this simulation indicates that our multi-agent environment can be used as a testbed platform to develop and validate other multi-agent control algorithms.Comment: 2019 IEEE International Conference on Industrial Technology (ICIT), Melbourne, Australi

Robotic Wireless Sensor Networks

In this chapter, we present a literature survey of an emerging, cutting-edge, and multi-disciplinary field of research at the intersection of Robotics and Wireless Sensor Networks (WSN) which we refer to as Robotic Wireless Sensor Networks (RWSN). We define a RWSN as an autonomous networked multi-robot system that aims to achieve certain sensing goals while meeting and maintaining certain communication performance requirements, through cooperative control, learning and adaptation. While both of the component areas, i.e., Robotics and WSN, are very well-known and well-explored, there exist a whole set of new opportunities and research directions at the intersection of these two fields which are relatively or even completely unexplored. One such example would be the use of a set of robotic routers to set up a temporary communication path between a sender and a receiver that uses the controlled mobility to the advantage of packet routing. We find that there exist only a limited number of articles to be directly categorized as RWSN related works whereas there exist a range of articles in the robotics and the WSN literature that are also relevant to this new field of research. To connect the dots, we first identify the core problems and research trends related to RWSN such as connectivity, localization, routing, and robust flow of information. Next, we classify the existing research on RWSN as well as the relevant state-of-the-arts from robotics and WSN community according to the problems and trends identified in the first step. Lastly, we analyze what is missing in the existing literature, and identify topics that require more research attention in the future