Reinforcement Learning for P2P Backup Applications

A five year study of file-system metadata shows that the number of files increases by 200% and only a select few file-types contribute for over 35% of the files that exist on a file-system. It is difficult to point out a permanent selection of files that a user really cares about. This project uses reinforcement learning (RL) to exploit the correlation between file-types, their usage patterns, multiple revisions etc., to extract out a selection of files which are “important” for an individual user. In this project, we also integrate this file-selection approach with an open-source P2P backup application called CommunityBackup. With this approach, such a backup application can dynamically find its sources every time a user relocates the sources. The survey also points out that most file-systems are only half-full on average, independent of the user job category. A P2P backup application allows peers to share this average half-empty file-system to maintain redundancy over a backup network. This project collects features that CommunityBackup can utilize for its peer selection using Q-learning algorithm to find out the geographically sparse, safe and consistent backup peers over a high-latency network. Another model presented in this project shows the use of an RL approach to improve the data-transfer throughput by adaptively raising the concurrency index to get around the ISP bottlenecks during urgent backup and sync run-times