Real-Time Neural Video Recovery and Enhancement on Mobile Devices

As mobile devices become increasingly popular for video streaming, it's crucial to optimize the streaming experience for these devices. Although deep learning-based video enhancement techniques are gaining attention, most of them cannot support real-time enhancement on mobile devices. Additionally, many of these techniques are focused solely on super-resolution and cannot handle partial or complete loss or corruption of video frames, which is common on the Internet and wireless networks. To overcome these challenges, we present a novel approach in this paper. Our approach consists of (i) a novel video frame recovery scheme, (ii) a new super-resolution algorithm, and (iii) a receiver enhancement-aware video bit rate adaptation algorithm. We have implemented our approach on an iPhone 12, and it can support 30 frames per second (FPS). We have evaluated our approach in various networks such as WiFi, 3G, 4G, and 5G networks. Our evaluation shows that our approach enables real-time enhancement and results in a significant increase in video QoE (Quality of Experience) of 24\% - 82\% in our video streaming system

Neural Video Recovery for Cloud Gaming

Cloud gaming is a multi-billion dollar industry. A client in cloud gaming sends its movement to the game server on the Internet, which renders and transmits the resulting video back. In order to provide a good gaming experience, a latency below 80 ms is required. This means that video rendering, encoding, transmission, decoding, and display have to finish within that time frame, which is especially challenging to achieve due to server overload, network congestion, and losses. In this paper, we propose a new method for recovering lost or corrupted video frames in cloud gaming. Unlike traditional video frame recovery, our approach uses game states to significantly enhance recovery accuracy and utilizes partially decoded frames to recover lost portions. We develop a holistic system that consists of (i) efficiently extracting game states, (ii) modifying H.264 video decoder to generate a mask to indicate which portions of video frames need recovery, and (iii) designing a novel neural network to recover either complete or partial video frames. Our approach is extensively evaluated using iPhone 12 and laptop implementations, and we demonstrate the utility of game states in the game video recovery and the effectiveness of our overall design

Advanced NOMA Assisted Semi-Grant-Free Transmission Schemes for Randomly Distributed Users

Non-orthogonal multiple access (NOMA) assisted semi-grant-free (SGF) transmission has recently received significant research attention due to its outstanding ability of serving grant-free (GF) users with grant-based (GB) users' spectrum, {\color{blue}which can greatly improve the spectrum efficiency and effectively relieve the massive access problem of 5G and beyond networks. In this paper, we investigate the performance of SGF schemes under more practical settings.} Firstly, we study the outage performance of the best user scheduling SGF scheme (BU-SGF) by considering the impacts of Rayleigh fading, path loss, and random user locations. Then, a fair SGF scheme is proposed by applying cumulative distribution function (CDF)-based scheduling (CS-SGF), which can also make full use of multi-user diversity. Moreover, by employing the theories of order statistics and stochastic geometry, we analyze the outage performances of both BU-SGF and CS-SGF schemes. Results show that full diversity orders can be achieved only when the served users' data rate is capped, which severely limit the rate performance of SGF schemes. To further address this issue, we propose a distributed power control strategy to relax such data rate constraint, and derive closed-form expressions of the two schemes' outage performances under this strategy. Finally, simulation results validate the fairness performance of the proposed CS-SGF scheme, the effectiveness of the power control strategy, and the accuracy of the theoretical analyses.Comment: 41 pages, 8 figure

Transferable Attack for Semantic Segmentation

We analysis performance of semantic segmentation models wrt. adversarial attacks, and observe that the adversarial examples generated from a source model fail to attack the target models. i.e The conventional attack methods, such as PGD and FGSM, do not transfer well to target models, making it necessary to study the transferable attacks, especially transferable attacks for semantic segmentation. We find two main factors to achieve transferable attack. Firstly, the attack should come with effective data augmentation and translation-invariant features to deal with unseen models. Secondly, stabilized optimization strategies are needed to find the optimal attack direction. Based on the above observations, we propose an ensemble attack for semantic segmentation to achieve more effective attacks with higher transferability. The source code and experimental results are publicly available via our project page: https://github.com/anucvers/TASS.Comment: Source code is available at: https://github.com/anucvers/TAS

A modified ant colony optimization algorithm for network coding resource minimization

The paper presents a modified ant colony optimization approach for the network coding resource minimization problem. It is featured with several attractive mechanisms specially devised for solving the network coding resource minimization problem: 1) a multi-dimensional pheromone maintenance mechanism is put forward to address the issue of pheromone overlapping; 2) problem-specific heuristic information is employed to enhance the heuristic search (neighboring area search) capability; 3) a tabu-table based path construction method is devised to facilitate the construction of feasible (link-disjoint) paths from the source to each receiver; 4) a local pheromone updating rule is developed to guide ants to construct appropriate promising paths; 5) a solution reconstruction method is presented, with the aim of avoiding prematurity and improving the global search efficiency of proposed algorithm. Due to the way it works, the ant colony optimization can well exploit the global and local information of routing related problems during the solution construction phase. The simulation results on benchmark instances demonstrate that with the five extended mechanisms integrated, our algorithm outperforms a number of existing algorithms with respect to the best solutions obtained and the computational time