Learning Combinations of Activation Functions

In the last decade, an active area of research has been devoted to design novel activation functions that are able to help deep neural networks to converge, obtaining better performance. The training procedure of these architectures usually involves optimization of the weights of their layers only, while non-linearities are generally pre-specified and their (possible) parameters are usually considered as hyper-parameters to be tuned manually. In this paper, we introduce two approaches to automatically learn different combinations of base activation functions (such as the identity function, ReLU, and tanh) during the training phase. We present a thorough comparison of our novel approaches with well-known architectures (such as LeNet-5, AlexNet, and ResNet-56) on three standard datasets (Fashion-MNIST, CIFAR-10, and ILSVRC-2012), showing substantial improvements in the overall performance, such as an increase in the top-1 accuracy for AlexNet on ILSVRC-2012 of 3.01 percentage points.Comment: 6 pages, 3 figures. Published as a conference paper at ICPR 2018. Code: https://bitbucket.org/francux/learning_combinations_of_activation_function

A Generalized Bayesian Approach for Localizing Static Natural Obstacles on Unpaved Roads

This paper presents an approach that implements sensor fusion and recursive Bayesian estimation (RBE) to improve a vehicle\u27s ability to perform obstacle detection and localization in unpaved road environments. The proposed approach utilizes RADAR, LiDAR and stereovision fully for sensor fusion to detect and localize static natural obstacles. Each sensor is characterized by a probabilistic sensor model which quantifies level of confidence (LOC) and probability of detection (POD) associatively. Deploying these sensor models enables the fusion of heterogeneous sensors without extensive formulations and with the incorporation of each sensor\u27s strengths. An Extended Kalman filter (EKF) is formulated and implemented for robust and computationally efficient RBE of obstacles\u27 locations while a sensor-equipped vehicle moves and observes them. Results with a test vehicle show the successful detection and localization of a static natural object on an unpaved road has demonstrated the effectiveness of the proposed approach

Detecting Road Obstacles by Erasing Them

Vehicles can encounter a myriad of obstacles on the road, and it is not feasible to record them all beforehand to train a detector. Our method selects image patches and inpaints them with the surrounding road texture, which tends to remove obstacles from those patches. It them uses a network trained to recognize discrepancies between the original patch and the inpainted one, which signals an erased obstacle. We also contribute a new dataset for monocular road obstacle detection, and show that our approach outperforms the state-of-the-art methods on both our new dataset and the standard Fishyscapes Lost & Found benchmark

Multi-Label Zero-Shot Human Action Recognition via Joint Latent Ranking Embedding

Human action recognition refers to automatic recognizing human actions from a video clip. In reality, there often exist multiple human actions in a video stream. Such a video stream is often weakly-annotated with a set of relevant human action labels at a global level rather than assigning each label to a specific video episode corresponding to a single action, which leads to a multi-label learning problem. Furthermore, there are many meaningful human actions in reality but it would be extremely difficult to collect/annotate video clips regarding all of various human actions, which leads to a zero-shot learning scenario. To the best of our knowledge, there is no work that has addressed all the above issues together in human action recognition. In this paper, we formulate a real-world human action recognition task as a multi-label zero-shot learning problem and propose a framework to tackle this problem in a holistic way. Our framework holistically tackles the issue of unknown temporal boundaries between different actions for multi-label learning and exploits the side information regarding the semantic relationship between different human actions for knowledge transfer. Consequently, our framework leads to a joint latent ranking embedding for multi-label zero-shot human action recognition. A novel neural architecture of two component models and an alternate learning algorithm are proposed to carry out the joint latent ranking embedding learning. Thus, multi-label zero-shot recognition is done by measuring relatedness scores of action labels to a test video clip in the joint latent visual and semantic embedding spaces. We evaluate our framework with different settings, including a novel data split scheme designed especially for evaluating multi-label zero-shot learning, on two datasets: Breakfast and Charades. The experimental results demonstrate the effectiveness of our framework.Comment: 27 pages, 10 figures and 7 tables. Technical report submitted to a journal. More experimental results/references were added and typos were correcte

Collaborative Group Learning

Collaborative learning has successfully applied knowledge transfer to guide a pool of small student networks towards robust local minima. However, previous approaches typically struggle with drastically aggravated student homogenization when the number of students rises. In this paper, we propose Collaborative Group Learning, an efficient framework that aims to diversify the feature representation and conduct an effective regularization. Intuitively, similar to the human group study mechanism, we induce students to learn and exchange different parts of course knowledge as collaborative groups. First, each student is established by randomly routing on a modular neural network, which facilitates flexible knowledge communication between students due to random levels of representation sharing and branching. Second, to resist the student homogenization, students first compose diverse feature sets by exploiting the inductive bias from sub-sets of training data, and then aggregate and distill different complementary knowledge by imitating a random sub-group of students at each time step. Overall, the above mechanisms are beneficial for maximizing the student population to further improve the model generalization without sacrificing computational efficiency. Empirical evaluations on both image and text tasks indicate that our method significantly outperforms various state-of-the-art collaborative approaches whilst enhancing computational efficiency.Comment: Accepted by AAAI 2021; Camera ready versio

A reinforcement learning-based link quality estimation strategy for RPL and its impact on topology management

Over the last few years, standardisation efforts are consolidating the role of the Routing Protocol for Low-Power and Lossy Networks (RPL) as the standard routing protocol for IPv6-based Wireless Sensor Networks (WSNs). Although many core functionalities are well defined, others are left implementation dependent. Among them, the definition of an efficient link-quality estimation (LQE) strategy is of paramount importance, as it influences significantly both the quality of the selected network routes and nodesâ\u80\u99 energy consumption. In this paper, we present RL-Probe, a novel strategy for link quality monitoring in RPL, which accurately measures link quality with minimal overhead and energy waste. To achieve this goal, RL-Probe leverages both synchronous and asynchronous monitoring schemes to maintain up-to-date information on link quality and to promptly react to sudden topology changes, e.g. due to mobility. Our solution relies on a reinforcement learning model to drive the monitoring procedures in order to minimise the overhead caused by active probing operations. The performance of the proposed solution is assessed by means of simulations and real experiments. Results demonstrated that RL-Probe helps in effectively improving packet loss rates, allowing nodes to promptly react to link quality variations as well as to link failures due to node mobility