On Energy Efficient Hierarchical Cross-Layer Design: Joint Power Control and Routing for Ad Hoc Networks

In this paper, a hierarchical cross-layer design approach is proposed to increase energy efficiency in ad hoc networks through joint adaptation of nodes' transmitting powers and route selection. The design maintains the advantages of the classic OSI model, while accounting for the cross-coupling between layers, through information sharing. The proposed joint power control and routing algorithm is shown to increase significantly the overall energy efficiency of the network, at the expense of a moderate increase in complexity. Performance enhancement of the joint design using multiuser detection is also investigated, and it is shown that the use of multiuser detection can increase the capacity of the ad hoc network significantly for a given level of energy consumption.Comment: To appear in the EURASIP Journal on Wireless Communications and Networking, Special Issue on Wireless Mobile Ad Hoc Network

Inherent Weight Normalization in Stochastic Neural Networks

Multiplicative stochasticity such as Dropout improves the robustness and generalizability of deep neural networks. Here, we further demonstrate that always-on multiplicative stochasticity combined with simple threshold neurons are sufficient operations for deep neural networks. We call such models Neural Sampling Machines (NSM). We find that the probability of activation of the NSM exhibits a self-normalizing property that mirrors Weight Normalization, a previously studied mechanism that fulfills many of the features of Batch Normalization in an online fashion. The normalization of activities during training speeds up convergence by preventing internal covariate shift caused by changes in the input distribution. The always-on stochasticity of the NSM confers the following advantages: the network is identical in the inference and learning phases, making the NSM suitable for online learning, it can exploit stochasticity inherent to a physical substrate such as analog non-volatile memories for in-memory computing, and it is suitable for Monte Carlo sampling, while requiring almost exclusively addition and comparison operations. We demonstrate NSMs on standard classification benchmarks (MNIST and CIFAR) and event-based classification benchmarks (N-MNIST and DVS Gestures). Our results show that NSMs perform comparably or better than conventional artificial neural networks with the same architecture