FReLU: Flexible Rectified Linear Units for Improving Convolutional Neural Networks

Rectified linear unit (ReLU) is a widely used activation function for deep convolutional neural networks. However, because of the zero-hard rectification, ReLU networks miss the benefits from negative values. In this paper, we propose a novel activation function called \emph{flexible rectified linear unit (FReLU)} to further explore the effects of negative values. By redesigning the rectified point of ReLU as a learnable parameter, FReLU expands the states of the activation output. When the network is successfully trained, FReLU tends to converge to a negative value, which improves the expressiveness and thus the performance. Furthermore, FReLU is designed to be simple and effective without exponential functions to maintain low cost computation. For being able to easily used in various network architectures, FReLU does not rely on strict assumptions by self-adaption. We evaluate FReLU on three standard image classification datasets, including CIFAR-10, CIFAR-100, and ImageNet. Experimental results show that the proposed method achieves fast convergence and higher performances on both plain and residual networks

Energy Efficient Coordinated Beamforming for Multi-cell MISO Systems

In this paper, we investigate the optimal energy efficient coordinated beamforming in multi-cell multiple-input single-output (MISO) systems with $K$ multiple-antenna base stations (BS) and $K$ single-antenna mobile stations (MS), where each BS sends information to its own intended MS with cooperatively designed transmit beamforming. We assume single user detection at the MS by treating the interference as noise. By taking into account a realistic power model at the BS, we characterize the Pareto boundary of the achievable energy efficiency (EE) region of the $K$ links, where the EE of each link is defined as the achievable data rate at the MS divided by the total power consumption at the BS. Since the EE of each link is non-cancave (which is a non-concave function over an affine function), characterizing this boundary is difficult. To meet this challenge, we relate this multi-cell MISO system to cognitive radio (CR) MISO channels by applying the concept of interference temperature (IT), and accordingly transform the EE boundary characterization problem into a set of fractional concave programming problems. Then, we apply the fractional concave programming technique to solve these fractional concave problems, and correspondingly give a parametrization for the EE boundary in terms of IT levels. Based on this characterization, we further present a decentralized algorithm to implement the multi-cell coordinated beamforming, which is shown by simulations to achieve the EE Pareto boundary.Comment: 6 pages, 2 figures, to be presented in IEEE GLOBECOM 201

An Energy Efficient Semi-static Power Control and Link Adaptation Scheme in UMTS HSDPA

High speed downlink packet access (HSDPA) has been successfully applied in commercial systems and improves user experience significantly. However, it incurs substantial energy consumption. In this paper, we address this issue by proposing a novel energy efficient semi-static power control and link adaptation scheme in HSDPA. Through estimating the EE under different modulation and coding schemes (MCSs) and corresponding transmit power, the proposed scheme can determine the most energy efficient MCS level and transmit power at the Node B. And then the Node B configure the optimal MCS level and transmit power. In order to decrease the signaling overhead caused by the configuration, a dual trigger mechanism is employed. After that, we extend the proposed scheme to the multiple input multiple output (MIMO) scenarios. Simulation results confirm the significant EE improvement of our proposed scheme. Finally, we give a discussion on the potential EE gain and challenge of the energy efficient mode switching between single input multiple output (SIMO) and MIMO configuration in HSDPA.Comment: 9 pages, 11 figures, accepted in EURASIP Journal on Wireless Communications and Networking, special issue on Green Radi

How to Fine-Tune BERT for Text Classification?

Language model pre-training has proven to be useful in learning universal language representations. As a state-of-the-art language model pre-training model, BERT (Bidirectional Encoder Representations from Transformers) has achieved amazing results in many language understanding tasks. In this paper, we conduct exhaustive experiments to investigate different fine-tuning methods of BERT on text classification task and provide a general solution for BERT fine-tuning. Finally, the proposed solution obtains new state-of-the-art results on eight widely-studied text classification datasets