Deep Learning with S-shaped Rectified Linear Activation Units

Rectified linear activation units are important components for state-of-the-art deep convolutional networks. In this paper, we propose a novel S-shaped rectified linear activation unit (SReLU) to learn both convex and non-convex functions, imitating the multiple function forms given by the two fundamental laws, namely the Webner-Fechner law and the Stevens law, in psychophysics and neural sciences. Specifically, SReLU consists of three piecewise linear functions, which are formulated by four learnable parameters. The SReLU is learned jointly with the training of the whole deep network through back propagation. During the training phase, to initialize SReLU in different layers, we propose a "freezing" method to degenerate SReLU into a predefined leaky rectified linear unit in the initial several training epochs and then adaptively learn the good initial values. SReLU can be universally used in the existing deep networks with negligible additional parameters and computation cost. Experiments with two popular CNN architectures, Network in Network and GoogLeNet on scale-various benchmarks including CIFAR10, CIFAR100, MNIST and ImageNet demonstrate that SReLU achieves remarkable improvement compared to other activation functions.Comment: Accepted by AAAI-1

DEVELOPMENT AND MECHANISM INVESTIGATION OF NOVEL THIN FILM NANOCOMPOSITE NANOFILTRATION MEMBRANES

Thin film composite (TFC) membrane based nanofiltration (NF) is an emerging technology that has shown great potential in water purification, waste treatment, and organic solvent separations. Great performance has already been achieved through a variety of modifications for different applications; however, a lack of understanding of the underlying water transport mechanism and multifunctional potential of TFC membranes present significant challenges. One clear example is thin film nanocomposite (TFN) membranes, which have shown enhanced water flux as well as other capabilities, such as antifouling, antibacterial, and anti-compaction properties. However, the underlying mechanism for these excellent properties exhibited by the TFN membranes remains largely unknown. In addition, these properties are mainly improved individually in previous studies, but in practical application, optimal multi-functional TFN membranes that exhibit all these great properties simultaneously are preferred. Motivated by the abovementioned questions and the need for real applications, the overall objective of this thesis is to use both experimental and modeling investigations to understand the mass transport mechanism and enhance the overall performance of TFN membranes. Firstly, I developed a novel multifunctional NF membrane by incorporating cellulose nanocrystal /silver (CNC/Ag) nanocomposites into the active layer of the thin film composite NF membrane. The CNC/Ag NF membrane exhibited excellent anti-fouling and antibacterial properties, in addition to an enhancement in water permeability without sacrificing salt rejection. Second, based upon the results observed from our TFN membranes experiments and other works done in recent years, I developed a novel water and solute transport model to explain the widely observed flux enhancement and solute rejection changes of TFN membranes using resistance-in-series theory and Monte Carlo simulation. The simulation results explain why a small amount of NPs addition (typically less than 0.1 wt% in the monomer solution for interfacial polymerization) would result in a significant flux increase without sacrificing rejection. Also, the range of the flux enhancement (1.1 to 2.5 times) and minor rejection change simulated by our model agree very well with the existing experimental data from a variety of previous publications. Finally, the TFC membrane real-time compaction model was developed based on scaling-law calculations combined with the viscoelastic behavior of the polymer. The effect of different morphological and mechanical properties, as well as the transmembrane pressure (TMP) are thoroughly discussed, which will not only give insight on the anti-compaction behavior of TFC membranes, but also contribute to future study of the TFN membrane compaction mechanism. In summary, this thesis has successfully demonstrated the preparation of a multifunctional TFN membrane with high performance under different assessment criteria, and the development of two mathematical models: a model that captures and explains the mass transport of TFN membranes, and a model that can be used to explain and predict TFN membrane compaction using viscoelastic theory. These works paved the way to future studies of novel TFN membranes, as well as developed a new set of math modeling techniques that can explain and predict the performance of TFN membranes.Ph.D

Effects of different chemical materials and cultural methods on growth and yield of winter wheat

To determine the effects of different chemical and cultural methods on the growth of winter wheat, six treatments were carried out: Conservational irrigation, non-irrigation, water absorbent polymers (WAP), liquid mulching film (LMF), water-saving irrigation (WSI) and subsoiling tillage (SST). The results show that winter wheat could use more water from soil profile though WAP, LMF and SST treatments; only LMF could use extra water for yield while both WAP and SST could not increase yield. SST could not increase yield of winter wheat. Both LMF and WAP treatments could help in maintaining leaf chlorophyll content and leaf water content which may help in maintaining photosynthetic ability in late growing periods. Furthermore, more dry matter partitioning to reproductive organs is observed in LMF and WAP treatments. LMF might be favorable for yield when grown under lower soil moisture conditions, while the application of WAP might not help in yield producing in field both in high or low soil moisture conditions. A reasonable irrigation quantity may be needed when applying WAP, while LMF could be used in any meteorological and/or soil water conditions.Keywords: Winter wheat, water absorbent polymers, liquid mulching film, subsoiling tillageAfrican Journal of Biotechnology Vol. 12(36), pp. 5522-552