On the Analysis of Trajectories of Gradient Descent in the Optimization of Deep Neural Networks

Theoretical analysis of the error landscape of deep neural networks has garnered significant interest in recent years. In this work, we theoretically study the importance of noise in the trajectories of gradient descent towards optimal solutions in multi-layer neural networks. We show that adding noise (in different ways) to a neural network while training increases the rank of the product of weight matrices of a multi-layer linear neural network. We thus study how adding noise can assist reaching a global optimum when the product matrix is full-rank (under certain conditions). We establish theoretical foundations between the noise induced into the neural network - either to the gradient, to the architecture, or to the input/output to a neural network - and the rank of product of weight matrices. We corroborate our theoretical findings with empirical results.Comment: 4 pages + 1 figure (main, excluding references), 5 pages + 4 figures (appendix

ADINE: An Adaptive Momentum Method for Stochastic Gradient Descent

Two major momentum-based techniques that have achieved tremendous success in optimization are Polyak's heavy ball method and Nesterov's accelerated gradient. A crucial step in all momentum-based methods is the choice of the momentum parameter $m$ which is always suggested to be set to less than $1$. Although the choice of $m < 1$ is justified only under very strong theoretical assumptions, it works well in practice even when the assumptions do not necessarily hold. In this paper, we propose a new momentum based method $\textit{ADINE}$, which relaxes the constraint of $m < 1$ and allows the learning algorithm to use adaptive higher momentum. We motivate our hypothesis on $m$ by experimentally verifying that a higher momentum ($\ge 1$) can help escape saddles much faster. Using this motivation, we propose our method $\textit{ADINE}$ that helps weigh the previous updates more (by setting the momentum parameter $> 1$), evaluate our proposed algorithm on deep neural networks and show that $\textit{ADINE}$ helps the learning algorithm to converge much faster without compromising on the generalization error.Comment: 8 + 1 pages, 12 figures, accepted at CoDS-COMAD 201

Structure-Based Phylogeny as a Diagnostic for Functional Characterization of Proteins with a Cupin Fold

The members of cupin superfamily exhibit large variations in their sequences, functions, organization of domains, quaternary associations and the nature of bound metal ion, despite having a conserved beta-barrel structural scaffold. Here, an attempt has been made to understand structure-function relationships among the members of this diverse superfamily and identify the principles governing functional diversity. The cupin superfamily also contains proteins for which the structures are available through world-wide structural genomics initiatives but characterized as "hypothetical". We have explored the feasibility of obtaining clues to functions of such proteins by means of comparative analysis with cupins of known structure and function.A 3-D structure-based phylogenetic approach was undertaken. Interestingly, a dendrogram generated solely on the basis of structural dissimilarity measure at the level of domain folds was found to cluster functionally similar members. This clustering also reflects an independent evolution of the two domains in bicupins. Close examination of structural superposition of members across various functional clusters reveals structural variations in regions that not only form the active site pocket but are also involved in interaction with another domain in the same polypeptide or in the oligomer.Structure-based phylogeny of cupins can influence identification of functions of proteins of yet unknown function with cupin fold. This approach can be extended to other proteins with a common fold that show high evolutionary divergence. This approach is expected to have an influence on the function annotation in structural genomics initiatives

Mutation causing self-aggregation in human γC-crystallin leading to congenital cataract

Purpose: Many forms of congenital hereditary cataract are associated with mutations in the crystallin genes. The authors focus attention on congenital lamellar cataract, which is associated with the R168W mutation in γC-crystallin, and congenital zonular pulverulent cataract, which is associated with a 5-bp insertion in the γC-crystallin gene. Methods: To understand the molecular phenotypes-i.e., the functional defects that have occurred in the mutant γC-crystallin molecule in two cases described-the authors cloned, expressed, isolated, and compared the solution state structural features of these mutants with those of normal (wild-type) γC-crystallin. Structural models of the wild-type and mutant have been generated using comparative modeling. Circular dichroism and fluorescence spectroscopic methods were used to determine the conformation of the proteins, and temperature dependent self-aggregation was used to observe the quaternary structural features. The structural stability of the proteins was monitored with the use of chemical and thermal denaturation. Results: The authors found that the 5-bp insertion led to a loss of secondary and tertiary structures of the molecule and to an enhanced tendency of self-aggregation into light-scattering particles, offering a possible factor in lens opacification. The R168W mutant, on the other hand, was remarkably similar to the wild-type molecule in its conformation and structural stability, but it differed in its ability to aggregate and scatter light. Conclusions: These results support the idea that unfolding or structural destabilization is not always necessary for crystallin-associated cataractogenesis

AI and IoT-Enabled smart exoskeleton system for rehabilitation of paralyzed people in connected communities

In recent years, the number of cases of spinal cord injuries, stroke and other nervous impairments have led to an increase in the number of paralyzed patients worldwide. Rehabilitation that can aid and enhance the lives of such patients is the need of the hour. Exoskeletons have been found as one of the popular means of rehabilitation. The existing exoskeletons use techniques that impose limitations on adaptability, instant response and continuous control. Also most of them are expensive, bulky, and requires high level of training. To overcome all the above limitations, this paper introduces an Artificial Intelligence (AI) powered Smart and light weight Exoskeleton System (AI-IoT-SES) which receives data from various sensors, classifies them intelligently and generates the desired commands via Internet of Things (IoT) for rendering rehabilitation and support with the help of caretakers for paralyzed patients in smart and connected communities. In the proposed system, the signals collected from the exoskeleton sensors are processed using AI-assisted navigation module, and helps the caretakers in guiding, communicating and controlling the movements of the exoskeleton integrated to the patients. The navigation module uses AI and IoT enabled Simultaneous Localization and Mapping (SLAM). The casualties of a paralyzed person are reduced by commissioning the IoT platform to exchange data from the intelligent sensors with the remote location of the caretaker to monitor the real time movement and navigation of the exoskeleton. The automated exoskeleton detects and take decisions on navigation thereby improving the life conditions of such patients. The experimental results simulated using MATLAB shows that the proposed system is the ideal method for rendering rehabilitation and support for paralyzed patients in smart communities. © 2013 IEEE. **Please note that there are multiple authors for this article therefore only the name of the first 5 including Federation University Australia affiliate “Venki Balasubramanian” is provided in this record*

Evaluation of in vitro antimicrobial property of seaweed (Halimeda tuna) from Tuticorin coast, Tamil Nadu, Southeast coast of India

The seaweed (Halimeda tuna) was examined for antibacterial and antifungal activity in vitro using the well diffusion method, minimum inhibitory concentration, minimum bactericidal concentration and minimum fungicidal concentration. The activity was against 10 bacterial strains (Staphylococcus aureus, Salmonella typhimurium, Salmonella paratyphi, Klebsiella oxytoca, Escherichia coli, Proteus mirabillis, Lactobacillus vulgaris, Pseudomonas sp., Klebsiella pneumonia and Vibrio cholerae) and nine fungal strains (Aspergillus niger, Aspergillus flavus, Alternaria alternaria, Candida albicans, Epidermophyton floccossum, Trichophyton mentagrophytes, Trichophyton rubrum, Pencillium sp. and Rhizopus sp.). The methanolic extracts in the present study exhibited a broad spectrum of antimicrobial activity compared to the ethanolic and chloroform extracts. Results of the present study confirm the potential use of seaweed extracts as a source of antimicrobial compound.Keywords: Halimeda tuna, minimum inhibitory concentration, minimum bactericidal concentration, minimum fungicidal concentration, antimicrobial activityAfrican Journal of Biotechnology Vol. 12(3), pp. 284-28

Steric hindrance in the upper 50 kDa domain of the motor Myo2p leads to cytokinesis defects in fission yeast

Cytokinesis in many eukaryotes requires a contractile actomyosin ring that is placed at the division site. In fission yeast, which is an attractive organism for the study of cytokinesis, actomyosin ring assembly and contraction requires the myosin II heavy chain Myo2p. Although myo2-E1, a temperature-sensitive mutant defective in the upper 50 kDa domain of Myo2p, has been studied extensively, the molecular basis of the cytokinesis defect is not understood. Here, we isolate myo2-E1-Sup2, an intragenic suppressor that contains the original mutation in myo2-E1 (G345R) and a second mutation in the upper 50 kDa domain (Y297C). Unlike myo2-E1-Sup1, a previously characterized myo2-E1 suppressor, myo2-E1-Sup2 reverses actomyosin ring contraction defects in vitro and in vivo. Structural analysis of available myosin motor domain conformations suggests that a steric clash in myo2-E1, which is caused by the replacement of a glycine with a bulky arginine, is relieved in myo2-E1-Sup2 by mutation of a tyrosine to a smaller cysteine. Our work provides insight into the function of the upper 50 kDa domain of Myo2p, informs a molecular basis for the cytokinesis defect in myo2-E1, and may be relevant to the understanding of certain cardiomyopathies

Mutation Causing Self-Aggregation in Human ␥C-Crystallin Leading to Congenital Cataract

PURPOSE. Many forms of congenital hereditary cataract are associated with mutations in the crystallin genes. The authors focus attention on congenital lamellar cataract, which is associated with the R168W mutation in ␥C-crystallin, and congenital zonular pulverulent cataract, which is associated with a 5-bp insertion in the ␥C-crystallin gene. METHODS. To understand the molecular phenotypes-i.e., the functional defects that have occurred in the mutant ␥C-crystallin molecule in two cases described-the authors cloned, expressed, isolated, and compared the solution state structural features of these mutants with those of normal (wild-type) ␥C-crystallin. Structural models of the wild-type and mutant have been generated using comparative modeling. Circular dichroism and fluorescence spectroscopic methods were used to determine the conformation of the proteins, and temperature dependent self-aggregation was used to observe the quaternary structural features. The structural stability of the proteins was monitored with the use of chemical and thermal denaturation. RESULTS. The authors found that the 5-bp insertion led to a loss of secondary and tertiary structures of the molecule and to an enhanced tendency of self-aggregation into light-scattering particles, offering a possible factor in lens opacification. The R168W mutant, on the other hand, was remarkably similar to the wild-type molecule in its conformation and structural stability, but it differed in its ability to aggregate and scatter light. CONCLUSIONS. These results support the idea that unfolding or structural destabilization is not always necessary for crystallinassociated cataractogenesis. (Invest Ophthalmol Vis Sci. 2006; 47:5212-5217

Motor activity dependent and independent functions for Myosin II in cytokinesis contribute to actomyosin ring assembly and contraction in Schizosaccharomyces pombe

Cytokinesis depends on a contractile actomyosin ring in many eukaryotes [1, 2, 3]. Myosin II is a key component of the actomyosin ring, although whether it functions as a motor or as an actin cross-linker to exert its essential role is disputed [1, 4, 5]. In Schizosaccharomyces pombe, the myo2-E1 mutation affects the upper 50 kDa sub-domain of the myosin II heavy chain, and cells carrying this lethal mutation are defective in actomyosin ring assembly at the non-permissive temperature [6, 7]. myo2-E1 also affects actomyosin ring contraction when rings isolated from permissive temperature-grown cells are incubated with ATP [8]. Here we report isolation of a compensatory suppressor mutation in the lower 50 kDa sub-domain (myo2-E1-Sup1) that reverses the inability of myo2-E1 to form colonies at the restrictive temperature. myo2-E1-Sup1 is capable of assembling normal actomyosin rings, although rings isolated from myo2-E1-Sup1 are defective in ATP-dependent contraction in vitro. Furthermore, the product of myo2-E1-Sup1 does not translocate actin filaments in motility assays in vitro. Superimposition of myo2-E1 and myo2-E1-Sup1 on available rigor and blebbistatin-bound myosin II structures suggests that myo2-E1-Sup1 may represent a novel actin translocation-defective allele. Actomyosin ring contraction and viability of myo2-E1-Sup1 cells depend on the late cytokinetic S. pombe myosin II isoform, Myp2p, a non-essential protein that is normally dispensable for actomyosin ring assembly and contraction. Our work reveals that Myo2p may function in two different and essential modes during cytokinesis: a motor activity-independent form that can promote actomyosin ring assembly and a motor activity-dependent form that supports ring contraction