A Circuit-Based Neural Network with Hybrid Learning of Backpropagation and Random Weight Change Algorithms.

A hybrid learning method of a software-based backpropagation learning and a hardware-based RWC learning is proposed for the development of circuit-based neural networks. The backpropagation is known as one of the most efficient learning algorithms. A weak point is that its hardware implementation is extremely difficult. The RWC algorithm, which is very easy to implement with respect to its hardware circuits, takes too many iterations for learning. The proposed learning algorithm is a hybrid one of these two. The main learning is performed with a software version of the BP algorithm, firstly, and then, learned weights are transplanted on a hardware version of a neural circuit. At the time of the weight transplantation, a significant amount of output error would occur due to the characteristic difference between the software and the hardware. In the proposed method, such error is reduced via a complementary learning of the RWC algorithm, which is implemented in a simple hardware. The usefulness of the proposed hybrid learning system is verified via simulations upon several classical learning problems

Porting the Sisal functional language to distributed-memory multiprocessors

Parallel computing is becoming increasingly ubiquitous in recent years. The sizes of application problems continuously increase for solving real-world problems. Distributed-memory multiprocessors have been regarded as a viable architecture of scalable and economical design for building large scale parallel machines. While these parallel machines can provide computational capabilities, programming such large-scale machines is often very difficult due to many practical issues including parallelization, data distribution, workload distribution, and remote memory latency. This thesis proposes to solve the programmability and performance issues of distributed-memory machines using the Sisal functional language. The programs written in Sisal will be automatically parallelized, scheduled and run on distributed-memory multiprocessors with no programmer intervention. Specifically, the proposed approach consists of the following steps. Given a program written in Sisal, the front end Sisal compiler generates a directed acyclic graph(DAG) to expose parallelism in the program. The DAG is partitioned and scheduled based on loop parallelism. The scheduled DAG is then translated to C programs with machine specific parallel constructs. The parallel C programs are finally compiled by the target machine specific compilers to generate executables. A distributed-memory parallel machine, the 80-processor ETL EM-X, has been chosen to perform experiments. The entire procedure has been implemented on the EMX multiprocessor. Four problems are selected for experiments: bitonic sorting, search, dot-product and Fast Fourier Transform. Preliminary execution results indicate that automatic parallelization of the Sisal programs based on loop parallelism is effective. The speedup for these four problems is ranging from 17 to 60 on a 64-processor EM-X. Preliminary experimental results further indicate that programming distributed-memory multiprocessors using a functional language indeed frees the programmers from lowl-evel programming details while allowing them to focus on algorithmic performance improvement

Intrinsically Evolvable Artificial Neural Networks

Dedicated hardware implementations of neural networks promise to provide faster, lower power operation when compared to software implementations executing on processors. Unfortunately, most custom hardware implementations do not support intrinsic training of these networks on-chip. The training is typically done using offline software simulations and the obtained network is synthesized and targeted to the hardware offline. The FPGA design presented here facilitates on-chip intrinsic training of artificial neural networks. Block-based neural networks (BbNN), the type of artificial neural networks implemented here, are grid-based networks neuron blocks. These networks are trained using genetic algorithms to simultaneously optimize the network structure and the internal synaptic parameters. The design supports online structure and parameter updates, and is an intrinsically evolvable BbNN platform supporting functional-level hardware evolution. Functional-level evolvable hardware (EHW) uses evolutionary algorithms to evolve interconnections and internal parameters of functional modules in reconfigurable computing systems such as FPGAs. Functional modules can be any hardware modules such as multipliers, adders, and trigonometric functions. In the implementation presented, the functional module is a neuron block. The designed platform is suitable for applications in dynamic environments, and can be adapted and retrained online. The online training capability has been demonstrated using a case study. A performance characterization model for RC implementations of BbNNs has also been presented

Jais and Jais-chat: Arabic-Centric Foundation and Instruction-Tuned Open Generative Large Language Models

We introduce Jais and Jais-chat, new state-of-the-art Arabic-centric foundation and instruction-tuned open generative large language models (LLMs). The models are based on the GPT-3 decoder-only architecture and are pretrained on a mixture of Arabic and English texts, including source code in various programming languages. With 13 billion parameters, they demonstrate better knowledge and reasoning capabilities in Arabic than any existing open Arabic and multilingual models by a sizable margin, based on extensive evaluation. Moreover, the models are competitive in English compared to English-centric open models of similar size, despite being trained on much less English data. We provide a detailed description of the training, the tuning, the safety alignment, and the evaluation of the models. We release two open versions of the model -- the foundation Jais model, and an instruction-tuned Jais-chat variant -- with the aim of promoting research on Arabic LLMs. Available at https://huggingface.co/inception-mbzuai/jais-13b-chatComment: Arabic-centric, foundation model, large-language model, LLM, generative model, instruction-tuned, Jais, Jais-cha

INTRASPECIFIC VARIATION IN DEHYDRATION TOLERANCE: INSIGHTS FROM THE TROPICAL PLANT \u3cem\u3eMARCHANTIA INFLEXA\u3c/em\u3e

Plants are threatened by global change, increasing variability in weather patterns, and associated abiotic stress. Consequently, there is an urgent need to enhance our ability to predict plant community dynamics, shifts in species distributions, and physiological responses to environmental challenges. By building a fundamental understanding of plant stress tolerance, it may be possibly to protect the ecological services, economic industries, and communities that depend on plants. Dehydration tolerance (DhT) is an important mechanism of water stress tolerance with promising translational applications. Here, I take advantage natural variation in DhT to gain a deeper insight into this complex trait. In addition, I address questions related to the causes and consequences of sexual dimorphisms in DhT. Understanding sexual dimorphisms in stress tolerance is critical because these dimorphisms can drive spatial segregation of the sexes, biased sex ratios, and may ultimately reduce sexual reproduction and population persistence. This work takes an integrated approach, addressing DhT on multiple scales from ecology, to physiology, to genomics in the tropical liverwort Marchantia inflexa. Initially, I tested for correlations between DhT and environmental dryness, sex differences in DhT, and genetic vs. plastic contributions to DhT variability. I found that patterns of variation in DhT are associated with environmental variability, including complex sexual dimorphisms, and derive from a combination of plasticity and genetic differences in DhT. Subsequently, I leveraged the variability in DhT to identify candidate DhT enhancing genes. In M. inflexa intraspecific differences in DhT are impacted by baseline variability among plants, as well as unique gene expression responses initiated during drying. In parallel, I assembled a draft genome assembly for M. inflexa, which was employed to investigate questions of sex chromosome evolution and sexual dimorphism in DhT. Finally, the bacteriome of M. inflexa was characterized and found to be extremely diverse and variable. Collectively, this work adds to a growing understanding of DhT and highlights the importance of sampling approaches that seek to comprehensively describe variability in DhT. I detected complex patterns of variability in DhT among populations and the sexes of M. inflexa, which were used to gain insight into the genetic intricacies of DhT

High-throughput sequencing for the analysis of genomic DNA and gene expression in Populus spp.

The genus Populus is an important crop and a model system to understand molecular processes of growth, development, and responses to environmental stimuli in trees. Moreover the entire genome of Populus trichocarpa was sequenced. The aim of this research was studying genomic variation and evolution in the poplar genus, and the effects of such variations in producing heterosis in two interspecific hybrids between Populus deltoides and P. nigra. Heterosis, intended as the superior performance of hybrid progeny compared to their inbred parents, has been one of the driving forces in poplar breeding. The two interspecific hybrids used in our experiments exhibit different levels of heterosis, i.e., their productivity is for one genotype much larger than that of parents and, for the other genotype, is similar to that of parents. The molecular bases of heterosis are still to be fully clarified, though it appears that variations in intergenic regions can have a role in the heterotic phenotype. Hence, we studied the extent of variation in the repetitive component of the genome (especially retrotransposons) and its possible consequences on gene and allelic expression. During this research, bioinformatic and genomic analyses were performed aiming i) to characterize the repetitive component of the poplar genome, by the isolation and characterization of LTR-retrotransposons in the P. trichocarpa genome, and the production of a database of such elements; moreover the previously undescribed structure of poplar centromeres was evaluated by means of NGS techniques; ii) to analyze poplar genome repetitive component and its expression, studying, by Illumina RNAseq, the transcription of previously isolated LTR-retrotransposons, in control and drought stressed plants; iii) to study the poplar transcriptome, also in relation to drought and, for an indirect evaluation of cis-regulatory sequence variation in the poplar hybrid, to the differential expression between alleles in genes expressed in control and drought stress. Concerning LTR-retrotransposons, we observed a relatively recent burst of retrotransposons activity, though counterbalanced by high levels of DNA loss. A huge fraction of retrotransposons belong to unknown superfamilies, i.e. they are non-autonomous retrotransposons because lacking coding capacity. These elements are especially expressed in poplars. We also individuated two distinct centromeric repeats, that occur in all three analysed poplar species. Gene expression was analysed mapping RNAseq data to the complete poplar transcriptome, and a reference expression dataset was established. In several instances, the two alleles in a hybrid are flanked by different DNA sequences, affecting tissue specificity or temporal regulation of expression of genes. We found allele specific expression in many of 200 randomly chosen genes in different stress conditions. This suggests a differential role for the two alleles during hybrid growth and in its interaction with the environment. It is possible that the functional diversity of the two parental alleles in the hybrid may have an impact on hybrid performance through allelic complementation