Synchronization for a class of generalized neural networks with interval time-varying delays and reaction-diffusion terms

In this paper, the synchronization problem for a class of generalized neural networks with interval time-varying delays and reaction-diffusion terms is investigated under Dirichlet boundary conditions and Neumann boundary conditions, respectively. Based on Lyapunov stability theory, both delay-derivative-dependent and delay-range-dependent conditions are derived in terms of linear matrix inequalities (LMIs), whose solvability heavily depends on the information of reaction-diffusion terms. The proposed generalized neural networks model includes reaction-diffusion local field neural networks and reaction-diffusion static neural networks as its special cases. The obtained synchronization results are easy to check and improve upon the existing ones. In our results, the assumptions for the differentiability and monotonicity on the activation functions are removed. It is assumed that the state delay belongs to a given interval, which means that the lower bound of delay is not restricted to be zero. Finally, the feasibility and effectiveness of the proposed methods is shown by simulation examples

AHP Aided Decision-Making in Virtual Machine Migration for Green Cloud

In this study, an analytical hierarchy process based model is proposed to perform the decision-making for virtual machine migration towards green cloud computing. The virtual machine migration evaluation index system is established based on the process of constructing hierarchies for evaluation of virtual machine migration, and selection of task usage. A comparative judgment of two hierarchies has been conducted. In the experimental study, five-point rating scale has been adopted to map the raw data to the scaled rating score; this rating method is used to analyze the performance of each virtual machine and its task usage data. The results show a significant improvement in the decision-making process for the virtual machine migration. The deduced results are useful for the system administrators to migrate the exact virtual machine, and then switch on the power of physical machine that the migrated virtual machine resides on. Thus the proposed method contributes to the green cloud computing environment

Federated Learning of Large Language Models with Parameter-Efficient Prompt Tuning and Adaptive Optimization

Federated learning (FL) is a promising paradigm to enable collaborative model training with decentralized data. However, the training process of Large Language Models (LLMs) generally incurs the update of significant parameters, which limits the applicability of FL techniques to tackle the LLMs in real scenarios. Prompt tuning can significantly reduce the number of parameters to update, but it either incurs performance degradation or low training efficiency. The straightforward utilization of prompt tuning in the FL often raises non-trivial communication costs and dramatically degrades performance. In addition, the decentralized data is generally non-Independent and Identically Distributed (non-IID), which brings client drift problems and thus poor performance. This paper proposes a Parameter-efficient prompt Tuning approach with Adaptive Optimization, i.e., FedPepTAO, to enable efficient and effective FL of LLMs. First, an efficient partial prompt tuning approach is proposed to improve performance and efficiency simultaneously. Second, a novel adaptive optimization method is developed to address the client drift problems on both the device and server sides to enhance performance further. Extensive experiments based on 10 datasets demonstrate the superb performance (up to 60.8\% in terms of accuracy) and efficiency (up to 97.59\% in terms of training time) of FedPepTAO compared with 9 baseline approaches. Our code is available at https://github.com/llm-eff/FedPepTAO.Comment: 18 pages, accepted by EMNLP 202

paper2repo: GitHub Repository Recommendation for Academic Papers

GitHub has become a popular social application platform, where a large number of users post their open source projects. In particular, an increasing number of researchers release repositories of source code related to their research papers in order to attract more people to follow their work. Motivated by this trend, we describe a novel item-item cross-platform recommender system, $\textit{paper2repo}$, that recommends relevant repositories on GitHub that match a given paper in an academic search system such as Microsoft Academic. The key challenge is to identify the similarity between an input paper and its related repositories across the two platforms, $\textit{without the benefit of human labeling}$. Towards that end, paper2repo integrates text encoding and constrained graph convolutional networks (GCN) to automatically learn and map the embeddings of papers and repositories into the same space, where proximity offers the basis for recommendation. To make our method more practical in real life systems, labels used for model training are computed automatically from features of user actions on GitHub. In machine learning, such automatic labeling is often called {\em distant supervision\/}. To the authors' knowledge, this is the first distant-supervised cross-platform (paper to repository) matching system. We evaluate the performance of paper2repo on real-world data sets collected from GitHub and Microsoft Academic. Results demonstrate that it outperforms other state of the art recommendation methods

Adjoint Tomography of Ambient Noise Data and Teleseismic P Waves: Methodology and Applications to Central California

Adjoint tomography has been recently applied to ambient seismic noise and teleseismic P waves separately to unveil fine-scale lithospheric structures beyond the resolving ability of traditional ray-based traveltime tomography. In this study, we propose an inversion scheme that alternates between frequency-dependent traveltime inversions of ambient noise surface waves and waveform inversions of teleseismic P waves to take advantage of their complementary sensitivities to the Earth's structure. We apply our method to ambient noise empirical Green's functions from 60 virtual sources, direct P and scattered waves from 11 teleseismic events recorded by a dense linear array (∼7 km station spacing) and other regional stations (∼40 km average station spacing) in central California. To evaluate the performance of the method, we compare tomographic results from ambient noise adjoint tomography, full-waveform inversion of teleseismic P waves, and the alternating inversion of the two data sets. Both applications to practical field data sets and synthetic checkerboard tests demonstrate the advantage of the alternating inversion over individual inversions as it combines the complementary sensitivities of the two independent data sets toward a more unified model. The three dimensional model from our alternating inversion not only shows major features of velocity anomalies and discontinuities in agreement with previous studies, but also reveals small-scale heterogeneities which provide new constraints on the geometry of the Isabella Anomaly and mantle dynamic processes in central California. The proposed alternating inversion scheme can be applied to other regions with similar array deployments for high-resolution lithospheric imaging.K. Wang (after January 2020) and Y. Yang are supported by the Australian Research Council Discovery Grants DP190102940. K. Wang (before January 2020) and Q. Liu are supported by the NSERC Discovery Grant 487237. This is contribution 1664 from the ARC Center of Excellence for Core to Crust Fluid Systems and 1465 in the GEMOC Key Center

STFNets: Learning Sensing Signals from the Time-Frequency Perspective with Short-Time Fourier Neural Networks

Recent advances in deep learning motivate the use of deep neural networks in Internet-of-Things (IoT) applications. These networks are modelled after signal processing in the human brain, thereby leading to significant advantages at perceptual tasks such as vision and speech recognition. IoT applications, however, often measure physical phenomena, where the underlying physics (such as inertia, wireless signal propagation, or the natural frequency of oscillation) are fundamentally a function of signal frequencies, offering better features in the frequency domain. This observation leads to a fundamental question: For IoT applications, can one develop a new brand of neural network structures that synthesize features inspired not only by the biology of human perception but also by the fundamental nature of physics? Hence, in this paper, instead of using conventional building blocks (e.g., convolutional and recurrent layers), we propose a new foundational neural network building block, the Short-Time Fourier Neural Network (STFNet). It integrates a widely-used time-frequency analysis method, the Short-Time Fourier Transform, into data processing to learn features directly in the frequency domain, where the physics of underlying phenomena leave better foot-prints. STFNets bring additional flexibility to time-frequency analysis by offering novel nonlinear learnable operations that are spectral-compatible. Moreover, STFNets show that transforming signals to a domain that is more connected to the underlying physics greatly simplifies the learning process. We demonstrate the effectiveness of STFNets with extensive experiments. STFNets significantly outperform the state-of-the-art deep learning models in all experiments. A STFNet, therefore, demonstrates superior capability as the fundamental building block of deep neural networks for IoT applications for various sensor inputs

Neuromorphic visual artificial synapse in-memory computing systems based on GeOx-coated MXene nanosheets

Artificial synapses with light signal perception capability offer the ability to neuromorphic visual signal processing system on demand. In light of the excellent optical and electrical characteristics, the low-dimensional materials have become one of the most favorable candidates of the key component for optoelectronic artificial synapses. Previously, our group originally proposed the synthesis of germanium oxide-coated MXene nanosheets. In this work, we further applied this technology into the optoelectronic synaptic thin-film transistors for the first time. The devices exhibited the adjustable postsynaptic current behaviors under the visible light inputs. Moreover, the potentiation and depression operation modes of the devices further improved the application potential of the devices in mimicking biological synapses. Regulated by the wavelength of incident lights, the proposed artificial synapse could effectively help detect the target area of the image. Eventually, we further showed the results of the devices in the projects of neural network computing task. The long-term potentiation/depression characteristics of the conductance were applied to the synaptic weight matrix for image identification and path recognition tasks. By adding knowledge transfer in the process of recognition, the epoch required for convergence has been greatly reduced. The result of high noise tolerance revealed the great potential of the proposed transistors in establishing high-efficiency and robustness hardware neuromorphic systems for in-memory computing

Advanced synaptic transistor device towards AI application in hardware perspective

For the past decades, the synaptic devices for the inmemory computing have been widely investigated due to the high-efficiency computing potential and the ability to mimic biological neurobehavior. However, the conventional twoterminal synaptic memristors show drawbacks of resistance reduction caused by large-scale paralleling and asynchronous storage-reading process, which limit its development. Recently, researchers have paid attention to the transistor-like artificial synapse. Due to the existence of insulator layer and the separation of input and read terminals, the three-terminal synaptic transistors are believed to have greater potential towards artificial intelligence (AI) application fields. In this work, a summary of recent progresses and the future challenges of synaptic transistors are discussed