Homophone Reveals the Truth: A Reality Check for Speech2Vec

Generating spoken word embeddings that possess semantic information is a fascinating topic. Compared with text-based embeddings, they cover both phonetic and semantic characteristics, which can provide richer information and are potentially helpful for improving ASR and speech translation systems. In this paper, we review and examine the authenticity of a seminal work in this field: Speech2Vec. First, a homophone-based inspection method is proposed to check the speech embeddings released by the author of Speech2Vec. There is no indication that these embeddings are generated by the Speech2Vec model. Moreover, through further analysis of the vocabulary composition, we suspect that a text-based model fabricates these embeddings. Finally, we reproduce the Speech2Vec model, referring to the official code and optimal settings in the original paper. Experiments showed that this model failed to learn effective semantic embeddings. In word similarity benchmarks, it gets a correlation score of 0.08 in MEN and 0.15 in WS-353-SIM tests, which is over 0.5 lower than those described in the original paper. Our data and code are available.Comment: Corrected typo

The enhanced soliton propagation and energy transfer in the coupled drift wave and energetic-particle-induced geodesic acoustic mode system

The evolution of the coupled drift wave (DW) and energetic-particle-induced geodesic acoustic mode (EGAM) nonlinear system is investigated using the fully nonlinear coupled DW-EGAM two-field equations, with emphasis on the turbulence spreading in the form of soliton and the nonlinear energy transfer between DW and EGAM. Four scenarios with different combinations of EGAM initial amplitudes and linear EGAM growth rates are designed to delineate the effects of linear EGAM drive and finite EGAM amplitude on DW nonlinear dynamic evolution. In presence of the linear EPs drive, the soliton propagation is enhanced, due to the generation of small radial scale structures. Two conservation laws of the nonlinear system are derived, including the energy conservation law. It is found that the energy of DW always decreases and that of EGAM always increases, leading to regulation of DW by EGAM.Comment: 19 figures, 9 page

Exploring Parameter-Efficient Fine-tuning for Improving Communication Efficiency in Federated Learning

Federated learning (FL) has emerged as a promising paradigm for enabling the collaborative training of models without centralized access to the raw data on local devices. In the typical FL paradigm (e.g., FedAvg), model weights are sent to and from the server each round to participating clients. However, this can quickly put a massive communication burden on the system, especially if more capable models beyond very small MLPs are employed. Recently, the use of pre-trained models has been shown effective in federated learning optimization and improving convergence. This opens the door for new research questions. Can we adjust the weight-sharing paradigm in federated learning, leveraging strong and readily-available pre-trained models, to significantly reduce the communication burden while simultaneously achieving excellent performance? To this end, we investigate the use of parameter-efficient fine-tuning in federated learning. Specifically, we systemically evaluate the performance of several parameter-efficient fine-tuning methods across a variety of client stability, data distribution, and differential privacy settings. By only locally tuning and globally sharing a small portion of the model weights, significant reductions in the total communication overhead can be achieved while maintaining competitive performance in a wide range of federated learning scenarios, providing insight into a new paradigm for practical and effective federated systems

FedPerfix: Towards Partial Model Personalization of Vision Transformers in Federated Learning

Personalized Federated Learning (PFL) represents a promising solution for decentralized learning in heterogeneous data environments. Partial model personalization has been proposed to improve the efficiency of PFL by selectively updating local model parameters instead of aggregating all of them. However, previous work on partial model personalization has mainly focused on Convolutional Neural Networks (CNNs), leaving a gap in understanding how it can be applied to other popular models such as Vision Transformers (ViTs). In this work, we investigate where and how to partially personalize a ViT model. Specifically, we empirically evaluate the sensitivity to data distribution of each type of layer. Based on the insights that the self-attention layer and the classification head are the most sensitive parts of a ViT, we propose a novel approach called FedPerfix, which leverages plugins to transfer information from the aggregated model to the local client as a personalization. Finally, we evaluate the proposed approach on CIFAR-100, OrganAMNIST, and Office-Home datasets and demonstrate its effectiveness in improving the model's performance compared to several advanced PFL methods.Comment: 2023 IEEE/CVF International Conference on Computer Vision (ICCV

Direct Preference Optimization for Neural Machine Translation with Minimum Bayes Risk Decoding

Minimum Bayes Risk (MBR) decoding can significantly improve translation performance of Multilingual Large Language Models (MLLMs). However, MBR decoding is computationally expensive and in this paper, we show how recently developed Reinforcement Learning (RL) technique, Direct Preference Optimization (DPO) can be used to fine-tune MLLMs so that we get the gains from MBR without the additional computation in inference. Our fine-tuned models have significantly improved performance on multiple NMT test sets compared to base MLLMs without preference optimization. Our method boosts the translation performance of MLLMs using relatively small monolingual fine-tuning sets

Measuring compactness of the urban landscape within a city territory for environmental capabilities: the case of 50 cities in eastern China

[EN] When a compact city is pursued as the strategy for urban sustainability the understanding of compactness is varied from the developed countries to the developing countries. In China the historical cities are characterized with high density and mixed land use. After a short time of motorization they still show compact forms in the central city. A large amount of newly developed areas are distributed in clusters near or far from the center. The crop land and natural habitat are encroached to different degrees. This paper developed an approach to measure the structural compactness of urban landscape patches within a city territory. It included six spatial metrics to measure the shape and density of the central agglomeration, the area configuration and distant relationship between the central agglomeration and the other clusters, and the distribution of all urban patches. By this approach the 50 cities in eastern China were categorized into five classes of forms: one center, multi-centers, centralized groups, cluster groups and scatter. Then the vegetation biomass loss with urban expansion was calculated based on remote sensing data, and used to assess the environmental capability of the five types of urban form. The suggestions of urban form optimization could be put forward for the five categories of cities.Shuang, CS.; Tong, Z.; Guangyu, L.; Yue, Y. (2018). Measuring compactness of the urban landscape within a city territory for environmental capabilities: the case of 50 cities in eastern China. En 24th ISUF International Conference. Book of Papers. Editorial Universitat Politècnica de València. 13-20. https://doi.org/10.4995/ISUF2017.2017.5094OCS132

An Algorithm for Finding Functional Modules and Protein Complexes in Protein-Protein Interaction Networks

Biological processes are often performed by a group of proteins rather than by individual proteins, and proteins in a same biological group form a densely connected subgraph in a protein-protein interaction network. Therefore, finding a densely connected subgraph provides useful information to predict the function or protein complex of uncharacterized proteins in the highly connected subgraph. We have developed an efficient algorithm and program for finding cliques and near-cliques in a protein-protein interaction network. Analysis of the interaction network of yeast proteins using the algorithm demonstrates that 59% of the near-cliques identified by our algorithm have at least one function shared by all the proteins within a near-clique, and that 56% of the near-cliques show a good agreement with the experimentally determined protein complexes catalogued in MIPS