Unbiased Watermark for Large Language Models

The recent advancements in large language models (LLMs) have sparked a growing apprehension regarding the potential misuse. One approach to mitigating this risk is to incorporate watermarking techniques into LLMs, allowing for the tracking and attribution of model outputs. This study examines a crucial aspect of watermarking: how significantly watermarks impact the quality of model-generated outputs. Previous studies have suggested a trade-off between watermark strength and output quality. However, our research demonstrates that it is possible to integrate watermarks without affecting the output probability distribution with appropriate implementation. We refer to this type of watermark as an unbiased watermark. This has significant implications for the use of LLMs, as it becomes impossible for users to discern whether a service provider has incorporated watermarks or not. Furthermore, the presence of watermarks does not compromise the performance of the model in downstream tasks, ensuring that the overall utility of the language model is preserved. Our findings contribute to the ongoing discussion around responsible AI development, suggesting that unbiased watermarks can serve as an effective means of tracking and attributing model outputs without sacrificing output quality

DataAI-6G: A System Parameters Configurable Channel Dataset for AI-6G Research

With the acceleration of the commercialization of fifth generation (5G) mobile communication technology and the research for 6G communication systems, the communication system has the characteristics of high frequency, multi-band, high speed movement of users and large antenna array. These bring many difficulties to obtain accurate channel state information (CSI), which makes the performance of traditional communication methods be greatly restricted. Therefore, there has been a lot of interest in using artificial intelligence (AI) instead of traditional methods to improve performance. A common and accurate dataset is essential for the research of AI communication. However, the common datasets nowadays still lack some important features, such as mobile features, spatial non-stationary features etc. To address these issues, we give a dataset for future 6G communication. In this dataset, we address these issues with specific simulation methods and accompanying code processing

A chromosomelevel genome assembly of the Asian arowana, Scleropages formosus

Asian arowana (Scleropages formosus), an ancient teleost belonging to the Order Osteoglossomorpha, has been a valuable ornamental fish with some varieties. However, its biological studies and breeding germplasm have been remarkably limited by the lack of a reference genome. To solve these problems, here we report high-quality genome sequences of three common varieties of Asian arowana (the golden, red and green arowana). We firstly generated a chromosome-level genome assembly of the golden arowana, on basis of the genetic linkage map constructed with the restriction site-associated DNA sequencing (RAD-seq). In addition, we obtained draft genome assemblies of the red and green varieties. Finally, we annotated 22,016, 21,256 and 21,524 protein-coding genes in the genome assemblies of golden, red and green varieties respectively. Our data were deposited in publicly accessible repositories to promote biological research and molecular breeding of Asian arowana

The Asian arowana (<i>Scleropages formosus</i>) genome provides new insights into the evolution of an early lineage of teleosts

The Asian arowana (Scleropages formosus), one of the world’s most expensive cultivated ornamental fishes, is an endangered species. It represents an ancient lineage of teleosts: the Osteoglossomorpha. Here, we provide a high-quality chromosome-level reference genome of a female golden-variety arowana using a combination of deep shotgun sequencing and high-resolution linkage mapping. In addition, we have also generated two draft genome assemblies for the red and green varieties. Phylogenomic analysis supports a sister group relationship between Osteoglossomorpha (bonytongues) and Elopomorpha (eels and relatives), with the two clades together forming a sister group of Clupeocephala which includes all the remaining teleosts. The arowana genome retains the full complement of eight Hox clusters unlike the African butterfly fish (Pantodon buchholzi), another bonytongue fish, which possess only five Hox clusters. Differential gene expression among three varieties provides insights into the genetic basis of colour variation. A potential heterogametic sex chromosome is identified in the female arowana karyotype, suggesting that the sex is determined by a ZW/ZZ sex chromosomal system. The high-quality reference genome of the golden arowana and the draft assemblies of the red and green varieties are valuable resources for understanding the biology, adaptation and behaviour of Asian arowanas

The Asian Arowana (Scleropages formosus) Genome Provides New Insights into the Evolution of an Early Lineage of Teleosts

The Asian arowana (Scleropages formosus), one of the world’s most expensive cultivated ornamental fishes, is an endangered species. It represents an ancient lineage of teleosts: the Osteoglossomorpha. Here, we provide a high-quality chromosome-level reference genome of a female golden-variety arowana using a combination of deep shotgun sequencing and high-resolution linkage mapping. In addition, we have also generated two draft genome assemblies for the red and green varieties. Phylogenomic analysis supports a sister group relationship between Osteoglossomorpha (bonytongues) and Elopomorpha (eels and relatives), with the two clades together forming a sister group of Clupeocephala which includes all the remaining teleosts. The arowana genome retains the full complement of eight Hox clusters unlike the African butterfly fish (Pantodon buchholzi), another bonytongue fish, which possess only five Hox clusters. Differential gene expression among three varieties provides insights into the genetic basis of colour variation. A potential heterogametic sex chromosome is identified in the female arowana karyotype, suggesting that the sex is determined by a ZW/ZZ sex chromosomal system. The high-quality reference genome of the golden arowana and the draft assemblies of the red and green varieties are valuable resources for understanding the biology, adaptation and behaviour of Asian arowanas

Decentralized Riemannian Algorithm for Nonconvex Minimax Problems

The minimax optimization over Riemannian manifolds (possibly nonconvex constraints) has been actively applied to solve many problems, such as robust dimensionality reduction and deep neural networks with orthogonal weights (Stiefel manifold). Although many optimization algorithms for minimax problems have been developed in the Euclidean setting, it is difficult to convert them into Riemannian cases, and algorithms for nonconvex minimax problems with nonconvex constraints are even rare. On the other hand, to address the big data challenges, decentralized (serverless) training techniques have recently been emerging since they can reduce communications overhead and avoid the bottleneck problem on the server node. Nonetheless, the algorithm for decentralized Riemannian minimax problems has not been studied. In this paper, we study the distributed nonconvex-strongly-concave minimax optimization problem over the Stiefel manifold and propose both deterministic and stochastic minimax methods. The Steifel manifold is a non-convex set. The global function is represented as the finite sum of local functions. For the deterministic setting, we propose DRGDA and prove that our deterministic method achieves a gradient complexity of O( epsilon(-2)) under mild conditions. For the stochastic setting, we propose DRSGDA and prove that our stochastic method achieves a gradient complexity of O( epsilon(-4)). The DRGDA and DRSGDA are the first algorithms for distributed minimax optimization with nonconvex constraints with exact convergence. Extensive experimental results on the Deep Neural Networks (DNNs) training over the Stiefel manifold demonstrate the efficiency of our algorithms

Faster Adaptive Federated Learning

Federated learning has attracted increasing attention with the emergence of distributed data. While extensive federated learning algorithms have been proposed for the non-convex distributed problem, the federated learning in practice still faces numerous challenges, such as the large training iterations to converge since the sizes of models and datasets keep increasing, and the lack of adaptivity by SGD-based model updates. Meanwhile, the study of adaptive methods in federated learning is scarce and existing works either lack a complete theoretical convergence guarantee or have slow sample complexity. In this paper, we propose an efficient adaptive algorithm (i.e., FAFED) based on the momentum-based variance reduced technique in cross-silo FL. We first explore how to design the adaptive algorithm in the FL setting. By providing a counter-example, we prove that a simple combination of FL and adaptive methods could lead to divergence. More importantly, we provide a convergence analysis for our method and prove that our algorithm is the first adaptive FL algorithm to reach the best-known samples O(epsilon(-3)) and O(epsilon(-2)) communication rounds to find an epsilon-stationary point without large batches. The experimental results on the language modeling task and image classification task with heterogeneous data demonstrate the efficiency of our algorithms

A Design Chart for the Analysis of the Maximum Strain of Reinforcement in GRPEs Considering the Arching and Stress History of the Subsoil

Geogrid-reinforced piled embankments (GRPEs) provide an economical and effective way to construct highways and railways on soft soil foundations. This paper proposed a new design method for GRPEs. The method was based on the soil arching and tensioned-membrane effects, the bearing capacity of the subsoil was considered as well. The originality of the proposed method lies in considering the stress history of the subsoil, and different over-consolidation ratios (OCRs) were used in calculating the settlement of subsoil. This design method, initially, established the vertical equilibrium of the unit body between the pile caps immediately above the subsoil. After that, the design charts were produced by solving the overall equilibrium equation from which engineers can intuitively obtain the maximum strain of reinforcement, and the tensile force can be used in the ultimate limit state analyses. The design method was then validated by three case studies, which showed good reliability with the maximum error being less than 18%. Parameter study results indicated that the maximum strain of reinforcement for the under-consolidated soil was 80–120% larger than that for normally consolidated soil and more than four times greater than that for over-consolidated soil

Numerical Study of Bearing Capacity of the Pile-Supported Embankments for the Flexible Floating, Rigid Floating and End-Bearing Piles

Floating pile-supported embankment involves more complex load transfer mechanisms, and there are no clear uniform guidelines available for its design. The concepts of flexible floating, rigid floating and end-bearing pile-supported embankments were proposed in this paper. Based on three typical field cases, their pile-soil interactions and soil arching effect were examined using the three-dimensional finite element method. Due to symmetry, only a half-width embankment model was simulated here. It has been found that the flexible floating piles carry the load mainly relying on the skin friction, but end-bearing piles rely on the pile tip resistance. The rigid floating piles were somehow in between. The earth pressure coefficient (K) in the end-bearing pile-supported embankment reached a maximum of 3.28, greater than Rankine passive values of the earth pressure coefficient (KP), in which the soil arching was fully developed. The K in the embankment with rigid floating pile reached 2.21, where soil arching might be partially formed. At the bottom of the flexible floating pile-supported embankment, the K tended to equal the Rankine active values of the earth pressure coefficient (Ka), and thus soil arching was insignificant. It has also been found that using rigid floating piles might significantly improve the bearing capacity of the embankments and was cost-effective for deep soft soil areas

Numerical Study of Bearing Capacity of the Pile-Supported Embankments for the Flexible Floating, Rigid Floating and End-Bearing Piles

Floating pile-supported embankment involves more complex load transfer mechanisms, and there are no clear uniform guidelines available for its design. The concepts of flexible floating, rigid floating and end-bearing pile-supported embankments were proposed in this paper. Based on three typical field cases, their pile-soil interactions and soil arching effect were examined using the three-dimensional finite element method. Due to symmetry, only a half-width embankment model was simulated here. It has been found that the flexible floating piles carry the load mainly relying on the skin friction, but end-bearing piles rely on the pile tip resistance. The rigid floating piles were somehow in between. The earth pressure coefficient (K) in the end-bearing pile-supported embankment reached a maximum of 3.28, greater than Rankine passive values of the earth pressure coefficient (KP), in which the soil arching was fully developed. The K in the embankment with rigid floating pile reached 2.21, where soil arching might be partially formed. At the bottom of the flexible floating pile-supported embankment, the K tended to equal the Rankine active values of the earth pressure coefficient (Ka), and thus soil arching was insignificant. It has also been found that using rigid floating piles might significantly improve the bearing capacity of the embankments and was cost-effective for deep soft soil areas