Layer-Wise Cross-View Decoding for Sequence-to-Sequence Learning

In sequence-to-sequence learning, the decoder relies on the attention mechanism to efficiently extract information from the encoder. While it is common practice to draw information from only the last encoder layer, recent work has proposed to use representations from different encoder layers for diversified levels of information. Nonetheless, the decoder still obtains only a single view of the source sequences, which might lead to insufficient training of the encoder layer stack due to the hierarchy bypassing problem. In this work, we propose layer-wise cross-view decoding, where for each decoder layer, together with the representations from the last encoder layer, which serve as a global view, those from other encoder layers are supplemented for a stereoscopic view of the source sequences. Systematic experiments show that we successfully address the hierarchy bypassing problem and substantially improve the performance of sequence-to-sequence learning with deep representations on diverse tasks.Comment: 9 pages, 6 figure

OpenDigger: Data Mining and Information Service System for Open Collaboration Digital Ecosystem

The widespread development and adoption of open-source software have built an ecosystem for open development and collaboration. In this ecosystem, individuals and organizations collaborate to create high-quality software that can be used by everyone. Social collaboration platforms like GitHub have further facilitated large-scale, distributed, and fine-grained code collaboration and technical interactions. Countless developers contribute code, review code, report bugs, and propose new features on these platforms every day, generating a massive amount of valuable behavioral data from the open collaboration process. This paper presents the design and implementation of OpenDigger, a comprehensive data mining and information service system for open collaboration in the digital ecosystem. The goal is to build a data infrastructure for the open-source domain and promote the continuous development of the open-source ecosystem. The metrics and analysis models in the OpenDigger system can mine various knowledge from the macro to micro levels in the open-source digital ecosystem. Through a unified information service interface, OpenDigger provides various open-source information services to different user groups, including governments, enterprises, foundations, and individuals. As a novel information service system in the open-source ecosystem, this paper demonstrates the effectiveness of the metrics and models in OpenDigger through several real-world scenarios, including products, tools, applications, and courses. It showcases the significant and diverse practical applications of the metrics and models in both algorithmic and business aspects.Comment: in Chinese languag

OpenPerf: A Benchmarking Framework for the Sustainable Development of the Open-Source Ecosystem

Benchmarking involves designing scientific test methods, tools, and frameworks to quantitatively and comparably assess specific performance indicators of certain test subjects. With the development of artificial intelligence, AI benchmarking datasets such as ImageNet and DataPerf have gradually become consensus standards in both academic and industrial fields. However, constructing a benchmarking framework remains a significant challenge in the open-source domain due to the diverse range of data types, the wide array of research issues, and the intricate nature of collaboration networks. This paper introduces OpenPerf, a benchmarking framework designed for the sustainable development of the open-source ecosystem. This framework defines 9 task benchmarking tasks in the open-source research, encompassing 3 data types: time series, text, and graphics, and addresses 6 research problems including regression, classification, recommendation, ranking, network building, and anomaly detection. Based on the above tasks, we implemented 3 data science task benchmarks, 2 index-based benchmarks, and 1 standard benchmark. Notably, the index-based benchmarks have been adopted by the China Electronics Standardization Institute as evaluation criteria for open-source community governance. Additionally, we have developed a comprehensive toolkit for OpenPerf, which not only offers robust data management, tool integration, and user interface capabilities but also adopts a Benchmarking-as-a-Service (BaaS) model to serve academic institutions, industries, and foundations. Through its application in renowned companies and institutions such as Alibaba, Ant Group, and East China Normal University, we have validated OpenPerf's pivotal role in the healthy evolution of the open-source ecosystem

VNI-Net: Vector Neurons-based Rotation-Invariant Descriptor for LiDAR Place Recognition

LiDAR-based place recognition plays a crucial role in Simultaneous Localization and Mapping (SLAM) and LiDAR localization. Despite the emergence of various deep learning-based and hand-crafting-based methods, rotation-induced place recognition failure remains a critical challenge. Existing studies address this limitation through specific training strategies or network structures. However, the former does not produce satisfactory results, while the latter focuses mainly on the reduced problem of SO(2) rotation invariance. Methods targeting SO(3) rotation invariance suffer from limitations in discrimination capability. In this paper, we propose a new method that employs Vector Neurons Network (VNN) to achieve SO(3) rotation invariance. We first extract rotation-equivariant features from neighboring points and map low-dimensional features to a high-dimensional space through VNN. Afterwards, we calculate the Euclidean and Cosine distance in the rotation-equivariant feature space as rotation-invariant feature descriptors. Finally, we aggregate the features using GeM pooling to obtain global descriptors. To address the significant information loss when formulating rotation-invariant descriptors, we propose computing distances between features at different layers within the Euclidean space neighborhood. This greatly improves the discriminability of the point cloud descriptors while ensuring computational efficiency. Experimental results on public datasets show that our approach significantly outperforms other baseline methods implementing rotation invariance, while achieving comparable results with current state-of-the-art place recognition methods that do not consider rotation issues

Learning Sequence Descriptor based on Spatiotemporal Attention for Visual Place Recognition

Sequence-based visual place recognition (sVPR) aims to match frame sequences with frames stored in a reference map for localization. Existing methods include sequence matching and sequence descriptor-based retrieval. The former is based on the assumption of constant velocity, which is difficult to hold in real scenarios and does not get rid of the intrinsic single frame descriptor mismatch. The latter solves this problem by extracting a descriptor for the whole sequence, but current sequence descriptors are only constructed by feature aggregation of multi-frames, with no temporal information interaction. In this paper, we propose a sequential descriptor extraction method to fuse spatiotemporal information effectively and generate discriminative descriptors. Specifically, similar features on the same frame focu on each other and learn space structure, and the same local regions of different frames learn local feature changes over time. And we use sliding windows to control the temporal self-attention range and adpot relative position encoding to construct the positional relationships between different features, which allows our descriptor to capture the inherent dynamics in the frame sequence and local feature motion

Unbalanced Oxidant-Antioxidant Status: A Potential Therapeutic Target for Coronary Chronic Total Occlusion in Very Old Patients

Unbalanced oxidant and antioxidant status played an important role in myocardial infarction. The present study was a clinical trial combined preclinically with targeted agent against cardiovascular injuries and ischemia in vivo model. We tried to confirm the association of unbalanced oxidant and antioxidant status with coronary chronic total occlusion (CTO) in 399 very old patients (80~89 years) and investigated the potential therapeutic value of purified polysaccharide from endothelium corneum gigeriae galli (PECGGp). We analyzed levels of circulating superoxide dismutase 3 (SOD3), nitric oxide (NO), endothelial nitric oxide synthase (eNOS), and malondialdehyde (MDA) in very old patients with coronary CTO. Levels of SOD3, NO, eNOS, and MDA in the cardiac tissue were measured in myocardial infarction rats. Levels of SOD3, eNOS, and NO were lowered (p<0.001) and levels of MDA were increased (p<0.001). PECGGp treatment increased levels of SOD3, eNOS, and NO (p<0.01) in cardiac tissue, while decreasing levels of MDA (p<0.01). PECGGp may suppress unbalanced oxidant and antioxidant status in infarcted myocardium by inhibiting levels of MDA and elevating NO, eNOS, and SOD3 levels. PECGGp could be considered as a potential therapeutic agent for coronary CTO in very old patients

Endoscopic rhizotomy for chronic lumbar zygapophysial joint pain.

BACKGROUND: Chronic lumbar zygapophysial joint pain is a common cause of chronic low back pain. Percutaneous radiofrequency ablation (RFA) is one of the effective management options; however, the results from the traditional RFA need to be improved in certain cases. The aim of this study is to investigate the effect of percutaneous radiofrequency ablation under endoscopic guidance (ERFA) for chronic low back pain secondary to facet joint arthritis. METHODS: This is a prospective study enrolled 60 patients. The cases were randomized into two groups: 30 patients in the control group underwent traditional percutaneous radiofrequency ablation, others underwent ERFA. The lumbar visual analog scale (VAS), MacNab score, and postoperative complications were used to evaluate the outcomes. All outcome assessments were performed at postoperative 1 day, 1 month, 3 months, 6 months, and 12 months. RESULTS: There was no difference between the two groups in preoperative VAS (P \u3e 0.05). VAS scores, except the postoperative first day, in all other postoperative time points were significantly lower than preoperative values each in both groups (P \u3c 0.05). There was no significant difference between the two groups in VAS at 1 day, 1 month, and 3 months after surgery (P \u3e 0.05). However, the EFRA demonstrated significant benefits at the time points of 3 months and 6 months (P \u3e 0.05). The MacNab scores of 1-year follow-up in the ERFA group were higher than that in the control group (P \u3c 0.05). The incidence of complications in the ERFA group was significantly less than that in the control group (P \u3c 0.05). CONCLUSIONS: ERFA may achieve more accurate and definite denervation on the nerves, which leads to longer lasting pain relief

Low temperature fabrication of hydrangea-like NiCo2S4 as electrode materials for high performance supercapacitors

Hydrangea-like NiCo2S4 as electrode materials for high performance supercapacitors was synthesized by using a facile low temperature (90 °C) two-step hydrothermal technique without surfactant or template. The special hydrangea-like structure and large specific surface area (74.8 m2/g) provided plenty of electro active sites which were beneficial to superior pseudocapacitive performance of NiCo2S4. The supercapacitors performance of NiCo2S4 was investigated by a three-electrode system. NiCo2S4 exhibited high specific capacitance with 1475 F g−1 at a current density of 3 A g−1, and a fairly high rate capacity with 1152 F g−1 at 20 A g−1. These results indicate that low temperature hydrothermal is a very promising method to prepare electrode materials for supercapacitors

Suspension and Measurement of Graphene and Bi2Se3 Atomic Membranes

Coupling high quality, suspended atomic membranes to specialized electrodes enables investigation of many novel phenomena, such as spin or Cooper pair transport in these two dimensional systems. However, many electrode materials are not stable in acids that are used to dissolve underlying substrates. Here we present a versatile and powerful multi-level lithographical technique to suspend atomic membranes, which can be applied to the vast majority of substrate, membrane and electrode materials. Using this technique, we fabricated suspended graphene devices with Al electrodes and mobility of 5500 cm^2/Vs. We also demonstrate, for the first time, fabrication and measurement of a free-standing thin Bi2Se3 membrane, which has low contact resistance to electrodes and a mobility of >~500 cm^2/Vs