SCCA: Shifted Cross Chunk Attention for long contextual semantic expansion

Sparse attention as a efficient method can significantly decrease the computation cost, but current sparse attention tend to rely on window self attention which block the global information flow. For this problem, we present Shifted Cross Chunk Attention (SCCA), using different KV shifting strategy to extend respective field in each attention layer. Except, we combine Dilated Attention(DA) and Dilated Neighborhood Attention(DNA) to present Shifted Dilated Attention(SDA). Both SCCA and SDA can accumulate attention results in multi head attention to obtain approximate respective field in full attention. In this paper, we conduct language modeling experiments using different pattern of SCCA and combination of SCCA and SDA. The proposed shifted cross chunk attention (SCCA) can effectively extend large language models (LLMs) to longer context combined with Positional interpolation(PI) and LoRA than current sparse attention. Notably, SCCA adopts LLaMA2 7B from 4k context to 8k in single V100. This attention pattern can provide a Plug-and-play fine-tuning method to extend model context while retaining their original architectures, and is compatible with most existing techniques.Comment: work in progres

Study of the Impact of the Big Data Era on Accounting and Auditing

Big data revolutionizes accounting and auditing, offering deep insights but also introducing challenges like data privacy and security. With data from IoT, social media, and transactions, traditional practices are evolving. Professionals must adapt to these changes, utilizing AI and machine learning for efficient data analysis and anomaly detection. Key to overcoming these challenges are enhanced analytics tools, continuous learning, and industry collaboration. By addressing these areas, the accounting and auditing fields can harness big data's potential while ensuring accuracy, transparency, and integrity in financial reporting. Keywords: Big Data, Accounting, Audit, Data Privacy, AI, Machine Learning, Transparency.Comment: 4 page

Effects of Hf, B, Cr and Zr alloying on mechanical properties and oxidation resistance of Nb-Si based ultrahigh temperature alloy

Multi-component Nb-Si based ultrahigh temperature alloys were prepared by vacuum non-consumable arc melting. The effects of Hf, B, Zr and Cr alloying on the phase selection, phase stability, both non-equilibrium and equilibrium microstructure, room-temperature fracture toughness, hardness and oxidation resistance at 1250 oC of the alloys have been investigated and estimated systematically. The results show that the addition of B or Cr promotes the formation of hypereutectic structures. The alloying with both Hf and B suppresses the formation of β(Nb,X)5Si3 and promotes the formation of α(Nb,X)5Si3 and γ(Nb,X)5Si3, while the alloying with Cr has no effect on the crystal structures of 5-3 silicides. The room-temperature fracture toughness of the alloys is always degraded by the addition of Cr but almost not influenced by the combined additions of Hf and B. The hardness of 5-3 silicides exhibits a tendency of γ \u3e α \u3e β. The macrohardness of the alloys increases with Cr addition, and it obviously reduces in the presence of Hf after 1450 oC/50 h heat-treatment. The best oxidation-resistant performance has been obtained for the alloy with both B and Cr additions. However, in the presence of B and/or Cr, the oxidation resistance of the alloys has been degraded by further addition of Hf. Both sizes and amounts of primary γ-(Nb, X)5Si3 increase with Zr contents in the alloy. Both adhesion and compactness of the scales are improved effectively by increase in Zr content. The mass gain and thickness of the scale decrease with increase in Zr contents, indicating that Zr addition can improve the oxidation resistance of the alloys significantly. Please click Additional Files below to see the full abstract

Multi-Modal Human Authentication Using Silhouettes, Gait and RGB

Whole-body-based human authentication is a promising approach for remote biometrics scenarios. Current literature focuses on either body recognition based on RGB images or gait recognition based on body shapes and walking patterns; both have their advantages and drawbacks. In this work, we propose Dual-Modal Ensemble (DME), which combines both RGB and silhouette data to achieve more robust performances for indoor and outdoor whole-body based recognition. Within DME, we propose GaitPattern, which is inspired by the double helical gait pattern used in traditional gait analysis. The GaitPattern contributes to robust identification performance over a large range of viewing angles. Extensive experimental results on the CASIA-B dataset demonstrate that the proposed method outperforms state-of-the-art recognition systems. We also provide experimental results using the newly collected BRIAR dataset

Triterpenoid saponins in Aralia elata subjected to combined nutrient availability and light quality

Combined light spectra and nitrogen (N) availability may modify contents of triterpenoid saponins (TSs) in leaves of Aralia elata (Miq.) Seem. In this study, A. elata seedlings were raised under light-emitting diode spectra in red- (26.6% red, 59.9% green, and 13.5% blue) and green-colours (12.6% red, 84.6% green, and 2.9% blue) both at a photosynthetic photon flux density of about 77.4 µmol m-2 s-1. N availability was employed at low and high rates of 30 and 90 mg kg-1, respectively. Aralosides-A and -VI did not show any responses to either light or N treatments (ranges of 1.98‒3.75 mg g-1 and 0.21‒1.41 mg g-1, respectively). Compared to the green light, the red light resulted in greater growth but lower foliar N assimilation and aralosides-B (~0.7 mg g-1) and -V concentrations (~16 mg kg-1). The high N availability resulted in greater growth, biomass, foliar chlorophyl and protein concentrations but lower N assimilation and TS concentrations. We conclude that araloside B can be taken as an objective TS harvested in A. elata food-used leaves as a bioactive compound that can be adjusted by light and N manipulations

Development and test of a mini-Data Acquisition system for the High-Luminosity LHC upgrade of the ATLAS Monitored Drift Tube detector

New front-end electronics including ASICs and FPGA boards are under development for the ATLAS Monitored Drift Tube (MDT) detector to handle the large data rates and harsh environment expected at high-luminosity LHC runs. A mobile Data Acquisition (miniDAQ) system is designed to perform integration tests of these front-end electronics. In addition, it will be used for surface commissioning of 96 small-radius MDT (sMDT) chambers and for integration and commissioning of new front-end electronics on the present ATLAS MDT chambers. Details of the miniDAQ hardware and firmware are described in this article. The miniDAQ system is also used to read out new front-end electronics on an sMDT prototype chamber using cosmic muons and results obtained are shown.Comment: 10 pages, 12 figure

IDET: Iterative Difference-Enhanced Transformers for High-Quality Change Detection

Change detection (CD) aims to detect change regions within an image pair captured at different times, playing a significant role for diverse real-world applications. Nevertheless, most of existing works focus on designing advanced network architectures to map the feature difference to the final change map while ignoring the influence of the quality of the feature difference. In this paper, we study the CD from a new perspective, i.e., how to optimize the feature difference to highlight changes and suppress unchanged regions, and propose a novel module denoted as iterative difference-enhanced transformers (IDET). IDET contains three transformers: two transformers for extracting the long-range information of the two images and one transformer for enhancing the feature difference. In contrast to the previous transformers, the third transformer takes the outputs of the first two transformers to guide the enhancement of the feature difference iteratively. To achieve more effective refinement, we further propose the multi-scale IDET-based change detection that uses multi-scale representations of the images for multiple feature difference refinements and proposes a coarse-to-fine fusion strategy to combine all refinements. Our final CD method outperforms seven state-of-the-art methods on six large-scale datasets under diverse application scenarios, which demonstrates the importance of feature difference enhancements and the effectiveness of IDET.Comment: conferenc

A study on the impact of pre-trained model on Just-In-Time defect prediction

Previous researchers conducting Just-In-Time (JIT) defect prediction tasks have primarily focused on the performance of individual pre-trained models, without exploring the relationship between different pre-trained models as backbones. In this study, we build six models: RoBERTaJIT, CodeBERTJIT, BARTJIT, PLBARTJIT, GPT2JIT, and CodeGPTJIT, each with a distinct pre-trained model as its backbone. We systematically explore the differences and connections between these models. Specifically, we investigate the performance of the models when using Commit code and Commit message as inputs, as well as the relationship between training efficiency and model distribution among these six models. Additionally, we conduct an ablation experiment to explore the sensitivity of each model to inputs. Furthermore, we investigate how the models perform in zero-shot and few-shot scenarios. Our findings indicate that each model based on different backbones shows improvements, and when the backbone's pre-training model is similar, the training resources that need to be consumed are much more closer. We also observe that Commit code plays a significant role in defect detection, and different pre-trained models demonstrate better defect detection ability with a balanced dataset under few-shot scenarios. These results provide new insights for optimizing JIT defect prediction tasks using pre-trained models and highlight the factors that require more attention when constructing such models. Additionally, CodeGPTJIT and GPT2JIT achieved better performance than DeepJIT and CC2Vec on the two datasets respectively under 2000 training samples. These findings emphasize the effectiveness of transformer-based pre-trained models in JIT defect prediction tasks, especially in scenarios with limited training data

GaitContour: Efficient Gait Recognition based on a Contour-Pose Representation

Gait recognition holds the promise to robustly identify subjects based on walking patterns instead of appearance information. In recent years, this field has been dominated by learning methods based on two principal input representations: dense silhouette masks or sparse pose keypoints. In this work, we propose a novel, point-based Contour-Pose representation, which compactly expresses both body shape and body parts information. We further propose a local-to-global architecture, called GaitContour, to leverage this novel representation and efficiently compute subject embedding in two stages. The first stage consists of a local transformer that extracts features from five different body regions. The second stage then aggregates the regional features to estimate a global human gait representation. Such a design significantly reduces the complexity of the attention operation and improves efficiency and performance simultaneously. Through large scale experiments, GaitContour is shown to perform significantly better than previous point-based methods, while also being significantly more efficient than silhouette-based methods. On challenging datasets with significant distractors, GaitContour can even outperform silhouette-based methods