399 research outputs found
SCCA: Shifted Cross Chunk Attention for long contextual semantic expansion
Sparse attention as a efficient method can significantly decrease the
computation cost, but current sparse attention tend to rely on window self
attention which block the global information flow. For this problem, we present
Shifted Cross Chunk Attention (SCCA), using different KV shifting strategy to
extend respective field in each attention layer. Except, we combine Dilated
Attention(DA) and Dilated Neighborhood Attention(DNA) to present Shifted
Dilated Attention(SDA). Both SCCA and SDA can accumulate attention results in
multi head attention to obtain approximate respective field in full attention.
In this paper, we conduct language modeling experiments using different pattern
of SCCA and combination of SCCA and SDA. The proposed shifted cross chunk
attention (SCCA) can effectively extend large language models (LLMs) to longer
context combined with Positional interpolation(PI) and LoRA than current sparse
attention. Notably, SCCA adopts LLaMA2 7B from 4k context to 8k in single V100.
This attention pattern can provide a Plug-and-play fine-tuning method to extend
model context while retaining their original architectures, and is compatible
with most existing techniques.Comment: work in progres
Study of the Impact of the Big Data Era on Accounting and Auditing
Big data revolutionizes accounting and auditing, offering deep insights but
also introducing challenges like data privacy and security. With data from IoT,
social media, and transactions, traditional practices are evolving.
Professionals must adapt to these changes, utilizing AI and machine learning
for efficient data analysis and anomaly detection. Key to overcoming these
challenges are enhanced analytics tools, continuous learning, and industry
collaboration. By addressing these areas, the accounting and auditing fields
can harness big data's potential while ensuring accuracy, transparency, and
integrity in financial reporting. Keywords: Big Data, Accounting, Audit, Data
Privacy, AI, Machine Learning, Transparency.Comment: 4 page
Effects of Hf, B, Cr and Zr alloying on mechanical properties and oxidation resistance of Nb-Si based ultrahigh temperature alloy
Multi-component Nb-Si based ultrahigh temperature alloys were prepared by vacuum non-consumable arc melting. The effects of Hf, B, Zr and Cr alloying on the phase selection, phase stability, both non-equilibrium and equilibrium microstructure, room-temperature fracture toughness, hardness and oxidation resistance at 1250 oC of the alloys have been investigated and estimated systematically. The results show that the addition of B or Cr promotes the formation of hypereutectic structures. The alloying with both Hf and B suppresses the formation of β(Nb,X)5Si3 and promotes the formation of α(Nb,X)5Si3 and γ(Nb,X)5Si3, while the alloying with Cr has no effect on the crystal structures of 5-3 silicides. The room-temperature fracture toughness of the alloys is always degraded by the addition of Cr but almost not influenced by the combined additions of Hf and B. The hardness of 5-3 silicides exhibits a tendency of γ \u3e α \u3e β. The macrohardness of the alloys increases with Cr addition, and it obviously reduces in the presence of Hf after 1450 oC/50 h heat-treatment. The best oxidation-resistant performance has been obtained for the alloy with both B and Cr additions. However, in the presence of B and/or Cr, the oxidation resistance of the alloys has been degraded by further addition of Hf.
Both sizes and amounts of primary γ-(Nb, X)5Si3 increase with Zr contents in the alloy. Both adhesion and compactness of the scales are improved effectively by increase in Zr content. The mass gain and thickness of the scale decrease with increase in Zr contents, indicating that Zr addition can improve the oxidation resistance of the alloys significantly.
Please click Additional Files below to see the full abstract
Multi-Modal Human Authentication Using Silhouettes, Gait and RGB
Whole-body-based human authentication is a promising approach for remote
biometrics scenarios. Current literature focuses on either body recognition
based on RGB images or gait recognition based on body shapes and walking
patterns; both have their advantages and drawbacks. In this work, we propose
Dual-Modal Ensemble (DME), which combines both RGB and silhouette data to
achieve more robust performances for indoor and outdoor whole-body based
recognition. Within DME, we propose GaitPattern, which is inspired by the
double helical gait pattern used in traditional gait analysis. The GaitPattern
contributes to robust identification performance over a large range of viewing
angles. Extensive experimental results on the CASIA-B dataset demonstrate that
the proposed method outperforms state-of-the-art recognition systems. We also
provide experimental results using the newly collected BRIAR dataset
Development and test of a mini-Data Acquisition system for the High-Luminosity LHC upgrade of the ATLAS Monitored Drift Tube detector
New front-end electronics including ASICs and FPGA boards are under
development for the ATLAS Monitored Drift Tube (MDT) detector to handle the
large data rates and harsh environment expected at high-luminosity LHC runs. A
mobile Data Acquisition (miniDAQ) system is designed to perform integration
tests of these front-end electronics. In addition, it will be used for surface
commissioning of 96 small-radius MDT (sMDT) chambers and for integration and
commissioning of new front-end electronics on the present ATLAS MDT chambers.
Details of the miniDAQ hardware and firmware are described in this article. The
miniDAQ system is also used to read out new front-end electronics on an sMDT
prototype chamber using cosmic muons and results obtained are shown.Comment: 10 pages, 12 figure
Triterpenoid saponins in Aralia elata subjected to combined nutrient availability and light quality
Combined light spectra and nitrogen (N) availability may modify contents of triterpenoid saponins (TSs) in leaves of Aralia elata (Miq.) Seem. In this study, A. elata seedlings were raised under light-emitting diode spectra in red- (26.6% red, 59.9% green, and 13.5% blue) and green-colours (12.6% red, 84.6% green, and 2.9% blue) both at a photosynthetic photon flux density of about 77.4 µmol m-2 s-1. N availability was employed at low and high rates of 30 and 90 mg kg-1, respectively. Aralosides-A and -VI did not show any responses to either light or N treatments (ranges of 1.98‒3.75 mg g-1 and 0.21‒1.41 mg g-1, respectively). Compared to the green light, the red light resulted in greater growth but lower foliar N assimilation and aralosides-B (~0.7 mg g-1) and -V concentrations (~16 mg kg-1). The high N availability resulted in greater growth, biomass, foliar chlorophyl and protein concentrations but lower N assimilation and TS concentrations. We conclude that araloside B can be taken as an objective TS harvested in A. elata food-used leaves as a bioactive compound that can be adjusted by light and N manipulations
IDET: Iterative Difference-Enhanced Transformers for High-Quality Change Detection
Change detection (CD) aims to detect change regions within an image pair
captured at different times, playing a significant role for diverse real-world
applications. Nevertheless, most of existing works focus on designing advanced
network architectures to map the feature difference to the final change map
while ignoring the influence of the quality of the feature difference. In this
paper, we study the CD from a new perspective, i.e., how to optimize the
feature difference to highlight changes and suppress unchanged regions, and
propose a novel module denoted as iterative difference-enhanced transformers
(IDET). IDET contains three transformers: two transformers for extracting the
long-range information of the two images and one transformer for enhancing the
feature difference. In contrast to the previous transformers, the third
transformer takes the outputs of the first two transformers to guide the
enhancement of the feature difference iteratively. To achieve more effective
refinement, we further propose the multi-scale IDET-based change detection that
uses multi-scale representations of the images for multiple feature difference
refinements and proposes a coarse-to-fine fusion strategy to combine all
refinements. Our final CD method outperforms seven state-of-the-art methods on
six large-scale datasets under diverse application scenarios, which
demonstrates the importance of feature difference enhancements and the
effectiveness of IDET.Comment: conferenc
A study on the impact of pre-trained model on Just-In-Time defect prediction
Previous researchers conducting Just-In-Time (JIT) defect prediction tasks
have primarily focused on the performance of individual pre-trained models,
without exploring the relationship between different pre-trained models as
backbones. In this study, we build six models: RoBERTaJIT, CodeBERTJIT,
BARTJIT, PLBARTJIT, GPT2JIT, and CodeGPTJIT, each with a distinct pre-trained
model as its backbone. We systematically explore the differences and
connections between these models. Specifically, we investigate the performance
of the models when using Commit code and Commit message as inputs, as well as
the relationship between training efficiency and model distribution among these
six models. Additionally, we conduct an ablation experiment to explore the
sensitivity of each model to inputs. Furthermore, we investigate how the models
perform in zero-shot and few-shot scenarios. Our findings indicate that each
model based on different backbones shows improvements, and when the backbone's
pre-training model is similar, the training resources that need to be consumed
are much more closer. We also observe that Commit code plays a significant role
in defect detection, and different pre-trained models demonstrate better defect
detection ability with a balanced dataset under few-shot scenarios. These
results provide new insights for optimizing JIT defect prediction tasks using
pre-trained models and highlight the factors that require more attention when
constructing such models. Additionally, CodeGPTJIT and GPT2JIT achieved better
performance than DeepJIT and CC2Vec on the two datasets respectively under 2000
training samples. These findings emphasize the effectiveness of
transformer-based pre-trained models in JIT defect prediction tasks, especially
in scenarios with limited training data
GaitContour: Efficient Gait Recognition based on a Contour-Pose Representation
Gait recognition holds the promise to robustly identify subjects based on
walking patterns instead of appearance information. In recent years, this field
has been dominated by learning methods based on two principal input
representations: dense silhouette masks or sparse pose keypoints. In this work,
we propose a novel, point-based Contour-Pose representation, which compactly
expresses both body shape and body parts information. We further propose a
local-to-global architecture, called GaitContour, to leverage this novel
representation and efficiently compute subject embedding in two stages. The
first stage consists of a local transformer that extracts features from five
different body regions. The second stage then aggregates the regional features
to estimate a global human gait representation. Such a design significantly
reduces the complexity of the attention operation and improves efficiency and
performance simultaneously. Through large scale experiments, GaitContour is
shown to perform significantly better than previous point-based methods, while
also being significantly more efficient than silhouette-based methods. On
challenging datasets with significant distractors, GaitContour can even
outperform silhouette-based methods
- …