VS-TransGRU: A Novel Transformer-GRU-based Framework Enhanced by Visual-Semantic Fusion for Egocentric Action Anticipation

Egocentric action anticipation is a challenging task that aims to make advanced predictions of future actions from current and historical observations in the first-person view. Most existing methods focus on improving the model architecture and loss function based on the visual input and recurrent neural network to boost the anticipation performance. However, these methods, which merely consider visual information and rely on a single network architecture, gradually reach a performance plateau. In order to fully understand what has been observed and capture the dependencies between current observations and future actions well enough, we propose a novel visual-semantic fusion enhanced and Transformer GRU-based action anticipation framework in this paper. Firstly, high-level semantic information is introduced to improve the performance of action anticipation for the first time. We propose to use the semantic features generated based on the class labels or directly from the visual observations to augment the original visual features. Secondly, an effective visual-semantic fusion module is proposed to make up for the semantic gap and fully utilize the complementarity of different modalities. Thirdly, to take advantage of both the parallel and autoregressive models, we design a Transformer based encoder for long-term sequential modeling and a GRU-based decoder for flexible iteration decoding. Extensive experiments on two large-scale first-person view datasets, i.e., EPIC-Kitchens and EGTEA Gaze+, validate the effectiveness of our proposed method, which achieves new state-of-the-art performance, outperforming previous approaches by a large margin.Comment: 12 pages, 7 figure

Cross-Spatial Pixel Integration and Cross-Stage Feature Fusion Based Transformer Network for Remote Sensing Image Super-Resolution

Remote sensing image super-resolution (RSISR) plays a vital role in enhancing spatial detials and improving the quality of satellite imagery. Recently, Transformer-based models have shown competitive performance in RSISR. To mitigate the quadratic computational complexity resulting from global self-attention, various methods constrain attention to a local window, enhancing its efficiency. Consequently, the receptive fields in a single attention layer are inadequate, leading to insufficient context modeling. Furthermore, while most transform-based approaches reuse shallow features through skip connections, relying solely on these connections treats shallow and deep features equally, impeding the model's ability to characterize them. To address these issues, we propose a novel transformer architecture called Cross-Spatial Pixel Integration and Cross-Stage Feature Fusion Based Transformer Network (SPIFFNet) for RSISR. Our proposed model effectively enhances global cognition and understanding of the entire image, facilitating efficient integration of features cross-stages. The model incorporates cross-spatial pixel integration attention (CSPIA) to introduce contextual information into a local window, while cross-stage feature fusion attention (CSFFA) adaptively fuses features from the previous stage to improve feature expression in line with the requirements of the current stage. We conducted comprehensive experiments on multiple benchmark datasets, demonstrating the superior performance of our proposed SPIFFNet in terms of both quantitative metrics and visual quality when compared to state-of-the-art methods

Modeling EHR with the openEHR approach: an exploratory study in China

Abstract Background The openEHR approach can improve the interoperability of electronic health record (EHR) through two-level modeling. Developing archetypes for the complete EHR dataset is essential for implementing a large-scale interoperable EHR system with the openEHR approach. Although the openEHR approach has been applied in different domains, the feasibility of archetyping a complete EHR dataset in a hospital has not been reported in academic literature, especially in a country where using openEHR is still in its infancy stage, like China. This paper presents a case study of modeling an EHR in China aiming to investigate the feasibility and challenges of archetyping a complete EHR dataset with the openEHR approach. Methods We proposed an archetype modeling method including an iterative process of collecting requirements, normalizing data elements, organizing concepts, searching corresponding archetypes, editing archetypes and reviewing archetypes. Two representative EHR systems from Chinese vendors and the existing Chinese EHR standards have been used as resources to identify the requirements of EHR in China, and a case study of modeling EHR in China has been conducted. Based on the models developed in this case study, we have implemented a clinical data repository (CDR) to verify the feasibility of modeling EHR with archetypes. Results Sixty four archetypes were developed to represent all requirements of a complete EHR dataset. 59 (91%) archetypes could be found in Clinical Knowledge Manager (CKM), of which 35 could be reused directly without change, and 23 required further development including two revisions, two new versions, 18 extensions and one specialization. Meanwhile, 6 (9%) archetypes were newly developed. The legacy data of the EHR system in hospitals could be integrated into the CDR developed with these archetypes successfully. Conclusions The existing archetypes in CKM can faithfully represent most of the EHR requirements in China except customizations for local hospital management. This case study verified the feasibility of modeling EHR with the openEHR approach and identified the fact that the challenges such as localization, tool support, and an agile publishing process still exist for a broader application of the openEHR approach

Fuzzy Graph Learning Regularized Sparse Filtering for Visual Domain Adaptation

Distribution mismatch can be easily found in multi-sensor systems, which may be caused by different shoot angles, weather conditions and so on. Domain adaptation aims to build robust classifiers using the knowledge from a well-labeled source domain, while applied on a related but different target domain. Pseudo labeling is a prevalent technique for class-wise distribution alignment. Therefore, numerous efforts have been spent on alleviating the issue of mislabeling. In this paper, unlike existing selective hard labeling works, we propose a fuzzy labeling based graph learning framework for matching conditional distribution. Specifically, we construct the cross-domain affinity graph by considering the fuzzy label matrix of target samples. In order to solve the problem of representation shrinkage, the paradigm of sparse filtering is introduced. Finally, a unified optimization method based on gradient descent is proposed. Extensive experiments show that our method achieves comparable or superior performance when compared to state-of-the-art works

Verifying the Feasibility of Implementing Semantic Interoperability in Different Countries Based on the OpenEHR Approach: Comparative Study of Acute Coronary Syndrome Registries

BackgroundThe semantic interoperability of health care information has been a critical challenge in medical informatics and has influenced the integration, sharing, analysis, and use of medical big data. International standard organizations have developed standards, approaches, and models to improve and implement semantic interoperability. The openEHR approach—one of the standout semantic interoperability approaches—has been implemented worldwide to improve semantic interoperability based on reused archetypes. ObjectiveThis study aimed to verify the feasibility of implementing semantic interoperability in different countries by comparing the openEHR-based information models of 2 acute coronary syndrome (ACS) registries from China and New Zealand. MethodsA semantic archetype comparison method was proposed to determine the semantics reuse degree of reused archetypes in 2 ACS-related clinical registries from 2 countries. This method involved (1) determining the scope of reused archetypes; (2) identifying corresponding data items within corresponding archetypes; (3) comparing the semantics of corresponding data items; and (4) calculating the number of mappings in corresponding data items and analyzing results. ResultsAmong the related archetypes in the two ACS-related, openEHR-based clinical registries from China and New Zealand, there were 8 pairs of reusable archetypes, which included 89 pairs of corresponding data items and 120 noncorresponding data items. Of the 89 corresponding data item pairs, 87 pairs (98%) were mappable and therefore supported semantic interoperability, and 71 pairs (80%) were labeled as “direct mapping” data items. Of the 120 noncorresponding data items, 114 (95%) data items were generated via archetype evolution, and 6 (5%) data items were generated via archetype localization. ConclusionsThe results of the semantic comparison between the two ACS-related clinical registries prove the feasibility of establishing the semantic interoperability of health care data from different countries based on the openEHR approach. Archetype reuse provides data on the degree to which semantic interoperability exists when using the openEHR approach. Although the openEHR community has effectively promoted archetype reuse and semantic interoperability by providing archetype modeling methods, tools, model repositories, and archetype design patterns, the uncontrolled evolution of archetypes and inconsistent localization have resulted in major challenges for achieving higher levels of semantic interoperability

An openEHR based approach to improve the semantic interoperability of clinical data registry

Abstract Background Clinical data registry is designed to collect and manage information about the practices and outcomes of a patient population for improving the quality and safety of care and facilitating novel researches. Semantic interoperability is a challenge when integrating the data from more than one clinical data registry. The openEHR approach can represent the information and knowledge semantics by multi-level modeling, and it advocates the use of collaborative modeling to facilitate reusing existing archetypes with consistent semantics so as to be a potential solution to improve the semantic interoperability. Methods This paper proposed an openEHR based approach to improve the semantic interoperability of clinical data registry. The approach consists of five steps: clinical data registry meta-information collection, data element definition, archetype modeling, template editing, and implementation. Through collaborative modeling and maximum reusing of existing archetype at the archetype modeling step, the approach can improve semantic interoperability. To verify the feasibility of the approach, this paper conducted a case study of building a Coronary Computed Tomography Angiography (CCTA) registry that can interoperate with an existing Electronic Health Record (EHR) system. Results The CCTA registry includes 183 data elements, which involves 20 archetypes. A total number of 45 CCTA data elements and EHR data elements have semantic overlap. Among them, 38 (84%) CCTA data elements can be found in the 10 reused EHR archetypes. These corresponding clinical data can be collected from the EHR system directly without transformation. The other 7 (16%) CCTA data elements correspond to one coarse-grained EHR data elements, and these clinical data can be collected with mapping rules. The results show that the approach can improve semantic interoperability of clinical data registry. Conclusions Using an openEHR based approach to develop clinical data registry can improve the semantic interoperability. Meanwhile, some challenges for broader semantic interoperability are identified, including domain experts’ involvement, archetype sharing and reusing, and archetype semantic mapping. Collaborative modeling, easy-to-use tools, and semantic relationship establishment are potential solutions for these challenges. This study provides some experience and insight about clinical modeling and clinical data registry development

Work Pattern of Neurology Nurses in a Chinese Hospital: a Time and Motion Study

Aim: To investigate nursing work patterns in a neurology department in a Chinese hospital. Background: Knowledge of nursing work patterns is important for nursing management and work design, and for the evaluation of organizational process change associated with the introduction of an innovation. Methods: A time‐and‐motion method was used to observe 14 registered nurses\u27 (RNs\u27) work patterns in a neurology department in a Chinese hospital. Results: There were 147 hr of observation. Overall, the most time‐consuming activities were verbal communication (28.5%) and documentation (28.3%), followed by indirect care (14.6%) and direct care (14%). Compared to support RNs, charge RNs spent 20% more time on documentation and 11% more time on verbal communication. Two‐thirds of the observed activities had a duration of less than 40 s. Conclusions: Communication, in verbal and written forms, consumed more than half of the nursing time. Conversely, nurses only spent about a quarter of their worktime on preparation for care provision and direct care provision. This reflects the significant role and resource‐consuming nature of communication to provide safe and quality care. Implications for Nursing Management: Communication methods need to be enhanced to improve nursing productivity. This may be achieved through the introduction of more effective nursing documentation methods

YOLO-DCTI: Small Object Detection in Remote Sensing Base on Contextual Transformer Enhancement

Object detection for remote sensing is a fundamental task in image processing of remote sensing; as one of the core components, small or tiny object detection plays an important role. Despite the considerable advancements achieved in small object detection with the integration of CNN and transformer networks, there remains untapped potential for enhancing the extraction and utilization of information associated with small objects. Particularly within transformer structures, this potential arises from the disregard of the complex and the intertwined interplay between spatial context information and channel information during the global modeling of pixel-level information within small objects. As a result, valuable information is prone to being obfuscated and annihilated. To mitigate this limitation, we propose an innovative framework, YOLO-DCTI, that capitalizes on the Contextual Transformer (CoT) framework for the detection of small or tiny objects. Specifically, within CoT, we seamlessly incorporate global residuals and local fusion mechanisms throughout the entire input-to-output pipeline. This integration facilitates a profound investigation into the network’s intrinsic representations at deeper levels and fosters the fusion of spatial contextual attributes with channel characteristics. Moreover, we propose an improved decoupled contextual transformer detection head structure, denoted as DCTI, to effectively resolve the feature conflicts that ensue from the concurrent classification and regression tasks. The experimental results on the Dota, VISDrone, and NWPU VHR-10 datasets show that, on the powerful real-time detection network YOLOv7, the speed and accuracy of tiny targets are better balanced

Network Collaborative Pruning Method for Hyperspectral Image Classification Based on Evolutionary Multi-Task Optimization

Neural network models for hyperspectral images classification are complex and therefore difficult to deploy directly onto mobile platforms. Neural network model compression methods can effectively optimize the storage space and inference time of the model while maintaining the accuracy. Although automated pruning methods can avoid designing pruning rules, they face the problem of search efficiency when optimizing complex networks. In this paper, a network collaborative pruning method is proposed for hyperspectral image classification based on evolutionary multi-task optimization. The proposed method allows classification networks to perform the model pruning task on multiple hyperspectral images simultaneously. Knowledge (the important local sparse structure of the network) is automatically searched and updated by using knowledge transfer between different tasks. The self-adaptive knowledge transfer strategy based on historical information and dormancy mechanism is designed to avoid possible negative transfer and unnecessary consumption of computing resources. The pruned networks can achieve high classification accuracy on hyperspectral data with limited labeled samples. Experiments on multiple hyperspectral images show that the proposed method can effectively realize the compression of the network model and the classification of hyperspectral images

Structure and Activity of the Streptomyces coelicolor A3(2) β‑<i>N</i>‑Acetyl­hexosaminidase Provides Further Insight into GH20 Family Catalysis and Inhibition

β-<i>N</i>-acetyl­hexosaminidases (HEX) are glycosidases that catalyze the glycosidic linkage hydrolysis of <i>gluco</i>- and <i>galacto</i>-configured <i>N</i>-acetyl-β-d-hexosaminides. These enzymes are important in human physiology and are candidates for the biocatalytic production of carbohydrates and glycomimetics. In this study, the three-dimensional structure of the wild-type and catalytically impaired E302Q HEX variant from the soil bacterium Streptomyces coelicolor A3(2) (<i>Sc</i>HEX) were solved in ligand-free forms and in the presence of 6-acetamido-6-deoxy-castanospermine (6-Ac-Cas). The E302Q variant was also trapped as an intermediate with oxazoline bound to the active center. Crystallographic evidence highlights structural variations in the loop 3 environment, suggesting conformational heterogeneity for important active-site residues of this GH20 family member. The enzyme was investigated for its β-<i>N</i>-acetyl­hexosaminidase activity toward chitooligomers and <i>p</i>NP-acetyl <i>gluco</i>- and <i>galacto-</i>configured <i>N</i>-acetyl hexosaminides. Kinetic analyses confirm the β(1–4) glycosidic linkage substrate preference, and HPLC profiles support an exoglycosidase mechanism, where the enzyme cleaves sugars from the nonreducing end of substrates. <i>Sc</i>HEX possesses significant activity toward chitooligosaccharides of varying degrees of polymerization, and the final hydrolytic reaction yielded pure GlcNAc without any byproduct, promising high applicability for the enzymatic production of this highly valued chemical. Thermostability and activation assays further suggest efficient conditions applicable to the enzymatic production of GlcNAc from chitooligomers