A Dual-Modality Emotion Recognition System of EEG and Facial Images and its Application in Educational Scene

With the development of computer science, people's interactions with computers or through computers have become more frequent. Some human-computer interactions or human-to-human interactions that are often seen in daily life: online chat, online banking services, facial recognition functions, etc. Only through text messaging, however, can the effect of information transfer be reduced to around 30% of the original. Communication becomes truly efficient when we can see one other's reactions and feel each other's emotions. This issue is especially noticeable in the educational field. Offline teaching is a classic teaching style in which teachers may determine a student's present emotional state based on their expressions and alter teaching methods accordingly. With the advancement of computers and the impact of Covid-19, an increasing number of schools and educational institutions are exploring employing online or video-based instruction. In such circumstances, it is difficult for teachers to get feedback from students. Therefore, an emotion recognition method is proposed in this thesis that can be used for educational scenarios, which can help teachers quantify the emotional state of students in class and be used to guide teachers in exploring or adjusting teaching methods. Text, physiological signals, gestures, facial photographs, and other data types are commonly used for emotion recognition. Data collection for facial images emotion recognition is particularly convenient and fast among them, although there is a problem that people may subjectively conceal true emotions, resulting in inaccurate recognition results. Emotion recognition based on EEG waves can compensate for this drawback. Taking into account the aforementioned issues, this thesis first employs the SVM-PCA to classify emotions in EEG data, then employs the deep-CNN to classify the emotions of the subject's facial images. Finally, the D-S evidence theory is used for fusing and analyzing the two classification results and obtains the final emotion recognition accuracy of 92%. The specific research content of this thesis is as follows: 1) The background of emotion recognition systems used in teaching scenarios is discussed, as well as the use of various single modality systems for emotion recognition. 2) Detailed analysis of EEG emotion recognition based on SVM. The theory of EEG signal generation, frequency band characteristics, and emotional dimensions is introduced. The EEG signal is first filtered and processed with artifact removal. The processed EEG signal is then used for feature extraction using wavelet transforms. It is finally fed into the proposed SVM-PCA for emotion recognition and the accuracy is 64%. 3) Using the proposed deep-CNN to recognize emotions in facial images. Firstly, the Adaboost algorithm is used to detect and intercept the face area in the image, and the gray level balance is performed on the captured image. Then the preprocessed images are trained and tested using the deep-CNN, and the average accuracy is 88%. 4) Fusion method based on decision-making layer. The data fusion at the decision level is carried out with the results of EEG emotion recognition and facial expression emotion recognition. The final dual-modality emotion recognition results and system accuracy of 92% are obtained using D-S evidence theory. 5) The dual-modality emotion recognition system's data collection approach is designed. Based on the process, the actual data in the educational scene is collected and analyzed. The final accuracy of the dual-modality system is 82%. Teachers can use the emotion recognition results as a guide and reference to improve their teaching efficacy

Video Action Recognition with Attentive Semantic Units

Visual-Language Models (VLMs) have significantly advanced action video recognition. Supervised by the semantics of action labels, recent works adapt the visual branch of VLMs to learn video representations. Despite the effectiveness proved by these works, we believe that the potential of VLMs has yet to be fully harnessed. In light of this, we exploit the semantic units (SU) hiding behind the action labels and leverage their correlations with fine-grained items in frames for more accurate action recognition. SUs are entities extracted from the language descriptions of the entire action set, including body parts, objects, scenes, and motions. To further enhance the alignments between visual contents and the SUs, we introduce a multi-region module (MRA) to the visual branch of the VLM. The MRA allows the perception of region-aware visual features beyond the original global feature. Our method adaptively attends to and selects relevant SUs with visual features of frames. With a cross-modal decoder, the selected SUs serve to decode spatiotemporal video representations. In summary, the SUs as the medium can boost discriminative ability and transferability. Specifically, in fully-supervised learning, our method achieved 87.8% top-1 accuracy on Kinetics-400. In K=2 few-shot experiments, our method surpassed the previous state-of-the-art by +7.1% and +15.0% on HMDB-51 and UCF-101, respectively.Comment: Accepted at ICCV 202

Knowledge-Driven Multi-Agent Reinforcement Learning for Computation Offloading in Cybertwin-Enabled Internet of Vehicles

By offloading computation-intensive tasks of vehicles to roadside units (RSUs), mobile edge computing (MEC) in the Internet of Vehicles (IoV) can relieve the onboard computation burden. However, existing model-based task offloading methods suffer from heavy computational complexity with the increase of vehicles and data-driven methods lack interpretability. To address these challenges, in this paper, we propose a knowledge-driven multi-agent reinforcement learning (KMARL) approach to reduce the latency of task offloading in cybertwin-enabled IoV. Specifically, in the considered scenario, the cybertwin serves as a communication agent for each vehicle to exchange information and make offloading decisions in the virtual space. To reduce the latency of task offloading, a KMARL approach is proposed to select the optimal offloading option for each vehicle, where graph neural networks are employed by leveraging domain knowledge concerning graph-structure communication topology and permutation invariance into neural networks. Numerical results show that our proposed KMARL yields higher rewards and demonstrates improved scalability compared with other methods, benefitting from the integration of domain knowledge

Influence of High Voltage Electrostatic Field (HVEF) on Vigour of Aged Rice (Oryza sativa L.) Seeds

The vigour restoration of aged rice seeds is of great significance in agriculture. This paper studied the biological effects of high voltage electrostatic field (HVEF) on aged rice seeds, including dry seeds and wet seeds soaked in sterile deionized water for 24 hours. The results showed that HVEF slightly affected the vigour of the aged dry rice seeds while the seed vigour and seedling growth of the aged wet rice seeds were significantly improved. The germination rate and germination potentiality also showed moderate improvement after exposure to HVEF with electric intensity less than t 450 kV•m-1. Compared to control, the vigour index of aged wet rice seeds was increased 31.96%. No significant effects of HVEF on dry aged rice seeds were found

PBFormer: Capturing Complex Scene Text Shape with Polynomial Band Transformer

We present PBFormer, an efficient yet powerful scene text detector that unifies the transformer with a novel text shape representation Polynomial Band (PB). The representation has four polynomial curves to fit a text's top, bottom, left, and right sides, which can capture a text with a complex shape by varying polynomial coefficients. PB has appealing features compared with conventional representations: 1) It can model different curvatures with a fixed number of parameters, while polygon-points-based methods need to utilize a different number of points. 2) It can distinguish adjacent or overlapping texts as they have apparent different curve coefficients, while segmentation-based or points-based methods suffer from adhesive spatial positions. PBFormer combines the PB with the transformer, which can directly generate smooth text contours sampled from predicted curves without interpolation. A parameter-free cross-scale pixel attention (CPA) module is employed to highlight the feature map of a suitable scale while suppressing the other feature maps. The simple operation can help detect small-scale texts and is compatible with the one-stage DETR framework, where no postprocessing exists for NMS. Furthermore, PBFormer is trained with a shape-contained loss, which not only enforces the piecewise alignment between the ground truth and the predicted curves but also makes curves' positions and shapes consistent with each other. Without bells and whistles about text pre-training, our method is superior to the previous state-of-the-art text detectors on the arbitrary-shaped text datasets.Comment: 9 pages, 8 figures, accepted by ACM MM 202

Schizandrin A Alleviates LPS-Induced Injury in Human Keratinocyte Cell Hacat Through a MicroRNA-127-Dependent Regulation

Background/Aims: Inflammatory skin diseases are the most common problems in dermatology. Schizandrin A (SchA) has been reported to have anti-inflammatory properties. Herein, we aimed to investigate the protective effects of SchA on lipopolysaccharide (LPS)-induced injury in keratinocyte HaCaT cells. Methods: Inflammation injury in HaCaT cells was induced by LPS treatment. Cell viability, apoptotic cell rate, and apoptosis-related proteins were analyzed by cell counting kit-8 (CCK-8) assay, Annexin V-(fluorescein isothiocyanate (FITC)/ Propidium Iodide (PI) double staining method, and western blot, respectively. The pro-inflammatory factors were analyzed by western blot and quantified by enzyme linked immunosorbent assay (ELISA). Expression of miR-127 in SchA-treated cells was analyzed by qRT-PCR. The effects of SchA on activations of p38MAPK/ERK and JNK pathways were analyzed by western blot. Results: SchA protected HaCaT cells from LPS-induced inflammation damage via promoting cell viability, suppressing apoptosis. Meanwhile, SchA inhibited IL-1β, IL-6, and TNF-α expression. miR-127 expression was up-regulated in LPS-treated HaCaT cells but down-regulated after SchA treatment. Overexpression of miR-127 inhibited cell growth and induced expression of IL-1β, IL-6 and TNF-α. Additionally, miR-127 overexpression impaired the protective effects of SchA, implying miR-127 might be correlated to the anti-inflammation property of SchA and also involved in inactivation of p38MAPK/ERK and JNK pathways by SchA. Conclusion: miR-127 is involved in the protective functions of SchA on LPS-induced inflammation injury in human keratinocyte cell HaCaT, which might inactivates of p38MAPK/ERK and JNK signaling pathways in HaCaT cells

The enteric nervous system deficits in autism spectrum disorder

Gastrointestinal (GI) disorders are common comorbidities in individuals with autism spectrum disorder (ASD), and abnormalities in these issues have been found to be closely related to the severity of core behavioral deficits in autism. The enteric nervous system (ENS) plays a crucial role in regulating various aspects of gut functions, including gastrointestinal motility. Dysfunctional wiring in the ENS not only results in various gastrointestinal issues, but also correlates with an increasing number of central nervous system (CNS) disorders, such as ASD. However, it remains unclear whether the gastrointestinal dysfunctions are a consequence of ASD or if they directly contribute to its pathogenesis. This review focuses on the deficits in the ENS associated with ASD, and highlights several high-risk genes for ASD, which are expressed widely in the gut and implicated in gastrointestinal dysfunction among both animal models and human patients with ASD. Furthermore, we provide a brief overview of environmental factors associated with gastrointestinal tract in individuals with autism. This could offer fresh perspectives on our understanding of ASD

Mutations in TUBB8 and Human Oocyte Meiotic Arrest

BACKGROUND Human reproduction depends on the fusion of a mature oocyte with a sperm cell to form a fertilized egg. The genetic events that lead to the arrest of human oocyte maturation are unknown. METHODS We sequenced the exomes of five members of a four-generation family, three of whom had infertility due to oocyte meiosis I arrest. We performed Sanger sequencing of a candidate gene, TUBB8, in DNA samples from these members, additional family members, and members of 23 other affected families. The expression of TUBB8 and all other β-tubulin isotypes was assessed in human oocytes, early embryos, sperm cells, and several somatic tissues by means of a quantitative reverse- transcriptase–polymerase-chain-reaction assay. We evaluated the effect of the TUBB8 mutations on the assembly of the heterodimer consisting of one α-tubulin polypeptide and one β-tubulin polypeptide (α/β-tubulin heterodimer) in vitro, on microtubule architecture in HeLa cells, on microtubule dynamics in yeast cells, and on spindle assembly in mouse and human oocytes. RESULTS We identified seven mutations in the primate-specific gene TUBB8 that were responsible for oocyte meiosis I arrest in 7 of the 24 families. TUBB8 expression is unique to oocytes and the early embryo, in which this gene accounts for almost all the expressed β-tubulin. The mutations affect chaperone-dependent folding and assembly of the α/β-tubulin heterodimer, disrupt microtubule behavior on expression in cultured cells, alter microtubule dynamics in vivo, and cause catastrophic spindle-assembly defects and maturation arrest on expression in mouse and human oocytes. CONCLUSIONS TUBB8 mutations have dominant-negative effects that disrupt microtubule behavior and oocyte meiotic spindle assembly and maturation, causing female infertility. (Funded by the National Basic Research Program of China and others.