A Survey on Deep Multi-modal Learning for Body Language Recognition and Generation

Body language (BL) refers to the non-verbal communication expressed through physical movements, gestures, facial expressions, and postures. It is a form of communication that conveys information, emotions, attitudes, and intentions without the use of spoken or written words. It plays a crucial role in interpersonal interactions and can complement or even override verbal communication. Deep multi-modal learning techniques have shown promise in understanding and analyzing these diverse aspects of BL. The survey emphasizes their applications to BL generation and recognition. Several common BLs are considered i.e., Sign Language (SL), Cued Speech (CS), Co-speech (CoS), and Talking Head (TH), and we have conducted an analysis and established the connections among these four BL for the first time. Their generation and recognition often involve multi-modal approaches. Benchmark datasets for BL research are well collected and organized, along with the evaluation of SOTA methods on these datasets. The survey highlights challenges such as limited labeled data, multi-modal learning, and the need for domain adaptation to generalize models to unseen speakers or languages. Future research directions are presented, including exploring self-supervised learning techniques, integrating contextual information from other modalities, and exploiting large-scale pre-trained multi-modal models. In summary, this survey paper provides a comprehensive understanding of deep multi-modal learning for various BL generations and recognitions for the first time. By analyzing advancements, challenges, and future directions, it serves as a valuable resource for researchers and practitioners in advancing this field. n addition, we maintain a continuously updated paper list for deep multi-modal learning for BL recognition and generation: https://github.com/wentaoL86/awesome-body-language

Private-Library-Oriented Code Generation with Large Language Models

Large language models (LLMs), such as Codex and GPT-4, have recently showcased their remarkable code generation abilities, facilitating a significant boost in coding efficiency. This paper will delve into utilizing LLMs for code generation in private libraries, as they are widely employed in everyday programming. Despite their remarkable capabilities, generating such private APIs poses a formidable conundrum for LLMs, as they inherently lack exposure to these private libraries during pre-training. To address this challenge, we propose a novel framework that emulates the process of programmers writing private code. This framework comprises two modules: APIFinder first retrieves potentially useful APIs from API documentation; and APICoder then leverages these retrieved APIs to generate private code. Specifically, APIFinder employs vector retrieval techniques and allows user involvement in the retrieval process. For APICoder, it can directly utilize off-the-shelf code generation models. To further cultivate explicit proficiency in invoking APIs from prompts, we continuously pre-train a reinforced version of APICoder, named CodeGenAPI. Our goal is to train the above two modules on vast public libraries, enabling generalization to private ones. Meanwhile, we create four private library benchmarks, including TorchDataEval, TorchDataComplexEval, MonkeyEval, and BeatNumEval, and meticulously handcraft test cases for each benchmark to support comprehensive evaluations. Numerous experiments on the four benchmarks consistently affirm the effectiveness of our approach. Furthermore, deeper analysis is also conducted to glean additional insights

Omics-based interpretation of synergism in a soil-derived cellulose-degrading microbial community

Reaching a comprehensive understanding of how nature solves the problem of degrading recalcitrant biomass may eventually allow development of more efficient biorefining processes. Here we interpret genomic and proteomic information generated from a cellulolytic microbial consortium (termed F1RT) enriched from soil. Analyses of reconstructed bacterial draft genomes from all seven uncultured phylotypes in F1RT indicate that its constituent microbes cooperate in both cellulose-degrading and other important metabolic processes. Support for cellulolytic inter-species cooperation came from the discovery of F1RT microbes that encode and express complimentary enzymatic inventories that include both extracellular cellulosomes and secreted free-enzyme systems. Metabolic reconstruction of the seven F1RT phylotypes predicted a wider genomic rationale as to how this particular community functions as well as possible reasons as to why biomass conversion in nature relies on a structured and cooperative microbial community

Identification and characterization of the Remorin gene family in Saccharum and the involvement of ScREM1.5e-1/-2 in SCMV infection on sugarcane

IntroductionRemorins (REMs) are plant-specific membrane-associated proteins that play important roles in plant–pathogen interactions and environmental adaptations. Group I REMs are extensively involved in virus infection. However, little is known about the REM gene family in sugarcane (Saccharum spp. hyrid), the most important sugar and energy crop around world.MethodsComparative genomics were employed to analyze the REM gene family in Saccharum spontaneum. Transcriptomics or RT-qPCR were used to analyze their expression files in different development stages or tissues under different treatments. Yeast two hybrid, bimolecular fluorescence complementation and co-immunoprecipitation assays were applied to investigate the protein interaction.ResultsIn this study, 65 REMs were identified from Saccharum spontaneum genome and classified into six groups based on phylogenetic tree analysis. These REMs contain multiple cis-elements associated with growth, development, hormone and stress response. Expression profiling revealed that among different SsREMs with variable expression levels in different developmental stages or different tissues. A pair of alleles, ScREM1.5e-1/-2, were isolated from the sugarcane cultivar ROC22. ScREM1.5e-1/-2 were highly expressed in leaves, with the former expressed at significantly higher levels than the latter. Their expression was induced by treatment with H2O2, ABA, ethylene, brassinosteroid, SA or MeJA, and varied upon Sugarcane mosaic virus (SCMV) infection. ScREM1.5e-1 was localized to the plasma membrane (PM), while ScREM1.5e-2 was localized to the cytoplasm or nucleus. ScREM1.5e-1/-2 can self-interact and interact with each other, and interact with VPgs from SCMV, Sorghum mosaic virus, or Sugarcane streak mosaic virus. The interactions with VPgs relocated ScREM1.5e-1 from the PM to the cytoplasm.DiscussionThese results reveal the origin, distribution and evolution of the REM gene family in sugarcane and may shed light on engineering sugarcane resistance against sugarcane mosaic pathogens

Droplet-like Fermi surfaces in the anti-ferromagnetic phase of EuFe2_2As2_2, an Fe-pnictide superconductor parent compound

Using angle resolved photoemission it is shown that the low lying electronic states of the iron pnictide parent compound EuFe$_2$As$_2$ are strongly modified in the magnetically ordered, low temperature, orthorhombic state compared to the tetragonal, paramagnetic case above the spin density wave transition temperature. Back-folded bands, reflected in the orthorhombic/ anti-ferromagnetic Brillouin zone boundary hybridize strongly with the non-folded states, leading to the opening of energy gaps. As a direct consequence, the large Fermi surfaces of the tetragonal phase fragment, the low temperature Fermi surface being comprised of small droplets, built up of electron and hole-like sections. These high resolution ARPES data are therefore in keeping with quantum oscillation and optical data from other undoped pnictide parent compounds.Comment: 4 figures, 6 page

Deep functional analysis of synII, a 770-kilobase synthetic yeast chromosome

INTRODUCTION Although much effort has been devoted to studying yeast in the past few decades, our understanding of this model organism is still limited. Rapidly developing DNA synthesis techniques have made a “build-to-understand” approach feasible to reengineer on the genome scale. Here, we report on the completion of a 770-kilobase synthetic yeast chromosome II (synII). SynII was characterized using extensive Trans-Omics tests. Despite considerable sequence alterations, synII is virtually indistinguishable from wild type. However, an up-regulation of translational machinery was observed and can be reversed by restoring the transfer RNA (tRNA) gene copy number. RATIONALE Following the “design-build-test-debug” working loop, synII was successfully designed and constructed in vivo. Extensive Trans-Omics tests were conducted, including phenomics, transcriptomics, proteomics, metabolomics, chromosome segregation, and replication analyses. By both complementation assays and SCRaMbLE (synthetic chromosome rearrangement and modification by loxP -mediated evolution), we targeted and debugged the origin of a growth defect at 37°C in glycerol medium. RESULTS To efficiently construct megabase-long chromosomes, we developed an I- Sce I–mediated strategy, which enables parallel integration of synthetic chromosome arms and reduced the overall integration time by 50% for synII. An I- Sce I site is introduced for generating a double-strand break to promote targeted homologous recombination during mitotic growth. Despite hundreds of modifications introduced, there are still regions sharing substantial sequence similarity that might lead to undesirable meiotic recombinations when intercrossing the two semisynthetic chromosome arm strains. Induction of the I- Sce I–mediated double-strand break is otherwise lethal and thus introduced a strong selective pressure for targeted homologous recombination. Since our strategy is designed to generate a markerless synII and leave the URA3 marker on the wild-type chromosome, we observed a tenfold increase in URA3 -deficient colonies upon I- Sce I induction, meaning that our strategy can greatly bias the crossover events toward the designated regions. By incorporating comprehensive phenotyping approaches at multiple levels, we demonstrated that synII was capable of powering the growth of yeast indistinguishably from wild-type cells (see the figure), showing highly consistent biological processes comparable to the native strain. Meanwhile, we also noticed modest but potentially significant up-regulation of the translational machinery. The main alteration underlying this change in expression is the deletion of 13 tRNA genes. A growth defect was observed in one very specific condition—high temperature (37°C) in medium with glycerol as a carbon source—where colony size was reduced significantly. We targeted and debugged this defect by two distinct approaches. The first approach involved phenotype screening of all intermediate strains followed by a complementation assay with wild-type sequences in the synthetic strain. By doing so, we identified a modification resulting from PCRTag recoding in TSC10 , which is involved in regulation of the yeast high-osmolarity glycerol (HOG) response pathway. After replacement with wild-type TSC10 , the defect was greatly mitigated. The other approach, debugging by SCRaMbLE, showed rearrangements in regions containing HOG regulation genes. Both approaches indicated that the defect is related to HOG response dysregulation. Thus, the phenotypic defect can be pinpointed and debugged through multiple alternative routes in the complex cellular interactome network. CONCLUSION We have demonstrated that synII segregates, replicates, and functions in a highly similar fashion compared with its wild-type counterpart. Furthermore, we believe that the iterative “design-build-test-debug” cycle methodology, established here, will facilitate progression of the Sc2.0 project in the face of the increasing synthetic genome complexity. SynII characterization. ( A ) Cell cycle comparison between synII and BY4741 revealed by the percentage of cells with separated CEN2-GFP dots, metaphase spindles, and anaphase spindles. ( B ) Replication profiling of synII (red) and BY4741 (black) expressed as relative copy number by deep sequencing. ( C ) RNA sequencing analysis revealed that the significant up-regulation of translational machinery in synII is induced by the deletion of tRNA genes in synII. </jats:sec

The oyster genome reveals stress adaptation and complexity of shell formation

The Pacific oyster Crassostrea gigas belongs to one of the most species-rich but genomically poorly explored phyla, the Mollusca. Here we report the sequencing and assembly of the oyster genome using short reads and a fosmid-pooling strategy, along with transcriptomes of development and stress response and the proteome of the shell. The oyster genome is highly polymorphic and rich in repetitive sequences, with some transposable elements still actively shaping variation. Transcriptome studies reveal an extensive set of genes responding to environmental stress. The expansion of genes coding for heat shock protein 70 and inhibitors of apoptosis is probably central to the oyster's adaptation to sessile life in the highly stressful intertidal zone. Our analyses also show that shell formation in molluscs is more complex than currently understood and involves extensive participation of cells and their exosomes. The oyster genome sequence fills a void in our understanding of the Lophotrochozoa. © 2012 Macmillan Publishers Limited. All rights reserved

AC Network-Constrained Peer-to-Peer Electricity Market Model in Low-voltage Power Distribution Networks

The rapid growth of Distributed Energy Resources (DERs) and demand response in modern power grids calls for decentralized electricity trading mechanisms to effectively induce and coordinate responses from massive DERs and load entities. This paper proposes a fully decentralized peer-to-peer (P2P) distribution electricity trading mechanism backboned by a modified version of Alternating Direction Method of Multipliers (ADMM). Comparing with the state-of-the-art P2P electricity markets, the proposed electricity trading mechanism comprehensively incorporates power network constraints formulated by an alternating current power flow model. In the proposed method, nodal voltages are designed as the only public variables shared among participants. Privacy of each participant is thus sufficiently preserved without exposing the bid data to the others. Meanwhile, the proposed P2P method can also ensure the global optimality of social welfare in electricity transactions. Comprehensive case studies are carried out using the IEEE 33-node distribution system to demonstrate the feasibility and efficiency of proposed P2P electricity trading mechanism. The results indicate that the proposed model can efficiently integrates network constraints in the market clearing process while maintaining computational performance, ensuring stable convergence regardless of distribution system congestion. The proposed P2P electricity market can achieve global maximization of social welfare, with a relative difference of no more than 5.0% compared to the conventional centralized market. Even in worst-case scenarios, the relative difference remains within 7.32%. Meanwhile, the method accurately accounts for network losses, guaranteeing the feasibility of trading plans within the physical network