Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentation

In this paper, we study the task of synthetic-to-real domain generalized semantic segmentation, which aims to learn a model that is robust to unseen real-world scenes using only synthetic data. The large domain shift between synthetic and real-world data, including the limited source environmental variations and the large distribution gap between synthetic and real-world data, significantly hinders the model performance on unseen real-world scenes. In this work, we propose the Style-HAllucinated Dual consistEncy learning (SHADE) framework to handle such domain shift. Specifically, SHADE is constructed based on two consistency constraints, Style Consistency (SC) and Retrospection Consistency (RC). SC enriches the source situations and encourages the model to learn consistent representation across style-diversified samples. RC leverages real-world knowledge to prevent the model from overfitting to synthetic data and thus largely keeps the representation consistent between the synthetic and real-world models. Furthermore, we present a novel style hallucination module (SHM) to generate style-diversified samples that are essential to consistency learning. SHM selects basis styles from the source distribution, enabling the model to dynamically generate diverse and realistic samples during training. Experiments show that our SHADE yields significant improvement and outperforms state-of-the-art methods by 5.07% and 8.35% on the average mIoU of three real-world datasets on single- and multi-source settings respectively

Style-Hallucinated Dual Consistency Learning: A Unified Framework for Visual Domain Generalization

Domain shift widely exists in the visual world, while modern deep neural networks commonly suffer from severe performance degradation under domain shift due to the poor generalization ability, which limits the real-world applications. The domain shift mainly lies in the limited source environmental variations and the large distribution gap between source and unseen target data. To this end, we propose a unified framework, Style-HAllucinated Dual consistEncy learning (SHADE), to handle such domain shift in various visual tasks. Specifically, SHADE is constructed based on two consistency constraints, Style Consistency (SC) and Retrospection Consistency (RC). SC enriches the source situations and encourages the model to learn consistent representation across style-diversified samples. RC leverages general visual knowledge to prevent the model from overfitting to source data and thus largely keeps the representation consistent between the source and general visual models. Furthermore, we present a novel style hallucination module (SHM) to generate style-diversified samples that are essential to consistency learning. SHM selects basis styles from the source distribution, enabling the model to dynamically generate diverse and realistic samples during training. Extensive experiments demonstrate that our versatile SHADE can significantly enhance the generalization in various visual recognition tasks, including image classification, semantic segmentation and object detection, with different models, i.e., ConvNets and Transformer.Comment: Accepted by IJCV. Journal extension of arXiv:2204.02548. Code is available at https://github.com/HeliosZhao/SHADE-VisualD

TTSVD: an efficient sparse decision making model with two-way trust recommendation in the AI enabled IoT systems

The convergence of AI and IoT enables data to be quickly explored and turned into vital decisions, and however, there are still some challenging issues to be further addressed. For example, lacking of enough data in AI-based decision making (so called Sparse Decision Making, SDM) will decrease the efficiency dramatically, or even disable the intelligent IoT networks. Taking the intelligent IoT networks as the network infrastructure, the recommendation systems have been facing such SDM problems. A naive solution is to introduce so-called trust information. However, trust information also maybe face the difficulty of sparse trust evidence (a.k.a sparse trust problem). In our work, an accurate sparse decision making model with two-way trust recommendation in the AI enabled IoT systems is proposed by us, named TT-SVD. Our model incorporates both trust information and rating information more completely, which can efficiently alleviate the above mentioned sparse trust problem and therefore be able to solve the cold start and data sparsity problems. Specifically, we first consider the two-fold trust influences from both trustees and trustors, which can be represented by a factor called trust propensity. To this end, we propose a dual model, including the trustor model (TrustorSVD) and a trustee model (TrusteeSVD) based on an existing rating-only recommendation model called SVD++, which are integrated by the weighted average and yield the final model, TT-SVD. The experimental results show that our model outperforms the state of the art including SVD and TrustSVD in both the ”all users” and ”cold start users” cases, and the accuracy improvement can reach a maximum of 29%. Complexity analysis shows that our model is equally suitable for the case of large sparse datasets. In a word, our model can effectively solve the sparse decision problem by introducing the two-way trust recommendation, and hence improve the efficiency of the intelligent recommendation systems

Predicting Continuous Locomotion Modes via Multidimensional Feature Learning from sEMG

Walking-assistive devices require adaptive control methods to ensure smooth transitions between various modes of locomotion. For this purpose, detecting human locomotion modes (e.g., level walking or stair ascent) in advance is crucial for improving the intelligence and transparency of such robotic systems. This study proposes Deep-STF, a unified end-to-end deep learning model designed for integrated feature extraction in spatial, temporal, and frequency dimensions from surface electromyography (sEMG) signals. Our model enables accurate and robust continuous prediction of nine locomotion modes and 15 transitions at varying prediction time intervals, ranging from 100 to 500 ms. In addition, we introduced the concept of 'stable prediction time' as a distinct metric to quantify prediction efficiency. This term refers to the duration during which consistent and accurate predictions of mode transitions are made, measured from the time of the fifth correct prediction to the occurrence of the critical event leading to the task transition. This distinction between stable prediction time and prediction time is vital as it underscores our focus on the precision and reliability of mode transition predictions. Experimental results showcased Deep-STP's cutting-edge prediction performance across diverse locomotion modes and transitions, relying solely on sEMG data. When forecasting 100 ms ahead, Deep-STF surpassed CNN and other machine learning techniques, achieving an outstanding average prediction accuracy of 96.48%. Even with an extended 500 ms prediction horizon, accuracy only marginally decreased to 93.00%. The averaged stable prediction times for detecting next upcoming transitions spanned from 28.15 to 372.21 ms across the 100-500 ms time advances.Comment: 10 pages,7 figure

Metagenomic Sequencing Identifies Highly Diverse Assemblages of Dinoflagellate Cysts in Sediments From Ships\u27 Ballast Tanks

Ships\u27 ballast tanks have long been known as vectors for the introduction of organisms. We applied next-generation sequencing to detect dinoflagellates (mainly as cysts) in 32 ballast tank sediments collected during 2001-2003 from ships entering the Great Lakes or Chesapeake Bay and subsequently archived. Seventy-three dinoflagellates were fully identified to species level by this metagenomic approach and single-cell polymerase chain reaction (PCR)-based sequencing, including 19 toxic species, 36 harmful algal bloom (HAB) forming species, 22 previously unreported as producing cysts, and 55 reported from ballast tank sediments for the first time (including 13 freshwater species), plus 545 operational taxonomic units (OTUs) not fully identified due to a lack of reference sequences, indicating tank sediments are repositories of many previously undocumented taxa. Analyses indicated great heterogeneity of species composition among samples from different sources. Light and scanning electron microscopy and single-cell PCR sequencing supported and confirmed results of the metagenomic approach. This study increases the number of fully identified dinoflagellate species from ballast tank sediments to 142 (\u3e 50% increase). From the perspective of ballast water management, the high diversity and spatiotemporal heterogeneity of dinoflagellates in ballast tanks argues for continuing research and stringent adherence to procedures intended to prevent unintended introduction of non-indigenous toxic and HAB-forming species

Gait Cycle-Inspired Learning Strategy for Continuous Prediction of Knee Joint Trajectory from sEMG

Predicting lower limb motion intent is vital for controlling exoskeleton robots and prosthetic limbs. Surface electromyography (sEMG) attracts increasing attention in recent years as it enables ahead-of-time prediction of motion intentions before actual movement. However, the estimation performance of human joint trajectory remains a challenging problem due to the inter- and intra-subject variations. The former is related to physiological differences (such as height and weight) and preferred walking patterns of individuals, while the latter is mainly caused by irregular and gait-irrelevant muscle activity. This paper proposes a model integrating two gait cycle-inspired learning strategies to mitigate the challenge for predicting human knee joint trajectory. The first strategy is to decouple knee joint angles into motion patterns and amplitudes former exhibit low variability while latter show high variability among individuals. By learning through separate network entities, the model manages to capture both the common and personalized gait features. In the second, muscle principal activation masks are extracted from gait cycles in a prolonged walk. These masks are used to filter out components unrelated to walking from raw sEMG and provide auxiliary guidance to capture more gait-related features. Experimental results indicate that our model could predict knee angles with the average root mean square error (RMSE) of 3.03(0.49) degrees and 50ms ahead of time. To our knowledge this is the best performance in relevant literatures that has been reported, with reduced RMSE by at least 9.5%

Ou: Automating the Parallelization of Zero-Knowledge Protocols

A zero-knowledge proof (ZKP) is a powerful cryptographic primitive used in many decentralized or privacy-focused applications. However, the high overhead of ZKPs can restrict their practical applicability. We design a programming language, Ou, aimed at easing the programmer\u27s burden when writing efficient ZKPs, and a compiler framework, Lian, that automates the analysis and distribution of statements to a computing cluster. Lian uses programming language semantics, formal methods, and combinatorial optimization to automatically partition an Ou program into efficiently sized chunks for parallel ZK-proving and/or verification. We contribute: • A front-end language where users can write proof statements as imperative programs in a familiar syntax; • A compiler architecture and implementation that automatically analyzes the program and compiles it into an optimized IR that can be lifted to a variety of ZKP constructions; and • A cutting algorithm, based on Pseudo-Boolean optimization and Integer Linear Programming, that reorders instructions and then partitions the program into efficiently sized chunks for parallel evaluation and efficient state reconciliation

Testing Electron-phonon Coupling for the Superconductivity in Kagome Metal CsV3Sb5\rm{CsV_3Sb_5}

In crystalline materials, electron-phonon coupling (EPC) is a ubiquitous many-body interaction that drives conventional Bardeen-Cooper-Schrieffer superconductivity. Recently, in a new kagome metal $\rm{CsV_3Sb_5}$, superconductivity that possibly intertwines with time-reversal and spatial symmetry-breaking orders is observed. Density functional theory calculations predicted weak EPC strength,$\lambda$, supporting an unconventional pairing mechanism in $\rm{CsV_3Sb_5}$. However, experimental determination of $\lambda$ is still missing, hindering a microscopic understanding of the intertwined ground state of $\rm{CsV_3Sb_5}$. Here, using 7-eV laser-based angle-resolved photoemission spectroscopy and Eliashberg function analysis, we determine an intermediate $\lambda$=0.45~0.6 at T=6 K for both Sb 5p and V 3d electronic bands, which can support a conventional superconducting transition temperature on the same magnitude of experimental value in $\rm{CsV_3Sb_5}$. Remarkably, the EPC on the V 3d-band enhances to $\lambda$~0.75 as the superconducting transition temperature elevated to 4.4 K in $\rm{Cs(V_{0.93}Nb_{0.07})_3Sb_5}$. Our results provide an important clue to understand the pairing mechanism in the Kagome superconductor $\rm{CsV_3Sb_5}$.Comment: To appear in Nature Communication

Robust estimation of bacterial cell count from optical density

Optical density (OD) is widely used to estimate the density of cells in liquid culture, but cannot be compared between instruments without a standardized calibration protocol and is challenging to relate to actual cell count. We address this with an interlaboratory study comparing three simple, low-cost, and highly accessible OD calibration protocols across 244 laboratories, applied to eight strains of constitutive GFP-expressing E. coli. Based on our results, we recommend calibrating OD to estimated cell count using serial dilution of silica microspheres, which produces highly precise calibration (95.5% of residuals <1.2-fold), is easily assessed for quality control, also assesses instrument effective linear range, and can be combined with fluorescence calibration to obtain units of Molecules of Equivalent Fluorescein (MEFL) per cell, allowing direct comparison and data fusion with flow cytometry measurements: in our study, fluorescence per cell measurements showed only a 1.07-fold mean difference between plate reader and flow cytometry data

A Few-Shot Learning-Based EEG and Stage Transition Sequence Generator for Improving Sleep Staging Performance

In this study, generative adversarial networks named SleepGAN are proposed to expand the training set for automatic sleep stage classification tasks by generating both electroencephalogram (EEG) epochs and sequence relationships of sleep stages. In order to reach high accuracy, most existing classification methods require substantial amounts of training data, but obtaining such quantities of real EEG epochs is expensive and time-consuming. We introduce few-shot learning, which is a method of training a GAN using a very small set of training data. This paper presents progressive Wasserstein divergence generative adversarial networks (GANs) and a relational memory generator to generate EEG epochs and stage transition sequences, respectively. For the evaluation of our generated data, we use single-channel EEGs from the public dataset Sleep-EDF. The addition of our augmented data and sequence to the training set was shown to improve the performance of the classification model. The accuracy of the model increased by approximately 1% after incorporating generated EEG epochs. Adding both the augmented data and sequence to the training set resulted in a further increase of 3%, from the original accuracy of 79.40% to 83.06%. The result proves that SleepGAN is a set of GANs capable of generating realistic EEG epochs and transition sequences under the condition of insufficient training data and can be used to enlarge the training dataset and improve the performance of sleep stage classification models in clinical practice