A Good Student is Cooperative and Reliable: CNN-Transformer Collaborative Learning for Semantic Segmentation

In this paper, we strive to answer the question "how to collaboratively learn convolutional neural network (CNN)-based and vision transformer (ViT)-based models by selecting and exchanging the reliable knowledge between them for semantic segmentation?" Accordingly, we propose an online knowledge distillation (KD) framework that can simultaneously learn compact yet effective CNN-based and ViT-based models with two key technical breakthroughs to take full advantage of CNNs and ViT while compensating their limitations. Firstly, we propose heterogeneous feature distillation (HFD) to improve students' consistency in low-layer feature space by mimicking heterogeneous features between CNNs and ViT. Secondly, to facilitate the two students to learn reliable knowledge from each other, we propose bidirectional selective distillation (BSD) that can dynamically transfer selective knowledge. This is achieved by 1) region-wise BSD determining the directions of knowledge transferred between the corresponding regions in the feature space and 2) pixel-wise BSD discerning which of the prediction knowledge to be transferred in the logit space. Extensive experiments on three benchmark datasets demonstrate that our proposed framework outperforms the state-of-the-art online distillation methods by a large margin, and shows its efficacy in learning collaboratively between ViT-based and CNN-based models.Comment: ICCV 202

Test-Time Adaptation for Nighttime Color-Thermal Semantic Segmentation

The ability to scene understanding in adverse visual conditions, e.g., nighttime, has sparked active research for RGB-Thermal (RGB-T) semantic segmentation. However, it is essentially hampered by two critical problems: 1) the day-night gap of RGB images is larger than that of thermal images, and 2) the class-wise performance of RGB images at night is not consistently higher or lower than that of thermal images. we propose the first test-time adaptation (TTA) framework, dubbed Night-TTA, to address the problems for nighttime RGBT semantic segmentation without access to the source (daytime) data during adaptation. Our method enjoys three key technical parts. Firstly, as one modality (e.g., RGB) suffers from a larger domain gap than that of the other (e.g., thermal), Imaging Heterogeneity Refinement (IHR) employs an interaction branch on the basis of RGB and thermal branches to prevent cross-modal discrepancy and performance degradation. Then, Class Aware Refinement (CAR) is introduced to obtain reliable ensemble logits based on pixel-level distribution aggregation of the three branches. In addition, we also design a specific learning scheme for our TTA framework, which enables the ensemble logits and three student logits to collaboratively learn to improve the quality of predictions during the testing phase of our Night TTA. Extensive experiments show that our method achieves state-of-the-art (SoTA) performance with a 13.07% boost in mIoU

SEPT: Towards Scalable and Efficient Visual Pre-Training

Recently, the self-supervised pre-training paradigm has shown great potential in leveraging large-scale unlabeled data to improve downstream task performance. However, increasing the scale of unlabeled pre-training data in real-world scenarios requires prohibitive computational costs and faces the challenge of uncurated samples. To address these issues, we build a task-specific self-supervised pre-training framework from a data selection perspective based on a simple hypothesis that pre-training on the unlabeled samples with similar distribution to the target task can bring substantial performance gains. Buttressed by the hypothesis, we propose the first yet novel framework for Scalable and Efficient visual Pre-Training (SEPT) by introducing a retrieval pipeline for data selection. SEPT first leverage a self-supervised pre-trained model to extract the features of the entire unlabeled dataset for retrieval pipeline initialization. Then, for a specific target task, SEPT retrievals the most similar samples from the unlabeled dataset based on feature similarity for each target instance for pre-training. Finally, SEPT pre-trains the target model with the selected unlabeled samples in a self-supervised manner for target data finetuning. By decoupling the scale of pre-training and available upstream data for a target task, SEPT achieves high scalability of the upstream dataset and high efficiency of pre-training, resulting in high model architecture flexibility. Results on various downstream tasks demonstrate that SEPT can achieve competitive or even better performance compared with ImageNet pre-training while reducing the size of training samples by one magnitude without resorting to any extra annotations.Comment: Accepted by AAAI 202

Composting of Municipal Sludge - Riverhead Wastewater Treatment Facility

A significant amount of biosolids is generated by the Riverhead Wastewater Treatment Facility (RHWTF) every year. Although biosolids have the potential to be transformed into compost through the composting process, the usual practice is to dispose them into landfills. Composting helps stabilize the organic matter in the biosolids (Oleszczuk, 2008), and the heat generated during the thermophilic phase also kills pathogens. The organic content of the sludge will be converted into stabilized humic substances through mineralization and, hence, the volume of the sludge is significantly reduced (Gouxue et al., 2001). These composted biosolids, once applied to the soil, can accelerate plant growth, improve soil moisture retention, increase organic matter in the soil, and control erosion of the topsoil (Liang et al., 2003). Since the RHWTF-generated biosolids have a very low carbon to nitrogen (C/N) ratio (8:1), they are usually landfilled. The fly ash (FA) generated from Corner Brook Pulp and Paper (CBPP), however, has a high carbon content; its addition to biosolids could increase the C/N ratio of biosolids. Therefore, the main objective of this study is to investigate the potential application of locally available carbonenriched ash from CBPP in improving the quality of biosolids generated by RHWTF, which serves the City of St. John’s, Mount Pearl, and Paradise

Fish Waste Based Lipopeptide Production and the Potential Application as a Bio-Dispersant for Oil Spill Control

There is a growing acceptance worldwide for the application of dispersants as a marine oil spill response strategy. The development of more effective dispersants with less toxicity and higher biodegradability would be a step forward in improving public acceptance and regulatory approvals for their use. By applying advances in environmental biotechnology, a bio-dispersant agent with a lipopeptide biosurfactant produced by Bacillus subtilis N3-1P as the key component was formulated in this study. The economic feasibility of producing biosurfactant (a high-added-value bioproduct) from fish waste-based peptone as a nutrient substrate was evaluated. Protein hydrolyzate was prepared from cod liver and head wastes obtained from fish processing facilities. Hydrolysis conditions (i.e., time, temperature, pH and enzyme to substrate level) for preparing protein hydrolyzates were optimized by response surface methodology using a factorial design. The critical micelle dilution (CMD) value for biosurfactant produced from the fish liver and head waste generated peptones was 54.72 and 47.59 CMD, respectively. Biosurfactant product generated by fish liver peptone had a low critical micelle concentration of 0.18 g L–1 and could reduce the surface tension of distilled water to 27.9 mN/m. Structure characterization proved that the generated biosurfactant product belongs to the lipopeptide class. An alternative to the key surfactant dioctyl sulfosuccinate sodium (DOSS) used in Corexit 9500 has been proposed based on a binary mixture of lipopeptides and DOSS that exhibited synergistic effects. Using the standard baffled flask test, a high dispersion efficiency of 76.8% for Alaska North Slope oil was achieved at a biodispersant composition of 80/20 (v/v) of lipopeptides/DOSS. The results show that fish waste can be utilized to produce a more effective, environmentally acceptable and cost-efficient biodispersant that can be applied to oil spills in the marine environment

UCP2 Mitigates the Loss of Human Spermatozoa Motility by Promoting mROS Elimination

Background/Aims: To demonstrate the function of uncoupling protein 2 (UCP2) in the regulation of human spermatozoa motility. Methods: Semen samples were collected from donors with either normal spermatozoa motility (normospermia [NS]) or poor spermatozoa motility (asthenospermia [AS]). UCP2 protein in spermatozoawas quantified by Western blotting. The level of mitochondrial reactive oxygen species (mROS) was evaluated by MitoSOX Red. The activity of mitochondrial membrane potential (MMP) in spermatozoa was evaluated by a JC-1 assay and the ATP level was monitored by a luciferin-luciferase assay. Results: UCP2 was expressed in both NS and AS groups, with the former exhibiting a higher level than the latter. Immunofluorescence analysis shows that UCP2 is mainly located at the mid-region of human spermatozoa. The inhibition of UCP2 by a highly selective inhibitor, Genipin, results in not only impaired spermatozoa mobility (P<.05) but also an elevated level of mROS (P<.05), suggesting that UCP2 is involved in the maintenance of the spermatozoa mobility, which probably is achieved by promoting mROS elimination. Furthermore, H2O2 treatment of spermatozoa increases the mROS level coupled with the loss of spermatozoa mobility. Unexpectedly, this treatment also has a positive impact on the expression of UCP2 within a certain range of supplemental H2O2, indicating the moderate mROS level possibly serves as a feedback signal to stimulate the expression of UCP2. Finally, the treatment of spermatozoa by an ROS scavenger, N-acetyl-l-cysteine (NAC),decreases the level of mROS and increases the curvilinear velocity (VCL) of spermatozoa, but the UCP2 level is not affected. Conclusion: These results suggest an UCP2–mROS–motility regulatory system exists for maintaining spermatozoa mobility in humans. In such a system, UCP2 fulfills its function by promoting mROS elimination, and slightly over-produced mROS in turn serves as a signal to stimulates the expression of UCP2. This regulatory system represents a new potential target for the discovery of novel pharmaceuticals for the treatment of patients with low spermatozoa motility

Structural and mechanistic insights into the biosynthesis of CDP-archaeol in membranes

The divergence of archaea, bacteria and eukaryotes was a fundamental step in evolution. One marker of this event is a major difference in membrane lipid chemistry between these kingdoms. Whereas the membranes of bacteria and eukaryotes primarily consist of straight fatty acids ester-bonded to glycerol-3-phosphate, archaeal phospholipids consist of isoprenoid chains ether-bonded to glycerol-1-phosphate. Notably, the mechanisms underlying the biosynthesis of these lipids remain elusive. Here, we report the structure of the CDP-archaeol synthase (CarS) of Aeropyrum pernix (ApCarS) in the CTP- and Mg(2+)-bound state at a resolution of 2.4 Å. The enzyme comprises a transmembrane domain with five helices and cytoplasmic loops that together form a large charged cavity providing a binding site for CTP. Identification of the binding location of CTP and Mg(2+) enabled modeling of the specific lipophilic substrate-binding site, which was supported by site-directed mutagenesis, substrate-binding affinity analyses, and enzyme assays. We propose that archaeol binds within two hydrophobic membrane-embedded grooves formed by the flexible transmembrane helix 5 (TM5), together with TM1 and TM4. Collectively, structural comparisons and analyses, combined with functional studies, not only elucidated the mechanism governing the biosynthesis of phospholipids with ether-bonded isoprenoid chains by CTP transferase, but also provided insights into the evolution of this enzyme superfamily from archaea to bacteria and eukaryotes.Cell Research advance online publication 29 September 2017; doi:10.1038/cr.2017.122

Mortality from gastrointestinal congenital anomalies at 264 hospitals in 74 low-income, middle-income, and high-income countries: a multicentre, international, prospective cohort study

Summary Background Congenital anomalies are the fifth leading cause of mortality in children younger than 5 years globally. Many gastrointestinal congenital anomalies are fatal without timely access to neonatal surgical care, but few studies have been done on these conditions in low-income and middle-income countries (LMICs). We compared outcomes of the seven most common gastrointestinal congenital anomalies in low-income, middle-income, and high-income countries globally, and identified factors associated with mortality. Methods We did a multicentre, international prospective cohort study of patients younger than 16 years, presenting to hospital for the first time with oesophageal atresia, congenital diaphragmatic hernia, intestinal atresia, gastroschisis, exomphalos, anorectal malformation, and Hirschsprung’s disease. Recruitment was of consecutive patients for a minimum of 1 month between October, 2018, and April, 2019. We collected data on patient demographics, clinical status, interventions, and outcomes using the REDCap platform. Patients were followed up for 30 days after primary intervention, or 30 days after admission if they did not receive an intervention. The primary outcome was all-cause, in-hospital mortality for all conditions combined and each condition individually, stratified by country income status. We did a complete case analysis. Findings We included 3849 patients with 3975 study conditions (560 with oesophageal atresia, 448 with congenital diaphragmatic hernia, 681 with intestinal atresia, 453 with gastroschisis, 325 with exomphalos, 991 with anorectal malformation, and 517 with Hirschsprung’s disease) from 264 hospitals (89 in high-income countries, 166 in middleincome countries, and nine in low-income countries) in 74 countries. Of the 3849 patients, 2231 (58·0%) were male. Median gestational age at birth was 38 weeks (IQR 36–39) and median bodyweight at presentation was 2·8 kg (2·3–3·3). Mortality among all patients was 37 (39·8%) of 93 in low-income countries, 583 (20·4%) of 2860 in middle-income countries, and 50 (5·6%) of 896 in high-income countries (p<0·0001 between all country income groups). Gastroschisis had the greatest difference in mortality between country income strata (nine [90·0%] of ten in lowincome countries, 97 [31·9%] of 304 in middle-income countries, and two [1·4%] of 139 in high-income countries; p≤0·0001 between all country income groups). Factors significantly associated with higher mortality for all patients combined included country income status (low-income vs high-income countries, risk ratio 2·78 [95% CI 1·88–4·11], p<0·0001; middle-income vs high-income countries, 2·11 [1·59–2·79], p<0·0001), sepsis at presentation (1·20 [1·04–1·40], p=0·016), higher American Society of Anesthesiologists (ASA) score at primary intervention (ASA 4–5 vs ASA 1–2, 1·82 [1·40–2·35], p<0·0001; ASA 3 vs ASA 1–2, 1·58, [1·30–1·92], p<0·0001]), surgical safety checklist not used (1·39 [1·02–1·90], p=0·035), and ventilation or parenteral nutrition unavailable when needed (ventilation 1·96, [1·41–2·71], p=0·0001; parenteral nutrition 1·35, [1·05–1·74], p=0·018). Administration of parenteral nutrition (0·61, [0·47–0·79], p=0·0002) and use of a peripherally inserted central catheter (0·65 [0·50–0·86], p=0·0024) or percutaneous central line (0·69 [0·48–1·00], p=0·049) were associated with lower mortality. Interpretation Unacceptable differences in mortality exist for gastrointestinal congenital anomalies between lowincome, middle-income, and high-income countries. Improving access to quality neonatal surgical care in LMICs will be vital to achieve Sustainable Development Goal 3.2 of ending preventable deaths in neonates and children younger than 5 years by 2030

The Philosophy of being in the Zhuangzi: Ontological-Existential Reconciliation

This is a study of the philosophy of being in the Zhuangzi as both the ontological “being in itself” that defines the ontological reality and the existential “being in human experience” that constitutes the very human existence. Addressing the problems of mysticism, skepticism and relativism in the Zhuangzi, it raises an alternative Chinese version of ontology that does not necessarily assume the ancient Greek pursuit of the determinate, universal and permanent truth. It argues that without any universal or transcendental Being, the Zhuangzi emphasizes the various manifestations of beings, through which the truth of reality and human existence are simultaneously constituted. The absolute truth of being lies in and only in the conditional constitution of each being itself upon its inviolably firm ontological grounding without opening for dualistic conflict with other beings. Through the ontological-existential approach to the beings, it seeks to overcome the modern Western philosophical separation between epistemology and metaphysics by revealing the intimate connection between human existence and ontological reality. In respecting the beings as the necessary and ceaseless presencing of themselves, the human being eventually reconciles himself in the temporal unfolding of the beings in his life and accomplishes his own existence in residing in the beings as both the mission and destiny of his life