Less is More: Focus Attention for Efficient DETR

DETR-like models have significantly boosted the performance of detectors and even outperformed classical convolutional models. However, all tokens are treated equally without discrimination brings a redundant computational burden in the traditional encoder structure. The recent sparsification strategies exploit a subset of informative tokens to reduce attention complexity maintaining performance through the sparse encoder. But these methods tend to rely on unreliable model statistics. Moreover, simply reducing the token population hinders the detection performance to a large extent, limiting the application of these sparse models. We propose Focus-DETR, which focuses attention on more informative tokens for a better trade-off between computation efficiency and model accuracy. Specifically, we reconstruct the encoder with dual attention, which includes a token scoring mechanism that considers both localization and category semantic information of the objects from multi-scale feature maps. We efficiently abandon the background queries and enhance the semantic interaction of the fine-grained object queries based on the scores. Compared with the state-of-the-art sparse DETR-like detectors under the same setting, our Focus-DETR gets comparable complexity while achieving 50.4AP (+2.2) on COCO. The code is available at https://github.com/huawei-noah/noah-research/tree/master/Focus-DETR and https://gitee.com/mindspore/models/tree/master/research/cv/Focus-DETR.Comment: 8 pages, 6 figures, accepted to ICCV202

Protein kinase CK2α is overexpressed in colorectal cancer and modulates cell proliferation and invasion via regulating EMT-related genes

<p>Abstract</p> <p>Background</p> <p>Protein kinase CK2 is a highly conserved, ubiquitous protein serine/threonine kinase that phosphorylates many substrates and has a global role in numerous biological and pathological processes. Overexpression of the protein kinase CK2α subunit (CK2α) has been associated with the malignant transformation of several tissues, with not nearly as much focus on the role of CK2α in colorectal cancer (CRC). The aims of this study are to investigate the function and regulatory mechanism of CK2α in CRC development.</p> <p>Methods</p> <p>Expression levels of CK2α were analyzed in 144 patients (104 with CRC and 40 with colorectal adenoma) by immunohistochemistry. Proliferation, senescence, motility and invasion assays as well as immunofluorescence staining and western blots were performed to assess the effect of CK2α in CRC.</p> <p>Results</p> <p>The immunohistochemical expression of nuclear CK2α was stronger in tumor tissues than in adenomas and normal colorectal tissues. Suppression of CK2α by small-interfering RNA or the CK2α activity inhibitor emodin inhibited proliferation of CRC cells, caused G0/G1 phase arrest, induced cell senescence, elevated the expression of p53/p21 and decreased the expression of C-myc. We also found that knockdown of CK2α suppressed cell motility and invasion. Significantly, CK2α inhibition resulted in β-catenin transactivation, decreased the expression levels of vimentin and the transcription factors snail1 and smad2/3, and increased the expression of E-cadherin, suggesting that CK2α regulates the epithelial-mesenchymal transition (EMT) process in cancer cells.</p> <p>Conclusions</p> <p>Our results indicate that CK2α plays an essential role in the development of CRC, and inhibition of CK2α may serve as a promising therapeutic strategy for human CRC.</p

More complex encoder is not all you need

U-Net and its variants have been widely used in medical image segmentation. However, most current U-Net variants confine their improvement strategies to building more complex encoder, while leaving the decoder unchanged or adopting a simple symmetric structure. These approaches overlook the true functionality of the decoder: receiving low-resolution feature maps from the encoder and restoring feature map resolution and lost information through upsampling. As a result, the decoder, especially its upsampling component, plays a crucial role in enhancing segmentation outcomes. However, in 3D medical image segmentation, the commonly used transposed convolution can result in visual artifacts. This issue stems from the absence of direct relationship between adjacent pixels in the output feature map. Furthermore, plain encoder has already possessed sufficient feature extraction capability because downsampling operation leads to the gradual expansion of the receptive field, but the loss of information during downsampling process is unignorable. To address the gap in relevant research, we extend our focus beyond the encoder and introduce neU-Net (i.e., not complex encoder U-Net), which incorporates a novel Sub-pixel Convolution for upsampling to construct a powerful decoder. Additionally, we introduce multi-scale wavelet inputs module on the encoder side to provide additional information. Our model design achieves excellent results, surpassing other state-of-the-art methods on both the Synapse and ACDC datasets

Microchannel structure of ceramic membranes for oxygen separation

Microchanneled ceramic membranes have demonstrated superior performance in oxygen separation from air over conventional membranes. In this study, the contributions of the microchannel structure to the superior performance were investigated. Compared with supported membranes, the microchanneled membranes provide fast pathways within the channels for gas diffusion as compared to the tortuous interconnection of pore channels in the supported membranes. The walls of the numerous channels provide a large surface for facilitating oxygen dissociation, which was confirmed by varying the channel wall surface using mesh templates with different aperture sizes. In summary, the microchannel structure facilitates gas diffusion and provides a large membrane active surface, resulting in high performance in oxygen separation

UPDP: A Unified Progressive Depth Pruner for CNN and Vision Transformer

Traditional channel-wise pruning methods by reducing network channels struggle to effectively prune efficient CNN models with depth-wise convolutional layers and certain efficient modules, such as popular inverted residual blocks. Prior depth pruning methods by reducing network depths are not suitable for pruning some efficient models due to the existence of some normalization layers. Moreover, finetuning subnet by directly removing activation layers would corrupt the original model weights, hindering the pruned model from achieving high performance. To address these issues, we propose a novel depth pruning method for efficient models. Our approach proposes a novel block pruning strategy and progressive training method for the subnet. Additionally, we extend our pruning method to vision transformer models. Experimental results demonstrate that our method consistently outperforms existing depth pruning methods across various pruning configurations. We obtained three pruned ConvNeXtV1 models with our method applying on ConvNeXtV1, which surpass most SOTA efficient models with comparable inference performance. Our method also achieves state-of-the-art pruning performance on the vision transformer model

Changes in habitat suitability for waterbirds of the Momoge Nature Reserve of China during 1990–2014

There is increasing empirical evidence that changes in habitat quality play an important role in determining species distributions and biodiversity. However, most research has focused on habitat quality, and we still lack approaches for tracking habitat quality dynamics. In this paper, by establishing qualitative and quantitative relationships between waterbird populations and key habitat indicators such as water abundance, food, shelter conditions and disturbance, we developed an object-oriented classification method, in conjunction with a geographic information systems (GIS) based centroid moving method, to assess habitat suitability dynamics for waterbirds at the Momoge Nature Reserve, China. Our results showed that habitat suitability improved during 1990–2000 and declined during 2000– 2014. Habitats with very good and good grades increased by 71.47 km2 (4.88%) during 1990–2000 and decreased by 200.66 km2 (13.78%) during 2000–2014. The habitat area with a good grade moved to the north, while the habitat area with a poor grade moved to the south during 1990–2014. This was mainly because the surrounding cropland area increased and shifted as oil and gas projects developed. These findings suggest that our object-oriented classification and centroid moving methods have great potential for use in biodiversity conservation and ecosystem management

Anti-Allergic Inflammatory Activity of Interleukin-37 Is Mediated by Novel Signaling Cascades in Human Eosinophils

IL-1 family regulatory cytokine IL-37b can suppress innate immunity and inflammatory activity in inflammatory diseases. In this study, IL-37b showed remarkable in vitro suppression of inflammatory tumor necrosis factor-α, IL-1β, IL-6, CCL2, and CXCL8 production in the coculture of human primary eosinophils and human bronchial epithelial BEAS-2B cells with the stimulation of bacterial toll-like receptor-2 ligand peptidoglycan, while antagonizing the activation of intracellular nuclear factor-κB, PI3K–Akt, extracellular signal-regulated kinase 1/2, and suppressing the gene transcription of allergic inflammation-related PYCARD, S100A9, and CAMP as demonstrated by flow cytometry, RNA-sequencing, and bioinformatics. Results therefore elucidated the novel anti-inflammation-related molecular mechanisms mediated by IL-37b. Using the house dust mite (HDM)-induced humanized asthmatic NOD/SCID mice for preclinical study, intravenous administration of IL-37b restored the normal plasma levels of eosinophil activators CCL11 and IL-5, suppressed the elevated concentrations of Th2 and asthma-related cytokines IL-4, IL-6, and IL-13 and inflammatory IL-17, CCL5, and CCL11 in lung homogenate of asthmatic mice. Histopathological results of lung tissue illustrated that IL-37b could mitigate the enhanced mucus, eosinophil infiltration, thickened airway wall, and goblet cells. Together with similar findings using the ovalbumin- and HDM-induced allergic asthmatic mice further validated the therapeutic potential of IL-37b in allergic asthma. The above results illustrate the novel IL-37-mediated regulation of intracellular inflammation mechanism linking bacterial infection and the activation of human eosinophils and confirm the in vivo anti-inflammatory activity of IL-37b on human allergic asthma

Isometric Immersions and Compensated Compactness

A fundamental problem in differential geometry is to characterize intrinsic metrics on a two-dimensional Riemannian manifold ${\mathcal M}^2$ which can be realized as isometric immersions into $\R^3$. This problem can be formulated as initial and/or boundary value problems for a system of nonlinear partial differential equations of mixed elliptic-hyperbolic type whose mathematical theory is largely incomplete. In this paper, we develop a general approach, which combines a fluid dynamic formulation of balance laws for the Gauss-Codazzi system with a compensated compactness framework, to deal with the initial and/or boundary value problems for isometric immersions in $\R^3$. The compensated compactness framework formed here is a natural formulation to ensure the weak continuity of the Gauss-Codazzi system for approximate solutions, which yields the isometric realization of two-dimensional surfaces in $\R^3$. As a first application of this approach, we study the isometric immersion problem for two-dimensional Riemannian manifolds with strictly negative Gauss curvature. We prove that there exists a $C^{1,1}$ isometric immersion of the two-dimensional manifold in $\R^3$ satisfying our prescribed initial conditions. TComment: 25 pages, 6 figue

Extracellular loops 2 and 3 of the calcitonin receptor selectively modify agonist binding and efficacy.

Class B peptide hormone GPCRs are targets for the treatment of major chronic disease. Peptide ligands of these receptors display biased agonism and this may provide future therapeutic advantage. Recent active structures of the calcitonin (CT) and glucagon-like peptide-1 (GLP-1) receptors reveal distinct engagement of peptides with extracellular loops (ECLs) 2 and 3, and mutagenesis of the GLP-1R has implicated these loops in dynamics of receptor activation. In the current study, we have mutated ECLs 2 and 3 of the human CT receptor (CTR), to interrogate receptor expression, peptide affinity and efficacy. Integration of these data with insights from the CTR and GLP-1R active structures, revealed marked diversity in mechanisms of peptide engagement and receptor activation between the CTR and GLP-1R. While the CTR ECL2 played a key role in conformational propagation linked to Gs/cAMP signalling this was mechanistically distinct from that of GLP-1R ECL2. Moreover, ECL3 was a hotspot for distinct ligand- and pathway- specific effects, and this has implications for the future design of biased agonists of class B GPCRs