Implicit Ray-Transformers for Multi-view Remote Sensing Image Segmentation

The mainstream CNN-based remote sensing (RS) image semantic segmentation approaches typically rely on massive labeled training data. Such a paradigm struggles with the problem of RS multi-view scene segmentation with limited labeled views due to the lack of considering 3D information within the scene. In this paper, we propose ''Implicit Ray-Transformer (IRT)'' based on Implicit Neural Representation (INR), for RS scene semantic segmentation with sparse labels (such as 4-6 labels per 100 images). We explore a new way of introducing multi-view 3D structure priors to the task for accurate and view-consistent semantic segmentation. The proposed method includes a two-stage learning process. In the first stage, we optimize a neural field to encode the color and 3D structure of the remote sensing scene based on multi-view images. In the second stage, we design a Ray Transformer to leverage the relations between the neural field 3D features and 2D texture features for learning better semantic representations. Different from previous methods that only consider 3D prior or 2D features, we incorporate additional 2D texture information and 3D prior by broadcasting CNN features to different point features along the sampled ray. To verify the effectiveness of the proposed method, we construct a challenging dataset containing six synthetic sub-datasets collected from the Carla platform and three real sub-datasets from Google Maps. Experiments show that the proposed method outperforms the CNN-based methods and the state-of-the-art INR-based segmentation methods in quantitative and qualitative metrics

Depth Restoration in Under-Display Time-of-Flight Imaging

Under-display imaging has recently received considerable attention in both academia and industry. As a variation of this technique, under-display ToF (UD-ToF) cameras enable depth sensing for full-screen devices. However, it also brings problems of image blurring, signal-to-noise ratio and ranging accuracy reduction. To address these issues, we propose a cascaded deep network to improve the quality of UD-ToF depth maps. The network comprises two subnets, with the first using a complex-valued network in raw domain to perform denoising, deblurring and raw measurements enhancement jointly, while the second refining depth maps in depth domain based on the proposed multi-scale depth enhancement block (MSDEB). To enable training, we establish a data acquisition device and construct a real UD-ToF dataset by collecting real paired ToF raw data. Besides, we also build a large-scale synthetic UD-ToF dataset through noise analysis. The quantitative and qualitative evaluation results on public datasets and ours demonstrate that the presented network outperforms state-of-the-art algorithms and can further promote full-screen devices in practical applications

Skeleton-Based Gesture Recognition With Learnable Paths and Signature Features

For the skeleton-based gesture recognition, graph convolutional networks (GCNs) have achieved remarkable performance since the human skeleton is a natural graph. However, the biological structure might not be the crucial one for motion analysis. Also, spatial differential information like joint distance and angle between bones may be overlooked during the graph convolution. In this paper, we focus on obtaining meaningful joint groups and extracting their discriminative features by the path signature (PS) theory. Firstly, to characterize the constraints and dependencies of various joints, we propose three types of paths, i.e., spatial, temporal, and learnable path. Especially, a learnable path generation mechanism can group joints together that are not directly connected or far away, according to their kinematic characteristic. Secondly, to obtain informative and compact features, a deep integration of PS with few parameters are introduced. All the computational process is packed into two modules, i.e., spatial-temporal path signature module (ST-PSM) and learnable path signature module (L-PSM) for the convenience of utilization. They are plug-and-play modules available for any neural network like CNNs and GCNs to enhance the feature extraction ability. Extensive experiments have conducted on three mainstream datasets (ChaLearn 2013, ChaLearn 2016, and AUTSL). We achieved the state-of-the-art results with simpler framework and much smaller model size. By inserting our two modules into the several GCN-based networks, we can observe clear improvements demonstrating the great effectiveness of our proposed method

Targeting the CD47/SIRPα pathway in malignancies: recent progress, difficulties and future perspectives

Since its initial report in 2015, CD47 has garnered significant attention as an innate immune checkpoint, raising expectations to become the next “PD-1.” The optimistic early stages of clinical development spurred a flurry of licensing deals for CD47-targeted molecules and company mergers or acquisitions for related assets. However, a series of setbacks unfolded recently, starting with the July 2023 announcement of discontinuing the phase 3 ENHANCE study on Magrolimab plus Azacitidine for higher-risk myelodysplastic syndromes (MDS). Subsequently, in August 2023, the termination of the ASPEN-02 program, assessing Evorpacept in combination with Azacitidine in MDS patients, was disclosed due to insufficient improvement compared to Azacitidine alone. These setbacks have cast doubt on the feasibility of targeting CD47 in the industry. In this review, we delve into the challenges of developing CD47-SIRPα-targeted drugs, analyze factors contributing to the mentioned setbacks, discuss future perspectives, and explore potential solutions for enhancing CD47-SIRPα-targeted drug development

Advances in alternative splicing identification: deep learning and pantranscriptome

In plants, alternative splicing is a crucial mechanism for regulating gene expression at the post-transcriptional level, which leads to diverse proteins by generating multiple mature mRNA isoforms and diversify the gene regulation. Due to the complexity and variability of this process, accurate identification of splicing events is a vital step in studying alternative splicing. This article presents the application of alternative splicing algorithms with or without reference genomes in plants, as well as the integration of advanced deep learning techniques for improved detection accuracy. In addition, we also discuss alternative splicing studies in the pan-genomic background and the usefulness of integrated strategies for fully profiling alternative splicing

Homologous haplotypes, expression, genetic effects and geographic distribution of the wheat yield gene TaGW2

BACKGROUND: TaGW2-6A, cloned in earlier research, strongly influences wheat grain width and TKW. Here, we mainly analyzed haplotypes of TaGW2-6B and their effects on TKW and interaction with haplotypes at TaGW2-6A. RESULTS: About 2.9 kb of the promoter sequences of TaGW2-6B and TaGW2-6D were cloned in 34 bread wheat cultivars. Eleven SNPs were detected in the promoter region of TaGW2-6B, forming 4 haplotypes, but no divergence was detected in the TaGW2-6D promoter or coding region. Three molecular markers including CAPS, dCAPS and ACAS, were developed to distinguish the TaGW2-6B haplotypes. Haplotype association analysis indicated that TaGW2-6B has a stronger influence than TaGW2-6A on TKW, and Hap-6B-1 was a favored haplotype increasing grain width and weight that had undergone strong positive selection in global wheat breeding. However, clear geographic distribution differences for TaGW2-6A haplotypes were found; Hap-6A-A was favored in Chinese, Australian and Russian cultivars, whereas Hap-6A-G was preferred in European, American and CIMMYT cultivars. This difference might be caused by a flowering and maturity time difference between the two haplotypes. Hap-6A-A is the earlier type. Haplotype interaction analysis between TaGW2-6A and TaGW2-6B showed additive effects between the favored haplotypes. Hap-6A-A/Hap-6B-1 was the best combination to increase TKW. Relative expression analysis of the three TaGW2 homoeologous genes in 22 cultivars revealed that TaGW2-6A underwent the highest expression. TaGW2-6D was the least expressed during grain development and TaGW2-6B was intermediate. Diversity of the three genes was negatively correlated with their effect on TKW. CONCLUSIONS: Genetic effects, expression patterns and historic changes of haplotypes at three homoeologous genes of TaGW2 influencing yield were dissected in wheat cultivars. Strong and constant selection to favored haplotypes has been found in global wheat breeding during the past century. This research also provides a valuable case for understanding interaction of genes that control complex traits in polyploid species

Path Signature Neural Network of Cortical Features for Prediction of Infant Cognitive Scores

Studies have shown that there is a tight connection between cognition skills and brain morphology during infancy. Nonetheless, it is still a great challenge to predict individual cognitive scores using their brain morphological features, considering issues like the excessive feature dimension, small sample size and missing data. Due to the limited data, a compact but expressive feature set is desirable as it can reduce the dimension and avoid the potential overfitting issue. Therefore, we pioneer the path signature method to further explore the essential hidden dynamic patterns of longitudinal cortical features. To form a hierarchical and more informative temporal representation, in this work, a novel cortical feature based path signature neural network (CF-PSNet) is proposed with stacked differentiable temporal path signature layers for prediction of individual cognitive scores. By introducing the existence embedding in path generation, we can improve the robustness against the missing data. Benefiting from the global temporal receptive field of CF-PSNet, characteristics consisted in the existing data can be fully leveraged. Further, as there is no need for the whole brain to work for a certain cognitive ability, a top K selection module is used to select the most influential brain regions, decreasing the model size and the risk of overfitting. Extensive experiments are conducted on an in-house longitudinal infant dataset within 9 time points. By comparing with several recent algorithms, we illustrate the state-of-the-art performance of our CF-PSNet (i.e., root mean square error of 0.027 with the time latency of 518 milliseconds for each sample)