Plan To Predict: Learning an Uncertainty-Foreseeing Model for Model-Based Reinforcement Learning

In Model-based Reinforcement Learning (MBRL), model learning is critical since an inaccurate model can bias policy learning via generating misleading samples. However, learning an accurate model can be difficult since the policy is continually updated and the induced distribution over visited states used for model learning shifts accordingly. Prior methods alleviate this issue by quantifying the uncertainty of model-generated samples. However, these methods only quantify the uncertainty passively after the samples were generated, rather than foreseeing the uncertainty before model trajectories fall into those highly uncertain regions. The resulting low-quality samples can induce unstable learning targets and hinder the optimization of the policy. Moreover, while being learned to minimize one-step prediction errors, the model is generally used to predict for multiple steps, leading to a mismatch between the objectives of model learning and model usage. To this end, we propose \emph{Plan To Predict} (P2P), an MBRL framework that treats the model rollout process as a sequential decision making problem by reversely considering the model as a decision maker and the current policy as the dynamics. In this way, the model can quickly adapt to the current policy and foresee the multi-step future uncertainty when generating trajectories. Theoretically, we show that the performance of P2P can be guaranteed by approximately optimizing a lower bound of the true environment return. Empirical results demonstrate that P2P achieves state-of-the-art performance on several challenging benchmark tasks.Comment: Accepted by NeurIPS202

Improving Robust Generalization by Direct PAC-Bayesian Bound Minimization

Recent research in robust optimization has shown an overfitting-like phenomenon in which models trained against adversarial attacks exhibit higher robustness on the training set compared to the test set. Although previous work provided theoretical explanations for this phenomenon using a robust PAC-Bayesian bound over the adversarial test error, related algorithmic derivations are at best only loosely connected to this bound, which implies that there is still a gap between their empirical success and our understanding of adversarial robustness theory. To close this gap, in this paper we consider a different form of the robust PAC-Bayesian bound and directly minimize it with respect to the model posterior. The derivation of the optimal solution connects PAC-Bayesian learning to the geometry of the robust loss surface through a Trace of Hessian (TrH) regularizer that measures the surface flatness. In practice, we restrict the TrH regularizer to the top layer only, which results in an analytical solution to the bound whose computational cost does not depend on the network depth. Finally, we evaluate our TrH regularization approach over CIFAR-10/100 and ImageNet using Vision Transformers (ViT) and compare against baseline adversarial robustness algorithms. Experimental results show that TrH regularization leads to improved ViT robustness that either matches or surpasses previous state-of-the-art approaches while at the same time requires less memory and computational cost

Neuromorphic Incremental on-chip Learning with Hebbian Weight Consolidation

As next-generation implantable brain-machine interfaces become pervasive on edge device, incrementally learning new tasks in bio-plasticity ways is urgently demanded for Neuromorphic chips. Due to the inherent characteristics of its structure, spiking neural networks are naturally well-suited for BMI-chips. Here we propose Hebbian Weight Consolidation, as well as an on-chip learning framework. HWC selectively masks synapse modifications for previous tasks, retaining them to store new knowledge from subsequent tasks while preserving the old knowledge. Leveraging the bio-plasticity of dendritic spines, the intrinsic self-organizing nature of Hebbian Weight Consolidation aligns naturally with the incremental learning paradigm, facilitating robust learning outcomes. By reading out spikes layer by layer and performing back-propagation on the external micro-controller unit, MLoC can efficiently accomplish on-chip learning. Experiments show that our HWC algorithm up to 23.19% outperforms lower bound that without incremental learning algorithm, particularly in more challenging monkey behavior decoding scenarios. Taking into account on-chip computing on Synsense Speck 2e chip, our proposed algorithm exhibits an improvement of 11.06%. This study demonstrates the feasibility of employing incremental learning for high-performance neural signal decoding in next-generation brain-machine interfaces.Comment: 12 pages, 6 figure

Improving 3D-aware Image Synthesis with A Geometry-aware Discriminator

3D-aware image synthesis aims at learning a generative model that can render photo-realistic 2D images while capturing decent underlying 3D shapes. A popular solution is to adopt the generative adversarial network (GAN) and replace the generator with a 3D renderer, where volume rendering with neural radiance field (NeRF) is commonly used. Despite the advancement of synthesis quality, existing methods fail to obtain moderate 3D shapes. We argue that, considering the two-player game in the formulation of GANs, only making the generator 3D-aware is not enough. In other words, displacing the generative mechanism only offers the capability, but not the guarantee, of producing 3D-aware images, because the supervision of the generator primarily comes from the discriminator. To address this issue, we propose GeoD through learning a geometry-aware discriminator to improve 3D-aware GANs. Concretely, besides differentiating real and fake samples from the 2D image space, the discriminator is additionally asked to derive the geometry information from the inputs, which is then applied as the guidance of the generator. Such a simple yet effective design facilitates learning substantially more accurate 3D shapes. Extensive experiments on various generator architectures and training datasets verify the superiority of GeoD over state-of-the-art alternatives. Moreover, our approach is registered as a general framework such that a more capable discriminator (i.e., with a third task of novel view synthesis beyond domain classification and geometry extraction) can further assist the generator with a better multi-view consistency.Comment: Accepted by NeurIPS 2022. Project page: https://vivianszf.github.io/geo

MCNTs@MnO2 nanocomposite cathode integrated with soluble O2-carrier Co-salen in electrolyte for high-performance Li-air batteries

Li–air batteries (LABs) are promising because of their high energy density. However, LABs are troubled by large electrochemical polarization during discharge and charge, side reactions from both carbon cathode surface/peroxide product and electrolyte/superoxide intermediate, as well as the requirement for pure O2. Here we report the solution using multiwall carbon nanotubes (MCNTs)@MnO2 nanocomposite cathode integrated with N,N′-bis(salicylidene)ethylenediaminocobalt(II) (CoII-salen) in electrolyte for LABs. The advantage of such a combination is that on one hand, the coating layer of δ-MnO2 with about 2–3 nm on MCNTs@MnO2 nanocomposite catalyzes Li2O2 decomposition during charge and suppresses side reactions between product Li2O2 and MCNT surface. On the other hand, CoII-salen works as a mobile O2-carrier and accelerates Li2O2 formation through the reaciton of (CoIII-salen)2-O22– + 2Li+ + 2e– → 2CoII-salen + Li2O2. This reaction route overcomes the pure O2 limitation and avoids the formation of aggressive superoxide intermediate (O2– or LiO2), which easily attacks organic electrolyte. By using this double-catalyst system of Co-salen/MCNTs@MnO2, the lifetime of LABs is prolonged to 300 cycles at 500 mA g–1 (0.15 mA cm–2) with fixed capacity of 1000 mAh g–1 (0.30 mAh cm–2) in dry air (21% O2). Furthermore, we up-scale the capacity to 500 mAh (5.2 mAh cm–2) in pouch-type batteries (∼4 g, 325 Wh kg–1). This study should pave a new way for the design and construction of practical LABs

LinkGAN: Linking GAN Latents to Pixels for Controllable Image Synthesis

This work presents an easy-to-use regularizer for GAN training, which helps explicitly link some axes of the latent space to a set of pixels in the synthesized image. Establishing such a connection facilitates a more convenient local control of GAN generation, where users can alter the image content only within a spatial area simply by partially resampling the latent code. Experimental results confirm four appealing properties of our regularizer, which we call LinkGAN. (1) The latent-pixel linkage is applicable to either a fixed region (\textit{i.e.}, same for all instances) or a particular semantic category (i.e., varying across instances), like the sky. (2) Two or multiple regions can be independently linked to different latent axes, which further supports joint control. (3) Our regularizer can improve the spatial controllability of both 2D and 3D-aware GAN models, barely sacrificing the synthesis performance. (4) The models trained with our regularizer are compatible with GAN inversion techniques and maintain editability on real images

Identification and validation of ferroptosis-related genes and immune cell infiltration in thyroid associated ophthalmopathy

Thyroid associated ophthalmopathy (TAO) is an orbital autoimmune inflammatory disease that is commonly associated with thyroid dysfunction. Although the etiology of TAO is unclear, ROS accumulation and oxidative stress have been closely linked to the pathogenesis of TAO. Ferroptosis is an iron-dependent programmed cell death characterized by intracellular labile iron levels, excessive accumulation of reactive oxygen species (ROS) and lipid peroxidation. Currently, there are few reports regarding the role of ferroptosis in TAO. This article aimed to identify ferroptosis-related genes (FRGs) with diagnostic and therapeutic potential in TAO and explore their relationship with immune cells and lncRNAs. GSE58331 was downloaded from Gene Expression Omnibus (GEO) database. A total of 162 DEGs were identified between 27 TAO samples and 22 health samples from GSE58331, among which six FRGs (CYBB, CTSB, SLC38A1, TLR4, PEX3, and ABCC1) were obtained. The AUC of SLC38A1, TLR4, PEX3 in lacrimal gland tissues was greater than 80 which suggested high diagnostic value in TAO. The result of immune cell infiltrate analysis indicated increased infiltration of monocytes (p < 0.001), macrophages M0(p = 0.039), mast cells activated (p = 0.008), and neutrophils (p = 0.045) in orbital tissues from TAO patients. Meanwhile, mast cells resting (p = 0.043) and macrophages M2 (p = 0.02) showed reduced infiltration in TAO samples. There were no gender differences in immune cell infiltration in the TAO patients. Two differentially expressed lncRNAs, LINC01140 and ZFHX4-AS1, in TAO groups were identified as ferroptosis-related lncRNAs. CYBB-LINC01140-TLR4, CYBB- LINC01140- SLC38A1, TLR4- LINC01140- SLC38A1, and CTSB- ZFHX4-AS1- CYBB may be potential RNA regulatory pathways in TAO. Targeted drugs and transcription factors for differential expressed FRGs were also screened out in our study. In vitro, experiments revealed that CTSB, PEX3, ABCC1 and ZFHX4-AS1(lncRNA) were differentially expressed in orbital fibroblasts (OFs) between TAO groups and healthy controls at the transcriptional level

propnet: Propagating 2D Annotation to 3D Segmentation for Gastric Tumors on CT Scans

**Background:** Accurate 3D CT scan segmentation of gastric tumors is pivotal for diagnosis and treatment. The challenges lie in the irregular shapes, blurred boundaries of tumors, and the inefficiency of existing methods. **Purpose:** We conducted a study to introduce a model, utilizing human-guided knowledge and unique modules, to address the challenges of 3D tumor segmentation. **Methods:** We developed the PropNet framework, propagating radiologists' knowledge from 2D annotations to the entire 3D space. This model consists of a proposing stage for coarse segmentation and a refining stage for improved segmentation, using two-way branches for enhanced performance and an up-down strategy for efficiency. **Results:** With 98 patient scans for training and 30 for validation, our method achieves a significant agreement with manual annotation (Dice of 0.803) and improves efficiency. The performance is comparable in different scenarios and with various radiologists' annotations (Dice between 0.785 and 0.803). Moreover, the model shows improved prognostic prediction performance (C-index of 0.620 vs. 0.576) on an independent validation set of 42 patients with advanced gastric cancer. **Conclusions:** Our model generates accurate tumor segmentation efficiently and stably, improving prognostic performance and reducing high-throughput image reading workload. This model can accelerate the quantitative analysis of gastric tumors and enhance downstream task performance

Gaussian Shell Maps for Efficient 3D Human Generation

Efficient generation of 3D digital humans is important in several industries, including virtual reality, social media, and cinematic production. 3D generative adversarial networks (GANs) have demonstrated state-of-the-art (SOTA) quality and diversity for generated assets. Current 3D GAN architectures, however, typically rely on volume representations, which are slow to render, thereby hampering the GAN training and requiring multi-view-inconsistent 2D upsamplers. Here, we introduce Gaussian Shell Maps (GSMs) as a framework that connects SOTA generator network architectures with emerging 3D Gaussian rendering primitives using an articulable multi shell--based scaffold. In this setting, a CNN generates a 3D texture stack with features that are mapped to the shells. The latter represent inflated and deflated versions of a template surface of a digital human in a canonical body pose. Instead of rasterizing the shells directly, we sample 3D Gaussians on the shells whose attributes are encoded in the texture features. These Gaussians are efficiently and differentiably rendered. The ability to articulate the shells is important during GAN training and, at inference time, to deform a body into arbitrary user-defined poses. Our efficient rendering scheme bypasses the need for view-inconsistent upsamplers and achieves high-quality multi-view consistent renderings at a native resolution of $512 \times 512$ pixels. We demonstrate that GSMs successfully generate 3D humans when trained on single-view datasets, including SHHQ and DeepFashion.Comment: Project page : https://rameenabdal.github.io/GaussianShellMaps

MCNTs@MnO2 Nanocomposite Cathode Integrated with Soluble O2Carrier Co-salen in Electrolyte for High-Performance LiAir Batteries

Li–air batteries (LABs) are promising because of their high energy density. However, LABs are troubled by large electrochemical polarization during discharge and charge, side reactions from both carbon cathode surface/peroxide product and electrolyte/superoxide intermediate, as well as the requirement for pure O2. Here we report the solution using multiwall carbon nanotubes (MCNTs)@MnO2 nanocomposite cathode integrated with N,N′-bis(salicylidene)ethylenediaminocobalt(II) (CoII-salen) in electrolyte for LABs. The advantage of such a combination is that on one hand, the coating layer of δ-MnO2 with about 2–3 nm on MCNTs@MnO2 nanocomposite catalyzes Li2O2 decomposition during charge and suppresses side reactions between product Li2O2 and MCNT surface. On the other hand, CoII-salen works as a mobile O2-carrier and accelerates Li2O2 formation through the reaciton of (CoIII-salen)2-O22– + 2Li+ + 2e– → 2CoII-salen + Li2O2. This reaction route overcomes the pure O2 limitation and avoids the formation of aggressive superoxide intermediate (O2– or LiO2), which easily attacks organic electrolyte. By using this double-catalyst system of Co-salen/MCNTs@MnO2, the lifetime of LABs is prolonged to 300 cycles at 500 mA g–1 (0.15 mA cm–2) with fixed capacity of 1000 mAh g–1 (0.30 mAh cm–2) in dry air (21% O2). Furthermore, we up-scale the capacity to 500 mAh (5.2 mAh cm–2) in pouch-type batteries (∼4 g, 325 Wh kg–1). This study should pave a new way for the design and construction of practical LABs