Identity-Seeking Self-Supervised Representation Learning for Generalizable Person Re-identification

This paper aims to learn a domain-generalizable (DG) person re-identification (ReID) representation from large-scale videos \textbf{without any annotation}. Prior DG ReID methods employ limited labeled data for training due to the high cost of annotation, which restricts further advances. To overcome the barriers of data and annotation, we propose to utilize large-scale unsupervised data for training. The key issue lies in how to mine identity information. To this end, we propose an Identity-seeking Self-supervised Representation learning (ISR) method. ISR constructs positive pairs from inter-frame images by modeling the instance association as a maximum-weight bipartite matching problem. A reliability-guided contrastive loss is further presented to suppress the adverse impact of noisy positive pairs, ensuring that reliable positive pairs dominate the learning process. The training cost of ISR scales approximately linearly with the data size, making it feasible to utilize large-scale data for training. The learned representation exhibits superior generalization ability. \textbf{Without human annotation and fine-tuning, ISR achieves 87.0\% Rank-1 on Market-1501 and 56.4\% Rank-1 on MSMT17}, outperforming the best supervised domain-generalizable method by 5.0\% and 19.5\%, respectively. In the pre-training$\rightarrow$fine-tuning scenario, ISR achieves state-of-the-art performance, with 88.4\% Rank-1 on MSMT17. The code is at \url{https://github.com/dcp15/ISR_ICCV2023_Oral}.Comment: ICCV 2023 Ora

Dynamic Layer Aggregation for Neural Machine Translation with Routing-by-Agreement

With the promising progress of deep neural networks, layer aggregation has been used to fuse information across layers in various fields, such as computer vision and machine translation. However, most of the previous methods combine layers in a static fashion in that their aggregation strategy is independent of specific hidden states. Inspired by recent progress on capsule networks, in this paper we propose to use routing-by-agreement strategies to aggregate layers dynamically. Specifically, the algorithm learns the probability of a part (individual layer representations) assigned to a whole (aggregated representations) in an iterative way and combines parts accordingly. We implement our algorithm on top of the state-of-the-art neural machine translation model TRANSFORMER and conduct experiments on the widely-used WMT14 English-German and WMT17 Chinese-English translation datasets. Experimental results across language pairs show that the proposed approach consistently outperforms the strong baseline model and a representative static aggregation model.Comment: AAAI 201

Generalizable Re-Identification from Videos with Cycle Association

In this paper, we are interested in learning a generalizable person re-identification (re-ID) representation from unlabeled videos. Compared with 1) the popular unsupervised re-ID setting where the training and test sets are typically under the same domain, and 2) the popular domain generalization (DG) re-ID setting where the training samples are labeled, our novel scenario combines their key challenges: the training samples are unlabeled, and collected form various domains which do no align with the test domain. In other words, we aim to learn a representation in an unsupervised manner and directly use the learned representation for re-ID in novel domains. To fulfill this goal, we make two main contributions: First, we propose Cycle Association (CycAs), a scalable self-supervised learning method for re-ID with low training complexity; and second, we construct a large-scale unlabeled re-ID dataset named LMP-video, tailored for the proposed method. Specifically, CycAs learns re-ID features by enforcing cycle consistency of instance association between temporally successive video frame pairs, and the training cost is merely linear to the data size, making large-scale training possible. On the other hand, the LMP-video dataset is extremely large, containing 50 million unlabeled person images cropped from over 10K Youtube videos, therefore is sufficient to serve as fertile soil for self-supervised learning. Trained on LMP-video, we show that CycAs learns good generalization towards novel domains. The achieved results sometimes even outperform supervised domain generalizable models. Remarkably, CycAs achieves 82.2% Rank-1 on Market-1501 and 49.0% Rank-1 on MSMT17 with zero human annotation, surpassing state-of-the-art supervised DG re-ID methods. Moreover, we also demonstrate the superiority of CycAs under the canonical unsupervised re-ID and the pretrain-and-finetune scenarios

Ce6-modified Fe ions-doped carbon dots as multifunctional nanoplatform for ferroptosis and photodynamic synergistic therapy of melanoma

Abstract Background Despite the higher sensitivity of melanoma towards ferroptosis and photodynamic therapy (PDT), the lack of efficient ferroptosis inducers and the poor solubility of photosensitizers restrict their synergistic strategies. With unique advantages, carbon dots (CDs) are expected to serve as innovative building blocks for combination therapy of cancers. Results Herein, an ferroptosis/PDT integrated nanoplatform for melanoma therapy is constructed based on chlorin e6-modified Fe ions-doped carbon dots (Fe-CDs@Ce6). As a novel type of iron-carbon hybrid nanoparticles, the as-prepared Fe-CDs can selectively activate ferroptosis, prevent angiogenesis and inhibit the migration of mouse skin melanoma cells (B16), but have no toxicity to normal cells. The nano-conjugated structures facilitate not only the aqueous dispersibility of Ce6, but also the self-accumulation ability of Fe-CDs@Ce6 within melanoma area without requiring extra targets. Moreover, the therapeutic effects of Fe-CDs@Ce6 are synergistically enhanced due to the increased GSH depletion by PDT and the elevated singlet oxygen (1O2) production efficiency by Fe-CDs. When combined with laser irradiation, the tumor growth can be significantly suppressed by Fe-CDs@Ce6 through cyclic administration. The T 2 -weighted magnetic resonance imaging (MRI) capability of Fe-CDs@Ce6 also reveals their potentials for cancer diagnosis and navigation therapy. Conclusions Our findings indicate the multifunctionality of Fe-CDs@Ce6 in effectively combining ferroptosis/PDT therapy, tumor targeting and MRI imaging, which enables Fe-CDs@Ce6 to become promising biocompatible nanoplatform for the treatment of melanoma. Graphic Abstrac

Additional file 1 of Ce6-modified Fe ions-doped carbon dots as multifunctional nanoplatform for ferroptosis and photodynamic synergistic therapy of melanoma

Additional file 1: Figure S1. PL spectra of Fe-CDs under different excitation wavelengths. Figure S2. Deconvoluted C 1s and O 1s XPS spectra of Fe-CDs. Figure S3. FTIR spectrum of Fe-CDs and Fe-CDs@Ce6. Figure S4. XPS full spectra, deconvoluted C 1s and O 1s spectra of Fe-CDs@Ce6. Figure S5. Evaluation of stability of Fe-CDs@Ce6 (2 mg/mL) in PBS (pH=7.4), acidic PBS (pH=5.0) and serum (10% FBS). Figure S6. Photographs of solution for the chromatic reaction of Fe ions and SA. From left to right: SA, SA+FeSO4, SA+Fe-CDs (0.2 mg/mL), SA+Fe-CDs (0.5 mg/mL), SA+GSH-reduced Fe-CDs (0.2 mg/mL), and SA+GSH-reduced Fe-CDs (0.5 mg/mL). Figure. S7 Photographs of solution for the chromogenic reaction of Fe2+ and K3[Fe(CN)6]. From left to right: (a) 0.1 mg/mL K3[Fe(CN)6], (b) 0.1 mg/mL K3[Fe(CN)6] + 0.01 mg/mL FeSO4, (c) 0.1 mg/mL K3[Fe(CN)6] + 0.01 mg/mL Fe2(SO4)3, (d) 0.1 mg/mL K3[Fe(CN)6] + 0.1 mg/mL Fe-CDs, (e) 0.1 mg/mL K3[Fe(CN)6] + 0.2 mg/mL Fe-CDs, (f) 0.1 mg/mL K3[Fe(CN)6] + 0.5 mg/mL Fe-CDs. Figure S8. Evaluation of Fenton reaction by measuring the UV-Vis absorption spectra of MB and H2O2 under different concentrations of Fe-CDs (A) and GSH reduced Fe-CDs (B) (insets show the solution photographs of MB+H2O2 and different concentrations of Fe-CDs and GSH reduced Fe-CDs). Figure. S9 ELISA assay of TNF-α (A) and IL-10 (B) expression level in B16 conditioned medium after Fe-CDs@Ce6+PDT treatment. Figure S10. A-B The levels of ROS in B16 cells after treatment with Fe-CDs (Scale bar: 500 μm). C-D The alteration of mitochondrial membrane potential in B16 cells after treatment with Fe-CDs (Scale bar：100 μm) (n=3, **p<0.01, ***p<0.001 were considered statistically significant). Figure S11. Western blot analysis for β-catenin, xCT, Lef1, HO1 and GPX4 (n=3, *p<0.05, **p<0.01, ***p<0.001 were considered statistically significant). Figure S12. Stimulation of tube formation in HUVECs using B16 cell supernatant after treatment (Scale bar: 200 μm and 100 μm from top to bottom) (n=3, ****p<0.0001 were considered statistically significant). Figure S13. A Macroscopic presentation of tumors, along with measurements of B tumor volume and C mouse body weight (n=6) (Scale bar: 1 cm). Figure. S14 Co-staining of Lyso- and Mito tracker with Fe-CDs@Ce6 at the incubation temperature of 4 and 37°C. Scale bar: 100 μm. Figure. S15 Time-dependent cellular uptake of Fe-CDs@Ce6 by B16 cells at the incubation temperature of 4 and 37°C. Scale bar: 100 μm. Figure S16. The result of H&E staining of main organs of nude mice after treatment (Scale bar: 100 μm). Figure S17. Volcano plot of melanoma in PBS vs Fe-CDs. Figure S18. Immunohistochemical analysis of β-catenin, Lef1, HO1 and GPX4 (n=3, **p<0.01, ***p<0.001, ****p<0.0001 were considered statistically significant). Figure S19. Western blot analysis for β-catenin, xCT, Lef1, HO1 and GPX4 (n=3, **p<0.01, ***p<0.001, ****p<0.0001 were considered statistically significant)