Learning from Noisy Crowd Labels with Logics

This paper explores the integration of symbolic logic knowledge into deep neural networks for learning from noisy crowd labels. We introduce Logic-guided Learning from Noisy Crowd Labels (Logic-LNCL), an EM-alike iterative logic knowledge distillation framework that learns from both noisy labeled data and logic rules of interest. Unlike traditional EM methods, our framework contains a ``pseudo-E-step'' that distills from the logic rules a new type of learning target, which is then used in the ``pseudo-M-step'' for training the classifier. Extensive evaluations on two real-world datasets for text sentiment classification and named entity recognition demonstrate that the proposed framework improves the state-of-the-art and provides a new solution to learning from noisy crowd labels.Comment: 12 pages, 7 figures, accepted by ICDE-202

Spatial Organization of CD28 Modulates T-cell Activation

T-cells are central to our success as a species. They confer specific and long-term immunity in a process known as adaptive immunity. During adaptive immune response, pathogen ingested by peripheral sentinel cells are brought to the local lymph nodes and presented to T-cells. T-cell recognizes the antigen via its receptor complex (TCR-CD3). The high affinity binding primes the cell for activation. With a positive costimulationary signal from CD28, the T-cell is fully activated, resulting in IL-2 secretions and cellular proliferation. Clinicians are increasingly harnessing the adaptive immune system to combat diseases such as cancer. Specifically, T-cells are activated and expanded ex vivo for adoptive immunotherapies. The ability to modulate T-cell activation is crucial in engineering appropriate effector cell populations for therapeutics. The focus of this thesis is to address the functional impact of CD28 spatial organization on T-cell activation. It has been observed that the spatial segregation of CD3 and CD28 by a few microns has resulted in poor activation of human T-cells. Lck, a Src family kinase (SFK) emerges as the instigator of the phenomenon. The kinase is associated with both CD3 and CD28 signal cascades. We propose a reaction diffusion model to describe the delicate balance between protein mobility and Lck de-activation. The work in this dissertation describes two probes to investigate Lck kinase activity, which permit real-time imaging of both the initiation of pLck activity and its duration. A FRET reporter is constructed to study the spatial and temporal initiation of the kinase activity. Embedded with the Lck membrane domain and contained a substrate for pLck to phosphorylate, the FRET biosensor reports the Lck kinase activity in real-time. Using microprinting to control CD3 and CD28 spatial organizations, the FRET reporter reveals that while T-cells require CD28 for significant IL-2 secretion, CD3 engagement is essential to initiate cellular activation through a spike in pLck kinase activity. Spatially, the reporter shows heightened kinase activity concentrated at the center of the cells upon CD3 engagement. To study the duration of pLck activity, a recruitment reporter is made. CD3 is found ubiquitously throughout the cellular membrane. And its activation by pLck induces the recruitment of a pair of tandem SH2-domain. The recruitment probe (also containing a pair of tandem SH2-domain) revealed curtailed pLck kinase activity due to CD3-CD28 segregation. Ultimately, understanding CD28 modulation of T-cell activation is clinically relevant as it provides new opportunities and targets for the development of therapeutics

Collaborative Noisy Label Cleaner: Learning Scene-aware Trailers for Multi-modal Highlight Detection in Movies

Movie highlights stand out of the screenplay for efficient browsing and play a crucial role on social media platforms. Based on existing efforts, this work has two observations: (1) For different annotators, labeling highlight has uncertainty, which leads to inaccurate and time-consuming annotations. (2) Besides previous supervised or unsupervised settings, some existing video corpora can be useful, e.g., trailers, but they are often noisy and incomplete to cover the full highlights. In this work, we study a more practical and promising setting, i.e., reformulating highlight detection as "learning with noisy labels". This setting does not require time-consuming manual annotations and can fully utilize existing abundant video corpora. First, based on movie trailers, we leverage scene segmentation to obtain complete shots, which are regarded as noisy labels. Then, we propose a Collaborative noisy Label Cleaner (CLC) framework to learn from noisy highlight moments. CLC consists of two modules: augmented cross-propagation (ACP) and multi-modality cleaning (MMC). The former aims to exploit the closely related audio-visual signals and fuse them to learn unified multi-modal representations. The latter aims to achieve cleaner highlight labels by observing the changes in losses among different modalities. To verify the effectiveness of CLC, we further collect a large-scale highlight dataset named MovieLights. Comprehensive experiments on MovieLights and YouTube Highlights datasets demonstrate the effectiveness of our approach. Code has been made available at: https://github.com/TencentYoutuResearch/HighlightDetection-CLCComment: Accepted to CVPR202

Equirectangular image construction method for standard CNNs for Semantic Segmentation

360{\deg} spherical images have advantages of wide view field, and are typically projected on a planar plane for processing, which is known as equirectangular image. The object shape in equirectangular images can be distorted and lack translation invariance. In addition, there are few publicly dataset of equirectangular images with labels, which presents a challenge for standard CNNs models to process equirectangular images effectively. To tackle this problem, we propose a methodology for converting a perspective image into equirectangular image. The inverse transformation of the spherical center projection and the equidistant cylindrical projection are employed. This enables the standard CNNs to learn the distortion features at different positions in the equirectangular image and thereby gain the ability to semantically the equirectangular image. The parameter, {\phi}, which determines the projection position of the perspective image, has been analyzed using various datasets and models, such as UNet, UNet++, SegNet, PSPNet, and DeepLab v3+. The experiments demonstrate that an optimal value of {\phi} for effective semantic segmentation of equirectangular images is 6{\pi}/16 for standard CNNs. Compared with the other three types of methods (supervised learning, unsupervised learning and data augmentation), the method proposed in this paper has the best average IoU value of 43.76%. This value is 23.85%, 10.7% and 17.23% higher than those of other three methods, respectively

Scene Consistency Representation Learning for Video Scene Segmentation

A long-term video, such as a movie or TV show, is composed of various scenes, each of which represents a series of shots sharing the same semantic story. Spotting the correct scene boundary from the long-term video is a challenging task, since a model must understand the storyline of the video to figure out where a scene starts and ends. To this end, we propose an effective Self-Supervised Learning (SSL) framework to learn better shot representations from unlabeled long-term videos. More specifically, we present an SSL scheme to achieve scene consistency, while exploring considerable data augmentation and shuffling methods to boost the model generalizability. Instead of explicitly learning the scene boundary features as in the previous methods, we introduce a vanilla temporal model with less inductive bias to verify the quality of the shot features. Our method achieves the state-of-the-art performance on the task of Video Scene Segmentation. Additionally, we suggest a more fair and reasonable benchmark to evaluate the performance of Video Scene Segmentation methods. The code is made available.Comment: Accepted to CVPR 202

Printing of Fine Metal Electrodes for Organic Thin‐Film Transistors

Attributed to the excellent mechanical flexibility and compatibility with low‐cost and high‐throughput printing processes, the organic thin‐film transistor (OTFT) is a promising technology of choice for a wide range of flexible and large‐area electronics applications. Among various printing techniques, the drop‐on‐demand inkjet printing is one of the most versatile ones to form patterned electrodes with the advantages of mask‐less patterning, non‐contact, low cost, and scalability to large‐area manufacturing. However, the limited positional accuracy of the inkjet printer system and the spreading of the ink droplets on the substrate surface, which is influenced by both the ink properties and the substrate surface energy, make it difficult to obtain fine‐line morphologies and define the exact channel length as required, especially for relatively narrow‐line and short‐channel patterns. This chapter introduces the printing of uniform fine silver electrodes and down scaling of the channel length by controlling ink wetting on polymer substrate. All‐solution‐processed/printable OTFTs with short channels (<20 µm) are also demonstrated by incorporating fine inkjet‐printed silver electrodes into a low‐voltage (<3 V) OTFT architecture. This work would provide a commercially competitive manufacturing approach to developing printable low‐voltage OTFTs for low‐power electronics applications