VQS: Linking Segmentations to Questions and Answers for Supervised Attention in VQA and Question-Focused Semantic Segmentation

Rich and dense human labeled datasets are among the main enabling factors for the recent advance on vision-language understanding. Many seemingly distant annotations (e.g., semantic segmentation and visual question answering (VQA)) are inherently connected in that they reveal different levels and perspectives of human understandings about the same visual scenes --- and even the same set of images (e.g., of COCO). The popularity of COCO correlates those annotations and tasks. Explicitly linking them up may significantly benefit both individual tasks and the unified vision and language modeling. We present the preliminary work of linking the instance segmentations provided by COCO to the questions and answers (QAs) in the VQA dataset, and name the collected links visual questions and segmentation answers (VQS). They transfer human supervision between the previously separate tasks, offer more effective leverage to existing problems, and also open the door for new research problems and models. We study two applications of the VQS data in this paper: supervised attention for VQA and a novel question-focused semantic segmentation task. For the former, we obtain state-of-the-art results on the VQA real multiple-choice task by simply augmenting the multilayer perceptrons with some attention features that are learned using the segmentation-QA links as explicit supervision. To put the latter in perspective, we study two plausible methods and compare them to an oracle method assuming that the instance segmentations are given at the test stage.Comment: To appear on ICCV 201

Towards A Unified Neural Architecture for Visual Recognition and Reasoning

Recognition and reasoning are two pillars of visual understanding. However, these tasks have an imbalance in focus; whereas recent advances in neural networks have shown strong empirical performance in visual recognition, there has been comparably much less success in solving visual reasoning. Intuitively, unifying these two tasks under a singular framework is desirable, as they are mutually dependent and beneficial. Motivated by the recent success of multi-task transformers for visual recognition and language understanding, we propose a unified neural architecture for visual recognition and reasoning with a generic interface (e.g., tokens) for both. Our framework enables the principled investigation of how different visual recognition tasks, datasets, and inductive biases can help enable spatiotemporal reasoning capabilities. Noticeably, we find that object detection, which requires spatial localization of individual objects, is the most beneficial recognition task for reasoning. We further demonstrate via probing that implicit object-centric representations emerge automatically inside our framework. Intriguingly, we discover that certain architectural choices such as the backbone model of the visual encoder have a significant impact on visual reasoning, but little on object detection. Given the results of our experiments, we believe that visual reasoning should be considered as a first-class citizen alongside visual recognition, as they are strongly correlated but benefit from potentially different design choices

Surface Functionalization of Black Phosphorus via Amine Compounds and Its Impacts on the Flame Retardancy and Thermal Decomposition Behaviors of Epoxy Resin.

Recently, lots of effort has been placed into stabilizing black phosphorus (BP) in the air to improve its compatibility with polymers. Herein, BP was chemically functionalized by aliphatic amine (DETA), aromatic amine (PPDA) and cyclamine (Pid) via a nucleophilic substitution reaction, aiming to develop an intensively reactive BP flame retardant for epoxy resin (EP). The -NH2 group on BP-DETA, BP-PPDA and BP-Pid reacted with the epoxide group at different temperatures. The lowest temperature was about 150 °C for BP-DETA. The impacts of three BP-NH2 were compared on the flame retardancy and thermal decomposition of EP. At 5 wt% loading, EP/BP-NH2 all passed UL 94 V 0 rating. The limiting oxygen index (LOI) of EP/BP-PPDA was as high as 32.3%. The heat release rate (HRR) of EP/BP-DETA greatly decreased by 46% and char residue increased by 73.8%, whereas HRR of EP/BP-Pid decreased by 11.5% and char residue increased by 50.8%, compared with EP. Average effective heat of combustion (av-EHC) of EP/BP-Pid was lower than that of EP/BP-DETA and EP/BP-PPDA. In view of the flame-retardant mechanism, BP nanosheets functionalized with aliphatic amine and aromatic amine played a dominant role in the condensed phase, while BP functionalized with cyclamine was more effective in the gas phase.post-print74881 K

Federated Learning of Shareable Bases for Personalization-Friendly Image Classification

Personalized federated learning (PFL) aims to harness the collective wisdom of clients' data while building personalized models tailored to individual clients' data distributions. Existing works offer personalization primarily to clients who participate in the FL process, making it hard to encompass new clients who were absent or newly show up. In this paper, we propose FedBasis, a novel PFL framework to tackle such a deficiency. FedBasis learns a set of few shareable ``basis'' models, which can be linearly combined to form personalized models for clients. Specifically for a new client, only a small set of combination coefficients, not the model weights, needs to be learned. This notion makes FedBasis more parameter-efficient, robust, and accurate than competitive PFL baselines, especially in the low data regime, without increasing the inference cost. To demonstrate the effectiveness and applicability of FedBasis, we also present a more practical PFL testbed for image classification, featuring larger data discrepancies across clients in both the image and label spaces as well as more faithful training and test splits.Comment: Preprin

SoC-Cluster as an Edge Server: an Application-driven Measurement Study

Huge electricity consumption is a severe issue for edge data centers. To this end, we propose a new form of edge server, namely SoC-Cluster, that orchestrates many low-power mobile system-on-chips (SoCs) through an on-chip network. For the first time, we have developed a concrete SoC-Cluster server that consists of 60 Qualcomm Snapdragon 865 SoCs in a 2U rack. Such a server has been commercialized successfully and deployed in large scale on edge clouds. The current dominant workload on those deployed SoC-Clusters is cloud gaming, as mobile SoCs can seamlessly run native mobile games. The primary goal of this work is to demystify whether SoC-Cluster can efficiently serve more general-purpose, edge-typical workloads. Therefore, we built a benchmark suite that leverages state-of-the-art libraries for two killer edge workloads, i.e., video transcoding and deep learning inference. The benchmark comprehensively reports the performance, power consumption, and other application-specific metrics. We then performed a thorough measurement study and directly compared SoC-Cluster with traditional edge servers (with Intel CPU and NVIDIA GPU) with respect to physical size, electricity, and billing. The results reveal the advantages of SoC-Cluster, especially its high energy efficiency and the ability to proportionally scale energy consumption with various incoming loads, as well as its limitations. The results also provide insightful implications and valuable guidance to further improve SoC-Cluster and land it in broader edge scenarios

Meta-Analysis: Overweight, Obesity, and Parkinson's Disease

Objective. Parkinson's disease (PD) is a severe neurological disease and its risk factors remain largely unknown. A meta-analysis was carried out to investigate the relationship of overweight and obesity with PD. Methods. We used PubMed, EMBASE, and the Chinese National Knowledge Infrastructure (CNKI) databases to identify studies of associations between overweight/obesity and PD. Overweight, obesity, and PD were used as keywords, and published works were retrieved until September 30, 2013. The extracted data were classified (BMI≥30,25≤BMI<30,  and BMI<25) according to BMI values and analyzed using RevMan5.2 and Stata11.0. Results. Four cohort studies and three case-control studies were used to evaluate the association between overweight/obesity and PD, including 2857 PD patients and 5, 683, 939 cases of non-PD controls. There was a statistically significant difference between 25≤BMI<30  and BMI<25 in the cohort study (RR=1.17, 95% CI, 1.03–1.32,  P=0.03), but there was no difference between BMI≥30  and BMI<25 or BMI≥30  and 25≤BMI<30, where the respective RR was 1.16 and 0.84; the respective 95% CI was 0.67–2.01 and 0.61–1.15, respectively, and the P values were 0.60 and 0.28, respectively. Case-control studies showed that there was no statistical difference between any two groups. Conclusion. Meta-analysis showed that overweight might be a potential risk factor of PD. Demonstration of a causal role of overweight/obesity in PD development could have important therapeutic implications