Object-oriented Neural Programming (OONP) for Document Understanding

We propose Object-oriented Neural Programming (OONP), a framework for semantically parsing documents in specific domains. Basically, OONP reads a document and parses it into a predesigned object-oriented data structure (referred to as ontology in this paper) that reflects the domain-specific semantics of the document. An OONP parser models semantic parsing as a decision process: a neural net-based Reader sequentially goes through the document, and during the process it builds and updates an intermediate ontology to summarize its partial understanding of the text it covers. OONP supports a rich family of operations (both symbolic and differentiable) for composing the ontology, and a big variety of forms (both symbolic and differentiable) for representing the state and the document. An OONP parser can be trained with supervision of different forms and strength, including supervised learning (SL) , reinforcement learning (RL) and hybrid of the two. Our experiments on both synthetic and real-world document parsing tasks have shown that OONP can learn to handle fairly complicated ontology with training data of modest sizes.Comment: accepted by ACL 201

Whole-Body Lesion Segmentation in 18F-FDG PET/CT

There has been growing research interest in using deep learning based method to achieve fully automated segmentation of lesion in Positron emission tomography computed tomography(PET CT) scans for the prognosis of various cancers. Recent advances in the medical image segmentation shows the nnUNET is feasible for diverse tasks. However, lesion segmentation in the PET images is not straightforward, because lesion and physiological uptake has similar distribution patterns. The Distinction of them requires extra structural information in the CT images. The present paper introduces a nnUNet based method for the lesion segmentation task. The proposed model is designed on the basis of the joint 2D and 3D nnUNET architecture to predict lesions across the whole body. It allows for automated segmentation of potential lesions. We evaluate the proposed method in the context of AutoPet Challenge, which measures the lesion segmentation performance in the metrics of dice score, false-positive volume and false-negative volume

Large inter-city inequality in consumption-based CO₂ emissions for China's pearl river basin cities

Cities are leading carbon mitigation but are heterogeneous in their mitigation policies due to different socioeconomic backgrounds. Given that cities are increasingly inextricably linked, formulating mitigation policies of different cities cannot be easily achieved without comprehensive carbon inventories, who taking the inter-city supply chains into account. The Pearl River Basin is one of the important economic zones in China, with huge disparity in its cities, but very limited information is available on their consumption-based CO2 emissions. To fill this gap, we compiled a consumption-based inventory of 47 cities in the Basin for 2012. We found that the total consumption-based emissions of 47 cities was 933.8 Mt, accounting for 13.1% of China's emissions. There were huge differences in the consumption-based emissions, ranging from 3.6 Mt (Heyuan City) to 153.1 Mt (Shenzhen City). The consumption-based emissions were highly concentrated in the largest seven cities, which accounted for 52.8% of the total emissions of the Basin. The consumption-based emissions per capita also varied greatly, from 1.2 to 14.5 tons per capita. Large scale infrastructure was the biggest driving force for most cities, resulting in 42.1% to 75.6% of the emissions. At sector-level, construction, heavy industry and services were leading in emissions, contributing more than 80% of emissions. The major inter-city carbon transfers occurred within upstream cities in the developing regions and downstream cities in the Pearl River Delta respectively, instead of the transfers between upstream and downstream cities. The findings highlight that the regional mitigation strategies could mainly focus on cities in intra-province boundary, rather than inter-province boundary, and also the city-level mitigation strategies should pay attention to the key emission sectors and drivers in respect of the heterogeneity of cities

Improving BERT with Hybrid Pooling Network and Drop Mask

Transformer-based pre-trained language models, such as BERT, achieve great success in various natural language understanding tasks. Prior research found that BERT captures a rich hierarchy of linguistic information at different layers. However, the vanilla BERT uses the same self-attention mechanism for each layer to model the different contextual features. In this paper, we propose a HybridBERT model which combines self-attention and pooling networks to encode different contextual features in each layer. Additionally, we propose a simple DropMask method to address the mismatch between pre-training and fine-tuning caused by excessive use of special mask tokens during Masked Language Modeling pre-training. Experiments show that HybridBERT outperforms BERT in pre-training with lower loss, faster training speed (8% relative), lower memory cost (13% relative), and also in transfer learning with 1.5% relative higher accuracies on downstream tasks. Additionally, DropMask improves accuracies of BERT on downstream tasks across various masking rates.Comment: 7 pages, 2 figure

Adaptive gradient-based block compressive sensing with sparsity for noisy images

This paper develops a novel adaptive gradient-based block compressive sensing (AGbBCS_SP) methodology for noisy image compression and reconstruction. The AGbBCS_SP approach splits an image into blocks by maximizing their sparsity, and reconstructs images by solving a convex optimization problem. In block compressive sensing, the commonly used square block shapes cannot always produce the best results. The main contribution of our paper is to provide an adaptive method for block shape selection, improving noisy image reconstruction performance. The proposed algorithm can adaptively achieve better results by using the sparsity of pixels to adaptively select block shape. Experimental results with different image sets demonstrate that our AGbBCS_SP method is able to achieve better performance, in terms of peak signal to noise ratio (PSNR) and computational cost, than several classical algorithms