3D Human Activity Recognition with Reconfigurable Convolutional Neural Networks

Human activity understanding with 3D/depth sensors has received increasing attention in multimedia processing and interactions. This work targets on developing a novel deep model for automatic activity recognition from RGB-D videos. We represent each human activity as an ensemble of cubic-like video segments, and learn to discover the temporal structures for a category of activities, i.e. how the activities to be decomposed in terms of classification. Our model can be regarded as a structured deep architecture, as it extends the convolutional neural networks (CNNs) by incorporating structure alternatives. Specifically, we build the network consisting of 3D convolutions and max-pooling operators over the video segments, and introduce the latent variables in each convolutional layer manipulating the activation of neurons. Our model thus advances existing approaches in two aspects: (i) it acts directly on the raw inputs (grayscale-depth data) to conduct recognition instead of relying on hand-crafted features, and (ii) the model structure can be dynamically adjusted accounting for the temporal variations of human activities, i.e. the network configuration is allowed to be partially activated during inference. For model training, we propose an EM-type optimization method that iteratively (i) discovers the latent structure by determining the decomposed actions for each training example, and (ii) learns the network parameters by using the back-propagation algorithm. Our approach is validated in challenging scenarios, and outperforms state-of-the-art methods. A large human activity database of RGB-D videos is presented in addition.Comment: This manuscript has 10 pages with 9 figures, and a preliminary version was published in ACM MM'14 conferenc

Barriers to the transition towards off-site construction in China: An Interpretive structural modeling approach

Off-site construction (OSC) has been recognized as an approach to transform the construction sector from a labor–intensive to a modernized and green industry. Despite a number of advantages, the development of OSC still remains its infancy in China due to various interactive barriers. Some studies have been conducted to explore the barriers to the OSC adoption. However, very few studies attempted to investigate the complex interrelationships among these barriers. In order to fill this gap, this study adopts Interpretive Structural Model (ISM) technique to explore the interrelationships amongst barriers to the OSC adoption in China. Firstly, critical barriers were identified through literature review and semi-structured interviews with various stakeholders. Then, the overall structure amongst barriers was revealed through ISM technique. By using the Matriced’ Impacts Croise's Multiplication Appliquée a UN Classement (MICMAC) technique, the barriers were classified into four groups according to their driving-power and dependence power. The results indicate that specific attentions should be given to inadequate policy and regulations, lacking knowledge and expertise, dominated traditional project process as well as low standardization. The research findings provide valuable information for policy-makers on the overall structure amongst barriers. These results shed lights on effectively developing measures to facilitate the OSC adoption in the construction sector

Potential Influences of Volcanic Eruptions on Future Global Land Monsoon Precipitation Changes

The global monsoon system is of exceptional socioeconomic importance owing to its impacts on two-thirds of the globe’s population. Major volcanic eruptions strongly influence global land monsoon (GLM) precipitation change. By using 60 plausible eruption scenarios sampled from reconstructed volcanic proxies over the past 2,500 years, 21st century volcanic influences on GLM precipitation projections are examined with an Earth system model under a moderate emission scenario. The decadal-scale ensemble spread with realistic eruptions (VOLC) increases by 17.5% and 20.1% compared to no-volcanic (NO-VOLC) and constant background-volcanic (VOLC-CONST) scenarios, respectively. Compared with NO-VOLC, the centennial mean VOLC GLM precipitation shows a 10% overall reduction and regionally, Asia is the most impacted. Changes in atmospheric circulation in the aftermath of large volcanic eruptions match the global warming response patterns well with opposite sign, with the North American monsoon precipitation enhanced following large volcanic eruptions, which is in sharp contrast to the robust decrease in Asian monsoon rainfall. Volcanic activity could delay the time of emergence of anthropogenic influence by five years on average over about 60% of the GLM area. Our results demonstrate the importance of statistical representation of potential volcanism for the projections of future monsoon variability. Quantifying volcanic impacts on regional climate projections and their socioeconomic influences on infrastructure planning, food security, and disaster management should be a priority of future work.publishedVersio

The signs of computer tomography combined with artificial intelligence can indicate the correlation between status of consciousness and primary brainstem hemorrhage of patients

BackgroundFor patients of primary brainstem hemorrhage (PBH), it is crucial to find a method that can quickly and accurately predict the correlation between status of consciousness and PBH.ObjectiveTo analyze the value of computer tomography (CT) signs in combination with artificial intelligence (AI) technique in predicting the correlation between status of consciousness and PBH.MethodsA total of 120 patients with PBH were enrolled from August 2011 to March 2021 according to the criteria. Patients were divided into three groups [consciousness, minimally conscious state (MCS) and coma] based on the status of consciousness. Then, first, Mann–Whitney U test and Spearman rank correlation test were used on the factors: gender, age, stages of intracerebral hemorrhage, CT signs with AI or radiology physicians, hemorrhage involving the midbrain or ventricular system. We collected hemorrhage volumes and mean CT values with AI. Second, those significant factors were screened out by the Mann–Whitney U test and those highly or moderately correlated by Spearman’s rank correlation test, and a further ordinal multinomial logistic regression analysis was performed to find independent predictors of the status of consciousness. At last, receiver operating characteristic (ROC) curves were drawn to calculate the hemorrhage volume for predictively assessing the status of consciousness.ResultsPreliminary meaningful variables include hemorrhage involving the midbrain or ventricular system, hemorrhage volume, grade of hematoma shape and density, and CT value from Mann–Whitney U test and Spearman rank correlation test. It is further shown by ordinal multinomial logistic regression analysis that hemorrhage volume and hemorrhage involving the ventricular system are two major predictors of the status of consciousness. It showed from ROC that the hemorrhage volumes of <3.040 mL, 3.040 ~ 6.225 mL and >6.225 mL correspond to consciousness, MCS or coma, respectively. If the hemorrhage volume is the same, hemorrhage involving the ventricular system should be correlated with more severe disorders of consciousness (DOC).ConclusionCT signs combined with AI can predict the correlation between status of consciousness and PBH. Hemorrhage volume and hemorrhage involving the ventricular system are two independent factors, with hemorrhage volume in particular reaching quantitative predictions

KnowCoder: Coding Structured Knowledge into LLMs for Universal Information Extraction

In this paper, we propose KnowCoder, a Large Language Model (LLM) to conduct Universal Information Extraction (UIE) via code generation. KnowCoder aims to develop a kind of unified schema representation that LLMs can easily understand and an effective learning framework that encourages LLMs to follow schemas and extract structured knowledge accurately. To achieve these, KnowCoder introduces a code-style schema representation method to uniformly transform different schemas into Python classes, with which complex schema information, such as constraints among tasks in UIE, can be captured in an LLM-friendly manner. We further construct a code-style schema library covering over $\textbf{30,000}$ types of knowledge, which is the largest one for UIE, to the best of our knowledge. To ease the learning process of LLMs, KnowCoder contains a two-phase learning framework that enhances its schema understanding ability via code pretraining and its schema following ability via instruction tuning. After code pretraining on around $1.5$B automatically constructed data, KnowCoder already attains remarkable generalization ability and achieves relative improvements by \textbf{49.8%} F1, compared to LLaMA2, under the few-shot setting. After instruction tuning, KnowCoder further exhibits strong generalization ability on unseen schemas and achieves up to \textbf{12.5%} and \textbf{21.9%}, compared to sota baselines, under the zero-shot setting and the low resource setting, respectively. Additionally, based on our unified schema representations, various human-annotated datasets can simultaneously be utilized to refine KnowCoder, which achieves significant improvements up to \textbf{7.5%} under the supervised setting

A modified deep neural network enables identification of foliage under complex background

For the sake of enhancing the identification ability of current network and meeting the needs of the high accuracy of distinguishing similar small objects (foliage) in the complex scenes, this paper proposes a modified region-based fully convolutional network which adopts Inception V3 accompanying with residual connection as the main framework. Incorporating deep residual learning module into Inception V3 can not only save the computational cost by factorising convolutions, but also mitigate the vanishing gradients causing the increasing depth of the network. Additionally, this combination can alleviate the degradation problem in the process of extracting features and providing proposals. Experimental results show that the modified approach can identify out different leaves with similar characteristics in one scene, and demonstrate the superiority of our proposed approach over some state-of-the-art deep neural networks, when it comes to recognise foliage in complicated environments

CGMNet: Semantic Change Detection via a Change-Aware Guided Multi-Task Network

Change detection (CD) is the main task in the remote sensing field. Binary change detection (BCD), which only focuses on the region of change, cannot meet current needs. Semantic change detection (SCD) is pivotal for identifying regions of change in sequential remote sensing imagery, focusing on discerning “from-to” transitions in land cover. The emphasis on features within these regions of change is critical for SCD efficacy. Traditional methodologies, however, often overlook this aspect. In order to address this gap, we introduce a change-aware guided multi-task network (CGMNet). This innovative network integrates a change-aware mask branch, leveraging prior knowledge of regions of change to enhance land cover classification in dual temporal remote sensing images. This strategic focus allows for the more accurate identification of altered regions. Furthermore, to navigate the complexities of remote sensing environments, we develop a global and local attention mechanism (GLAM). This mechanism adeptly captures both overarching and fine-grained spatial details, facilitating more nuanced analysis. Our rigorous testing on two public datasets using state-of-the-art methods yielded impressive results. CGMNet achieved Overall Score metrics of 58.77% on the Landsat-SCD dataset and 37.06% on the SECOND dataset. These outcomes not only demonstrate the exceptional performance of the method but also signify its superiority over other comparative algorithms