Novel-view Synthesis and Pose Estimation for Hand-Object Interaction from Sparse Views

Hand-object interaction understanding and the barely addressed novel view synthesis are highly desired in the immersive communication, whereas it is challenging due to the high deformation of hand and heavy occlusions between hand and object. In this paper, we propose a neural rendering and pose estimation system for hand-object interaction from sparse views, which can also enable 3D hand-object interaction editing. We share the inspiration from recent scene understanding work that shows a scene specific model built beforehand can significantly improve and unblock vision tasks especially when inputs are sparse, and extend it to the dynamic hand-object interaction scenario and propose to solve the problem in two stages. We first learn the shape and appearance prior knowledge of hands and objects separately with the neural representation at the offline stage. During the online stage, we design a rendering-based joint model fitting framework to understand the dynamic hand-object interaction with the pre-built hand and object models as well as interaction priors, which thereby overcomes penetration and separation issues between hand and object and also enables novel view synthesis. In order to get stable contact during the hand-object interaction process in a sequence, we propose a stable contact loss to make the contact region to be consistent. Experiments demonstrate that our method outperforms the state-of-the-art methods. Code and dataset are available in project webpage https://iscas3dv.github.io/HO-NeRF

CD13 Inhibition Enhances Cytotoxic Effect of Chemotherapy Agents

Multidrug resistance (MDR) of hepatocellular carcinoma is a serious problem. Although CD13 is a biomarker in human liver cancer stem cells, the relationship between CD13 and MDR remains uncertain. This study uses liver cancer cell model to understand the role of CD13 in enhancing the cytotoxic effect of chemotherapy agents. Cytotoxic agents can induce CD13 expression. CD13 inhibitor, bestatin, enhances the antitumor effect of cytotoxic agents. Meanwhile, CD13-targeting siRNA and neutralizing antibody can enhance the cytotoxic effect of 5-fluorouracil (5FU). CD13 overexpression increases cell survival upon cytotoxic agents treatment, while the knockdown of CD13 causes hypersensitivity of cells to cytotoxic agents treatment. Mechanistically, the inhibition of CD13 leads to the increase of cellular reactive oxygen species (ROS). BC-02 is a novel mutual prodrug (hybrid drug) of bestatin and 5FU. Notably, BC-02 can inhibit cellular activity in both parental and drug-resistant cells, accompanied with significantly increased ROS level. Moreover, the survival time of Kunming mice bearing H22 cells under BC-02 treatment is comparable to the capecitabine treatment at maximum dosage. These data implicate a therapeutic method to reverse MDR by targeting CD13, and indicate that BC-02 is a potent antitumor compound

Fine-Grained Video Retrieval With Scene Sketches

Benefiting from the intuitiveness and naturalness of sketch interaction, sketch-based video retrieval (SBVR) has received considerable attention in the video retrieval research area. However, most existing SBVR research still lacks the capability of accurate video retrieval with fine-grained scene content. To address this problem, in this paper we investigate a new task, which focuses on retrieving the target video by utilizing a fine-grained storyboard sketch depicting the scene layout and major foreground instances’ visual characteristics (e.g., appearance, size, pose, etc.) of video; we call such a task “fine-grained scene-level SBVR”. The most challenging issue in this task is how to perform scene-level cross-modal alignment between sketch and video. Our solution consists of two parts. First, we construct a scene-level sketch-video dataset called SketchVideo, in which sketch-video pairs are provided and each pair contains a clip-level storyboard sketch and several keyframe sketches (corresponding to video frames). Second, we propose a novel deep learning architecture called Sketch Query Graph Convolutional Network (SQ-GCN). In SQ-GCN, we first adaptively sample the video frames to improve video encoding efficiency, and then construct appearance and category graphs to jointly model visual and semantic alignment between sketch and video. Experiments show that our fine-grained scene-level SBVR framework with SQ-GCN architecture outperforms the state-of-the-art fine-grained retrieval methods. The SketchVideo dataset and SQ-GCN code are available in the project webpage https://iscas-mmsketch.github.io/FG-SL-SBVR/

robust maximum likelihood estimation by sparse bundle adjustment using the l1 norm

Sparse bundle adjustment is widely used in many computer vision applications. In this paper, we propose a method for performing bundle adjustments using the L1 norm. After linearizing the mapping function in bundle adjustment on its first order, the kernel step is to compute the L1 norm equations. Considering the sparsity of the Jacobian matrix in linearizing, we find two practical methods to solve the L1 norm equations. The first one is an interior-point method, which transfer the original problem to a problem of solving a sequence of L2 norm equations, and the second one is a decomposition method which uses the differentiability of linear programs and represents the optimal updating of parameters of 3D points by the updating variables of camera parameters. The experiments show that the method performs better for both synthetically generated and real data sets in the presence of outliers or Laplacian noise compared with the L2 norm bundle adjustment, and the method is efficient among the state of the art L1 minimization methods. © 2012 IEEE.IEEESparse bundle adjustment is widely used in many computer vision applications. In this paper, we propose a method for performing bundle adjustments using the L1 norm. After linearizing the mapping function in bundle adjustment on its first order, the kernel step is to compute the L1 norm equations. Considering the sparsity of the Jacobian matrix in linearizing, we find two practical methods to solve the L1 norm equations. The first one is an interior-point method, which transfer the original problem to a problem of solving a sequence of L2 norm equations, and the second one is a decomposition method which uses the differentiability of linear programs and represents the optimal updating of parameters of 3D points by the updating variables of camera parameters. The experiments show that the method performs better for both synthetically generated and real data sets in the presence of outliers or Laplacian noise compared with the L2 norm bundle adjustment, and the method is efficient among the state of the art L1 minimization methods. © 2012 IEEE

t-maze: a tangible programming tool for children

ACM Special Interest Group on Computer-Human Interaction (SIGCHI); University of MichiganThis paper presents a tangible programming tool 'T-Maze' for children aged 5 to 9. Children could use T-Maze to create their own maze maps and complete some maze escaping tasks by the tangible programming blocks and sensors. T-Maze uses a camera to, in real-time, catch the programming sequence of the wooden blocks' arrangement, which will be used to analyze the semantic correctness and enable the children to receive feedbacks immediately. And children could join in the game by controlling the sensors during program's running. A user study shows that T-Maze is an interesting programming approach for children and easy to learn and use. © 2011 ACM

Ground Subsidence over Beijing-Tianjin-Hebei Region during Three Periods of 1992 to 2014 Monitored by Interferometric SAR

The Beijing-Tianjin-Hebei region suffers the most serious ground subsidence in China, which has caused huge economic losses every year. Therefore, ground subsidence was listed as an important mission in the project of geographic conditions monitoring over Beijing-Tianjin-Hebei launched by the National Administration of Surveying, Mapping and Geoinformation in 2013. In this paper, we propose a methodology of ground subsidence monitoring over wide area, which is entitled "multiple master-image coherent target small-baseline interferometric SAR (MCTSB-InSAR)". MCTSB-InSAR is an improved time series InSAR technique with some unique features. SAR datasets used for ground subsidence monitoring over the Beijing-Tianjin-Hebei region include ERS-1/2 SAR images acquired between 1992 to 2000, ENVISAT ASAR images acquired between 2003 to 2010 and RADARSAT-2 images acquired between 2012 to 2014. This research represents a first ever effort on mapping ground subsidence over Beijing-Tianjin-Hebei region and over such as a long time span in China. In comparison with more than 120 leveling measurements collected in Beijing and Tianjin, the derived subsidence velocity has the accuracy of 8.7mm/year (1992—2000), 4.7mm/year (2003—2010), and 5.4mm/year (2012—2014) respectively. The spatial-temporal characteristics of the development of ground subsidence in Beijing and Tianjin are analyzed. In general, ground subsidence in Beijing kept continuously expanding in the period of 1992 to 2014. While, ground subsidence in Tianjin had already been serious in 1990s, had dramatically expanded during 2000s, and started to alleviate in recent years. The monitoring result is of high significance for prevention and mitigation of ground subsidence disaster, for making development plan, for efficient and effective utilization of water resource, and for adjustment of economic framework of this region. The result also indicates the effectiveness and reliability of the MCTSB-InSAR method. Thus, the MCTSB-InSAR method is applicable to monitoring ground subsidence over large areas in the future

Analysis of Land Use/Cover Dynamic Change in XiŽan, using Modified BP Neural Network Classification for Remote Sensing Images

Land use/cover change (LUCC) is one of the most important fields of global environment change research. It reveals the evolution process of natural ecological environment influenced by human activity. In this paper, land use and cover in XiŽan region in 2000 and 2003 were achieved by modified back propagation neutral network (BPNN) using remotely sensed data Landsat TM/ETM and DEM. Meanwhile the result of the BPNN classification was compared with the Maximum Likelihood Classification (MLC). The result showed that the BPNN had a higher accuracy. By analysing the results, we drew conclusions as follows: (1) the classification accuracy of BP neutral network was improved obviously; (2) the area of urban built-up land increased quickly, and its annual increasing rate was 12% from 2000 to 2003; (3) the area of woodland also rose fast whose annual increasing rate was 2.48%; (4) the increasing rate of orchard was astonishing which reached to 15.30% per year. These figures showed that since the great development of western China, the economy in XiŽan region had grown very rapidly and the ecological environment here had been also improved a lot.vokMyynti MTT tietopalvelu

a novel fast method for l∞ problems in multiview geometry

Optimization using the L∞ norm is an increasingly important area in multiview geometry. Previous work has shown that globally optimal solutions can be computed reliably using the formulation of generalized fractional programming, in which algorithms solve a sequence of convex problems independently to approximate the optimal L∞ norm error. We found the sequence of convex problems are highly related and we propose a method to derive a Newton-like step from any given point. In our method, the feasible region of the current involved convex problem is contracted gradually along with the Newton-like steps, and the updated point locates on the boundary of the new feasible region. We propose an effective strategy to make the boundary point become an interior point through one dimension augmentation and relaxation. Results are presented and compared to the state of the art algorithms on simulated and real data for some multiview geometry problems with improved performance on both runtime and Newton-like iterations. © 2012 Springer-Verlag.Google; National Robotics Engineering Center (NREC); Adobe; Microsoft Research; Mitsubishi ElectricOptimization using the L∞ norm is an increasingly important area in multiview geometry. Previous work has shown that globally optimal solutions can be computed reliably using the formulation of generalized fractional programming, in which algorithms solve a sequence of convex problems independently to approximate the optimal L∞ norm error. We found the sequence of convex problems are highly related and we propose a method to derive a Newton-like step from any given point. In our method, the feasible region of the current involved convex problem is contracted gradually along with the Newton-like steps, and the updated point locates on the boundary of the new feasible region. We propose an effective strategy to make the boundary point become an interior point through one dimension augmentation and relaxation. Results are presented and compared to the state of the art algorithms on simulated and real data for some multiview geometry problems with improved performance on both runtime and Newton-like iterations. © 2012 Springer-Verlag