JALAD: Joint Accuracy- and Latency-Aware Deep Structure Decoupling for Edge-Cloud Execution

Recent years have witnessed a rapid growth of deep-network based services and applications. A practical and critical problem thus has emerged: how to effectively deploy the deep neural network models such that they can be executed efficiently. Conventional cloud-based approaches usually run the deep models in data center servers, causing large latency because a significant amount of data has to be transferred from the edge of network to the data center. In this paper, we propose JALAD, a joint accuracy- and latency-aware execution framework, which decouples a deep neural network so that a part of it will run at edge devices and the other part inside the conventional cloud, while only a minimum amount of data has to be transferred between them. Though the idea seems straightforward, we are facing challenges including i) how to find the best partition of a deep structure; ii) how to deploy the component at an edge device that only has limited computation power; and iii) how to minimize the overall execution latency. Our answers to these questions are a set of strategies in JALAD, including 1) A normalization based in-layer data compression strategy by jointly considering compression rate and model accuracy; 2) A latency-aware deep decoupling strategy to minimize the overall execution latency; and 3) An edge-cloud structure adaptation strategy that dynamically changes the decoupling for different network conditions. Experiments demonstrate that our solution can significantly reduce the execution latency: it speeds up the overall inference execution with a guaranteed model accuracy loss.Comment: conference, copyright transfered to IEE

AdaCompress: Adaptive Compression for Online Computer Vision Services

With the growth of computer vision based applications and services, an explosive amount of images have been uploaded to cloud servers which host such computer vision algorithms, usually in the form of deep learning models. JPEG has been used as the {\em de facto} compression and encapsulation method before one uploads the images, due to its wide adaptation. However, standard JPEG configuration does not always perform well for compressing images that are to be processed by a deep learning model, e.g., the standard quality level of JPEG leads to 50\% of size overhead (compared with the best quality level selection) on ImageNet under the same inference accuracy in popular computer vision models including InceptionNet, ResNet, etc. Knowing this, designing a better JPEG configuration for online computer vision services is still extremely challenging: 1) Cloud-based computer vision models are usually a black box to end-users; thus it is difficult to design JPEG configuration without knowing their model structures. 2) JPEG configuration has to change when different users use it. In this paper, we propose a reinforcement learning based JPEG configuration framework. In particular, we design an agent that adaptively chooses the compression level according to the input image's features and backend deep learning models. Then we train the agent in a reinforcement learning way to adapt it for different deep learning cloud services that act as the {\em interactive training environment} and feeding a reward with comprehensive consideration of accuracy and data size. In our real-world evaluation on Amazon Rekognition, Face++ and Baidu Vision, our approach can reduce the size of images by 1/2 -- 1/3 while the overall classification accuracy only decreases slightly.Comment: ACM Multimedi

WGIT*: Workspace-Guided Informed Tree for Motion Planning in Restricted Environments

The motion planning of robots faces formidable challenges in restricted environments, particularly in the aspects of rapidly searching feasible solutions and converging towards optimal solutions. This paper proposes Workspace-guided Informed Tree (WGIT*) to improve planning efficiency and ensure high-quality solutions in restricted environments. Specifically, WGIT* preprocesses the workspace by constructing a hierarchical structure to obtain critical restricted regions and connectivity information sequentially. The refined workspace information guides the sampling and exploration of WGIT*, increasing the sample density in restricted areas and prioritizing the search tree exploration in promising directions, respectively. Furthermore, WGIT* utilizes gradually enriched configuration space information as feedback to rectify the guidance from the workspace and balance the information of the two spaces, which leads to efficient convergence toward the optimal solution. The theoretical analysis highlights the valuable properties of the proposed WGIT*. Finally, a series of simulations and experiments verify the ability of WGIT* to quickly find initial solutions and converge towards optimal solutions

WGIT*: Workspace-Guided Informed Tree for Motion Planning in Restricted Environments

The motion planning of robots faces formidable challenges in restricted environments, particularly in the aspects of rapidly searching feasible solutions and converging towards optimal solutions. This paper proposes Workspace-guided Informed Tree (WGIT*) to improve planning efficiency and ensure high-quality solutions in restricted environments. Specifically, WGIT* preprocesses the workspace by constructing a hierarchical structure to obtain critical restricted regions and connectivity information sequentially. The refined workspace information guides the sampling and exploration of WGIT*, increasing the sample density in restricted areas and prioritizing the search tree exploration in promising directions, respectively. Furthermore, WGIT* utilizes gradually enriched configuration space information as feedback to rectify the guidance from the workspace and balance the information of the two spaces, which leads to efficient convergence toward the optimal solution. The theoretical analysis highlights the valuable properties of the proposed WGIT*. Finally, a series of simulations and experiments verify the ability of WGIT* to quickly find initial solutions and converge towards optimal solutions

Deep Learning based 3D Segmentation: A Survey

3D object segmentation is a fundamental and challenging problem in computer vision with applications in autonomous driving, robotics, augmented reality and medical image analysis. It has received significant attention from the computer vision, graphics and machine learning communities. Traditionally, 3D segmentation was performed with hand-crafted features and engineered methods which failed to achieve acceptable accuracy and could not generalize to large-scale data. Driven by their great success in 2D computer vision, deep learning techniques have recently become the tool of choice for 3D segmentation tasks as well. This has led to an influx of a large number of methods in the literature that have been evaluated on different benchmark datasets. This paper provides a comprehensive survey of recent progress in deep learning based 3D segmentation covering over 150 papers. It summarizes the most commonly used pipelines, discusses their highlights and shortcomings, and analyzes the competitive results of these segmentation methods. Based on the analysis, it also provides promising research directions for the future.Comment: Under review of ACM Computing Surveys, 36 pages, 10 tables, 9 figure

Exponential stability of Cohen-Grossberg neural networks with multiple time-varying delays and distributed delays

Maybe because Cohen-Grossberg neural networks with multiple time-varying delays and distributed delays cannot be converted into the vector-matrix forms, the stability results of such networks are relatively few and the stability conditions in the linear matrix inequality forms have not been established. So this paper investigates the exponential stability of the networks and gives the sufficient condition in the linear matrix inequality forms. Two examples are provided to demonstrate the effectiveness of the theoretical results

First Place Solution to the CVPR'2023 AQTC Challenge: A Function-Interaction Centric Approach with Spatiotemporal Visual-Language Alignment

Affordance-Centric Question-driven Task Completion (AQTC) has been proposed to acquire knowledge from videos to furnish users with comprehensive and systematic instructions. However, existing methods have hitherto neglected the necessity of aligning spatiotemporal visual and linguistic signals, as well as the crucial interactional information between humans and objects. To tackle these limitations, we propose to combine large-scale pre-trained vision-language and video-language models, which serve to contribute stable and reliable multimodal data and facilitate effective spatiotemporal visual-textual alignment. Additionally, a novel hand-object-interaction (HOI) aggregation module is proposed which aids in capturing human-object interaction information, thereby further augmenting the capacity to understand the presented scenario. Our method achieved first place in the CVPR'2023 AQTC Challenge, with a Recall@1 score of 78.7\%. The code is available at https://github.com/tomchen-ctj/CVPR23-LOVEU-AQTC.Comment: Winner of CVPR2023 Long-form Video Understanding and Generation Challenge (Track 3

Progastrin stimulates colonic cell proliferation via CCK2R- and β-arrestin-dependent suppression of BMP2

Background & Aims Progastrin stimulates colonic mucosal proliferation and carcinogenesis through the cholecystokinin 2 receptor (CCK2R) - partly by increasing the number of colonic progenitor cells. However, little is known about the mechanisms by which progastrin stimulates colonic cell proliferation. We investigated the role of bone morphogenetic proteins (BMPs) in progastrin induction of colonic cell proliferation via CCK2R. Methods We performed microarray analysis to compare changes in gene expression in the colonic mucosa of mice that express a human progastrin transgene, gastrin knockout mice, and C57BL/6 mice (controls); the effects of progastrin were also determined on in vitro colonic crypt cultures from cholecystokinin 2 receptor knockout and wild-type mice. Human colorectal and gastric cancer cells that expressed CCK2R were incubated with progastrin or Bmp2; levels of β-arrestin 1 and 2 were knocked down using small interfering RNAs. Cells were analyzed for progastrin binding, proliferation, changes in gene expression, and symmetric cell division. Results The BMP pathway was down-regulated in the colons of human progastrin mice compared with controls. Progastrin suppressed transcription of Bmp2 through a pathway that required CCK2R and was mediated by β-arrestin 1 and 2. In mouse colonic epithelial cells, down-regulation of Bmp2 led to decreased phosphorylation of Smads1/5/8 and suppression of inhibitor of DNA binding 4. In human gastric and colorectal cancer cell lines, CCK2R was necessary and sufficient for progastrin binding and induction of proliferation; these effects were blocked when cells were incubated with recombinant Bmp2. Incubation with progastrin increased the number of CD44+, bromodeoxyuridine+, and NUMB+ cells, indicating an increase in symmetric divisions of putative cancer stem cells. Conclusions Progastrin stimulates proliferation in colons of mice and cultured human cells via CCK2R- and β-arrestin 1 and 2-dependent suppression of Bmp2 signaling. This process promotes symmetric cell division. © 2013 by the AGA Institute