Collaborative Video Analytics on Distributed Edges with Multiagent Deep Reinforcement Learning

Deep Neural Network (DNN) based video analytics empowers many computer vision-based applications to achieve high recognition accuracy. To reduce inference delay and bandwidth cost for video analytics, the DNN models can be deployed on the edge nodes, which are proximal to end users. However, the processing capacity of an edge node is limited, potentially incurring substantial delay if the inference requests on an edge node is overloaded. While efforts have been made to enhance video analytics by optimizing the configurations on a single edge node, we observe that multiple edge nodes can work collaboratively by utilizing the idle resources on each other to improve the overall processing capacity and resource utilization. To this end, we propose a Multiagent Reinforcement Learning (MARL) based approach, named as EdgeVision, for collaborative video analytics on distributed edges. The edge nodes can jointly learn the optimal policies for video preprocessing, model selection, and request dispatching by collaborating with each other to minimize the overall cost. We design an actor-critic-based MARL algorithm with an attention mechanism to learn the optimal policies. We build a multi-edge-node testbed and conduct experiments with real-world datasets to evaluate the performance of our method. The experimental results show our method can improve the overall rewards by 33.6%-86.4% compared with the most competitive baseline methods

Third-Party Aligner for Neural Word Alignments

Word alignment is to find translationally equivalent words between source and target sentences. Previous work has demonstrated that self-training can achieve competitive word alignment results. In this paper, we propose to use word alignments generated by a third-party word aligner to supervise the neural word alignment training. Specifically, source word and target word of each word pair aligned by the third-party aligner are trained to be close neighbors to each other in the contextualized embedding space when fine-tuning a pre-trained cross-lingual language model. Experiments on the benchmarks of various language pairs show that our approach can surprisingly do self-correction over the third-party supervision by finding more accurate word alignments and deleting wrong word alignments, leading to better performance than various third-party word aligners, including the currently best one. When we integrate all supervisions from various third-party aligners, we achieve state-of-the-art word alignment performances, with averagely more than two points lower alignment error rates than the best third-party aligner. We released our code at https://github.com/sdongchuanqi/Third-Party-Supervised-Aligner.Comment: 12 pages, 4 figures, findings of emnlp 202

Disambiguated Lexically Constrained Neural Machine Translation

Lexically constrained neural machine translation (LCNMT), which controls the translation generation with pre-specified constraints, is important in many practical applications. Current approaches to LCNMT typically assume that the pre-specified lexical constraints are contextually appropriate. This assumption limits their application to real-world scenarios where a source lexicon may have multiple target constraints, and disambiguation is needed to select the most suitable one. In this paper, we propose disambiguated LCNMT (D-LCNMT) to solve the problem. D-LCNMT is a robust and effective two-stage framework that disambiguates the constraints based on contexts at first, then integrates the disambiguated constraints into LCNMT. Experimental results show that our approach outperforms strong baselines including existing data augmentation based approaches on benchmark datasets, and comprehensive experiments in scenarios where a source lexicon corresponds to multiple target constraints demonstrate the constraint disambiguation superiority of our approach.Comment: Accepted at ACL 2023 as a long paper (Findings), 12 pages, 3 figure

AlphaCrystal: Contact map based crystal structure prediction using deep learning

Crystal structure prediction is one of the major unsolved problems in materials science. Traditionally, this problem is formulated as a global optimization problem for which global search algorithms are combined with first principle free energy calculations to predict the ground-state crystal structure given only a material composition or a chemical system. These ab initio algorithms usually cannot exploit a large amount of implicit physicochemical rules or geometric constraints (deep knowledge) of atom configurations embodied in a large number of known crystal structures. Inspired by the deep learning enabled breakthrough in protein structure prediction, herein we propose AlphaCrystal, a crystal structure prediction algorithm that combines a deep residual neural network model that learns deep knowledge to guide predicting the atomic contact map of a target crystal material followed by reconstructing its 3D crystal structure using genetic algorithms. Based on the experiments of a selected set of benchmark crystal materials, we show that our AlphaCrystal algorithm can predict structures close to the ground truth structures. It can also speed up the crystal structure prediction process by predicting and exploiting the predicted contact map so that it has the potential to handle relatively large systems. We believe that our deep learning based ab initio crystal structure prediction method that learns from existing material structures can be used to scale up current crystal structure prediction practice. To our knowledge, AlphaCrystal is the first neural network based algorithm for crystal structure contact map prediction and the first method for directly reconstructing crystal structures from materials composition, which can be further optimized by DFT calculations.Comment: 13 pages; 5 figure

Computational Screening of New Perovskite Materials Using Transfer Learning and Deep Learning

As one of the most studied materials, perovskites exhibit a wealth of superior properties that lead to diverse applications. Computational prediction of novel stable perovskite structures has big potential in the discovery of new materials for solar panels, superconductors, thermal electric, and catalytic materials, etc. By addressing one of the key obstacles of machine learning based materials discovery, the lack of sufficient training data, this paper proposes a transfer learning based approach that exploits the high accuracy of the machine learning model trained with physics-informed structural and elemental descriptors. This gradient boosting regressor model (the transfer learning model) allows us to predict the formation energy with sufficient precision of a large number of materials of which only the structural information is available. The enlarged training set is then used to train a convolutional neural network model (the screening model) with the generic Magpie elemental features with high prediction power. Extensive experiments demonstrate the superior performance of our transfer learning model and screening model compared to the baseline models. We then applied the screening model to filter out promising new perovskite materials out of 21,316 hypothetical perovskite structures with a large portion of them confirmed by existing literature

Electron dynamics in topological insulator based semiconductor-metal interfaces (topological p-n interface based on Bi2Se3 class)

Single-Dirac-cone topological insulators (TI) are the first experimentally discovered class of three dimensional topologically ordered electronic systems, and feature robust, massless spin-helical conducting surface states that appear at any interface between a topological insulator and normal matter that lacks the topological insulator ordering. This topologically defined surface environment has been theoretically identified as a promising platform for observing a wide range of new physical phenomena, and possesses ideal properties for advanced electronics such as spin-polarized conductivity and suppressed scattering. A key missing step in enabling these applications is to understand how topologically ordered electrons respond to the interfaces and surface structures that constitute a device. Here we explore this question by using the surface deposition of cathode (Cu/In/Fe) and anode materials (NO$_2$) and control of bulk doping in Bi$_2$Se$_3$ from P-type to N-type charge transport regimes to generate a range of topological insulator interface scenarios that are fundamental to device development. The interplay of conventional semiconductor junction physics and three dimensional topological electronic order is observed to generate novel junction behaviors that go beyond the doped-insulator paradigm of conventional semiconductor devices and greatly alter the known spin-orbit interface phenomenon of Rashba splitting. Our measurements for the first time reveal new classes of diode-like configurations that can create a gap in the interface electron density near a topological Dirac point and systematically modify the topological surface state Dirac velocity, allowing far reaching control of spin-textured helical Dirac electrons inside the interface and creating advantages for TI superconductors as a Majorana fermion platform over spin-orbit semiconductors.Comment: 14 pages, 4 Figure

Efficient Token-Guided Image-Text Retrieval with Consistent Multimodal Contrastive Training

Image-text retrieval is a central problem for understanding the semantic relationship between vision and language, and serves as the basis for various visual and language tasks. Most previous works either simply learn coarse-grained representations of the overall image and text, or elaborately establish the correspondence between image regions or pixels and text words. However, the close relations between coarse- and fine-grained representations for each modality are important for image-text retrieval but almost neglected. As a result, such previous works inevitably suffer from low retrieval accuracy or heavy computational cost. In this work, we address image-text retrieval from a novel perspective by combining coarse- and fine-grained representation learning into a unified framework. This framework is consistent with human cognition, as humans simultaneously pay attention to the entire sample and regional elements to understand the semantic content. To this end, a Token-Guided Dual Transformer (TGDT) architecture which consists of two homogeneous branches for image and text modalities, respectively, is proposed for image-text retrieval. The TGDT incorporates both coarse- and fine-grained retrievals into a unified framework and beneficially leverages the advantages of both retrieval approaches. A novel training objective called Consistent Multimodal Contrastive (CMC) loss is proposed accordingly to ensure the intra- and inter-modal semantic consistencies between images and texts in the common embedding space. Equipped with a two-stage inference method based on the mixed global and local cross-modal similarity, the proposed method achieves state-of-the-art retrieval performances with extremely low inference time when compared with representative recent approaches.Comment: Code is publicly available: https://github.com/LCFractal/TGD

Naringenin prevents TGF-β1 secretion from breast cancer and suppresses pulmonary metastasis by inhibiting PKC activation

Presenting the incidence of pulmonary metastasis (mice with metastasis/total mice). Tumor-bearing mice treated with naringenin or 1D11 were imaged on day 24 using bags to avoid the bioluminescence from primary tumor. The mice with pulmonary metastases were numbered based on the bioluminescence signal. (TIF 26 kb

Petrographic characterization to build an accurate rock model using micro-CT: Case study on low-permeable to tight turbidite sandstone from Eocene Shahejie Formation

Pore scale flow simulations heavily depend on petrographic characterizing and modeling of reservoir rocks. Mineral phase segmentation and pore network modeling are crucial stages in micro-CT based rock modeling. The success of the pore network model (PNM) to predict petrophysical properties relies on image segmentation, image resolution and most importantly nature of rock (homogenous, complex or microporous). The pore network modeling has experienced extensive research and development during last decade, however the application of these models to a variety of naturally heterogenous reservoir rock is still a challenge. In this paper, four samples from a low permeable to tight sandstone reservoir were used to characterize their petrographic and petrophysical properties using high-resolution micro-CT imaging. The phase segmentation analysis from micro-CT images shows that 5-6% microporous regions are present in kaolinite rich sandstone (E3 and E4), while 1.7-1.8% are present in illite rich sandstone (El and E2). The pore system percolates without micropores in El and E2 while it does not percolate without micropores in E3 and E4. In El and E2, total MICP porosity is equal to the volume percent of macrospores determined from micro-CT images, which indicate that the macropores are well connected and microspores do not play any role in non-wetting fluid (mercury) displacement process. Whereas in E3 and E4 sandstones, the volume percent of micropores is far less (almost 50%) than the total MICP porosity which means that almost half of the pore space was not detected by the micro-CT scan. PNM behaved well in El and E2 where better agreement exists in PNM and MICP measurements. While E3 and E4 exhibit multiscale pore space which cannot be addressed with single scale PNM method, a multiscale approach is needed to characterize such complex rocks. This study provides helpful insights towards the application of existing micro-CT based petrographic characterization methodology to naturally complex petroleum reservoir rocks