NavGPT: Explicit Reasoning in Vision-and-Language Navigation with Large Language Models

Trained with an unprecedented scale of data, large language models (LLMs) like ChatGPT and GPT-4 exhibit the emergence of significant reasoning abilities from model scaling. Such a trend underscored the potential of training LLMs with unlimited language data, advancing the development of a universal embodied agent. In this work, we introduce the NavGPT, a purely LLM-based instruction-following navigation agent, to reveal the reasoning capability of GPT models in complex embodied scenes by performing zero-shot sequential action prediction for vision-and-language navigation (VLN). At each step, NavGPT takes the textual descriptions of visual observations, navigation history, and future explorable directions as inputs to reason the agent's current status, and makes the decision to approach the target. Through comprehensive experiments, we demonstrate NavGPT can explicitly perform high-level planning for navigation, including decomposing instruction into sub-goal, integrating commonsense knowledge relevant to navigation task resolution, identifying landmarks from observed scenes, tracking navigation progress, and adapting to exceptions with plan adjustment. Furthermore, we show that LLMs is capable of generating high-quality navigational instructions from observations and actions along a path, as well as drawing accurate top-down metric trajectory given the agent's navigation history. Despite the performance of using NavGPT to zero-shot R2R tasks still falling short of trained models, we suggest adapting multi-modality inputs for LLMs to use as visual navigation agents and applying the explicit reasoning of LLMs to benefit learning-based models

A Novel Latin Square Image Cipher

In this paper, we introduce a symmetric-key Latin square image cipher (LSIC) for grayscale and color images. Our contributions to the image encryption community include 1) we develop new Latin square image encryption primitives including Latin Square Whitening, Latin Square S-box and Latin Square P-box ; 2) we provide a new way of integrating probabilistic encryption in image encryption by embedding random noise in the least significant image bit-plane; and 3) we construct LSIC with these Latin square image encryption primitives all on one keyed Latin square in a new loom-like substitution-permutation network. Consequently, the proposed LSIC achieve many desired properties of a secure cipher including a large key space, high key sensitivities, uniformly distributed ciphertext, excellent confusion and diffusion properties, semantically secure, and robustness against channel noise. Theoretical analysis show that the LSIC has good resistance to many attack models including brute-force attacks, ciphertext-only attacks, known-plaintext attacks and chosen-plaintext attacks. Experimental analysis under extensive simulation results using the complete USC-SIPI Miscellaneous image dataset demonstrate that LSIC outperforms or reach state of the art suggested by many peer algorithms. All these analysis and results demonstrate that the LSIC is very suitable for digital image encryption. Finally, we open source the LSIC MATLAB code under webpage https://sites.google.com/site/tuftsyuewu/source-code.Comment: 26 pages, 17 figures, and 7 table

Droplets as Carriers for Flexible Electronic Devices

Coupling soft bodies and dynamic motions with multifunctional flexible electronics is challenging, but is essential in satisfying the urgent and soaring demands of fully soft and comprehensive robotic systems that can perform tasks in spite of rigorous spatial constraints. Here, the mobility and adaptability of liquid droplets with the functionality of flexible electronics, and techniques to use droplets as carriers for flexible devices are combined. The resulting active droplets (ADs) with volumes ranging from 150 to 600 µL can conduct programmable functions, such as sensing, actuation, and energy harvesting defined by the carried flexible devices and move under the excitation of gravitational force or magnetic force. They work in both dry and wet environments, and adapt to the surrounding environment through reversible shape shifting. These ADs can achieve controllable motions at a maximum velocity of 226 cm min−1 on a dry surface and 32 cm min-1 in a liquid environment. The conceptual system may eventually lead to individually addressable ADs that offer sophisticated functions for high-throughput molecule analysis, drug assessment, chemical synthesis, and information collection

Scaling Data Generation in Vision-and-Language Navigation

Recent research in language-guided visual navigation has demonstrated a significant demand for the diversity of traversable environments and the quantity of supervision for training generalizable agents. To tackle the common data scarcity issue in existing vision-and-language navigation datasets, we propose an effective paradigm for generating large-scale data for learning, which applies 1200+ photo-realistic environments from HM3D and Gibson datasets and synthesizes 4.9 million instruction trajectory pairs using fully-accessible resources on the web. Importantly, we investigate the influence of each component in this paradigm on the agent's performance and study how to adequately apply the augmented data to pre-train and fine-tune an agent. Thanks to our large-scale dataset, the performance of an existing agent can be pushed up (+11% absolute with regard to previous SoTA) to a significantly new best of 80% single-run success rate on the R2R test split by simple imitation learning. The long-lasting generalization gap between navigating in seen and unseen environments is also reduced to less than 1% (versus 8% in the previous best method). Moreover, our paradigm also facilitates different models to achieve new state-of-the-art navigation results on CVDN, REVERIE, and R2R in continuous environments.Comment: ICCV 202

Cathepsin B Regulates Collagen Expression by Fibroblasts via Prolonging TLR2/NF- κ

Fibroblasts are essential for tissue repair due to producing collagens, and lysosomal proteinase cathepsin B (CatB) is involved in promoting chronic inflammation. We herein report that CatB regulates the expression of collagens III and IV by fibroblasts in response to a TLR2 agonist, lipopolysaccharide from Porphyromonas gingivalis (P.g. LPS). In cultured human BJ fibroblasts, mRNA expression of CatB was significantly increased, while that of collagens III and IV was significantly decreased at 24 h after challenge with P.g. LPS (1 μg/mL). The P.g. LPS-decreased collagen expression was completely inhibited by CA-074Me, the specific inhibitor of CatB. Surprisingly, expression of collagens III and IV was significantly increased in the primary fibroblasts from CatB-deficient mice after challenge with P.g. LPS. The increase of CatB was accompanied with an increase of 8-hydroxy-2′-deoxyguanosine (8-OHdG) and a decrease of IκBα. Furthermore, the P.g. LPS-increased 8-OHdG and decreased IκBα were restored by CA-074Me. Moreover, 87% of CatB and 86% of 8-OHdG were colocalized with gingival fibroblasts of chronic periodontitis patients. The findings indicate the critical role of CatB in regulating the expression of collagens III and IV by fibroblasts via prolonging TLR2/NF-κB activation and oxidative stress. CatB-specific inhibitors may therefore improve chronic inflammation-delayed tissue repair