VoxPoser: Composable 3D Value Maps for Robotic Manipulation with Language Models

Large language models (LLMs) are shown to possess a wealth of actionable knowledge that can be extracted for robot manipulation in the form of reasoning and planning. Despite the progress, most still rely on pre-defined motion primitives to carry out the physical interactions with the environment, which remains a major bottleneck. In this work, we aim to synthesize robot trajectories, i.e., a dense sequence of 6-DoF end-effector waypoints, for a large variety of manipulation tasks given an open-set of instructions and an open-set of objects. We achieve this by first observing that LLMs excel at inferring affordances and constraints given a free-form language instruction. More importantly, by leveraging their code-writing capabilities, they can interact with a vision-language model (VLM) to compose 3D value maps to ground the knowledge into the observation space of the agent. The composed value maps are then used in a model-based planning framework to zero-shot synthesize closed-loop robot trajectories with robustness to dynamic perturbations. We further demonstrate how the proposed framework can benefit from online experiences by efficiently learning a dynamics model for scenes that involve contact-rich interactions. We present a large-scale study of the proposed method in both simulated and real-robot environments, showcasing the ability to perform a large variety of everyday manipulation tasks specified in free-form natural language. Videos and code at https://voxposer.github.i

Full Stack Optimization of Transformer Inference: a Survey

Recent advances in state-of-the-art DNN architecture design have been moving toward Transformer models. These models achieve superior accuracy across a wide range of applications. This trend has been consistent over the past several years since Transformer models were originally introduced. However, the amount of compute and bandwidth required for inference of recent Transformer models is growing at a significant rate, and this has made their deployment in latency-sensitive applications challenging. As such, there has been an increased focus on making Transformer models more efficient, with methods that range from changing the architecture design, all the way to developing dedicated domain-specific accelerators. In this work, we survey different approaches for efficient Transformer inference, including: (i) analysis and profiling of the bottlenecks in existing Transformer architectures and their similarities and differences with previous convolutional models; (ii) implications of Transformer architecture on hardware, including the impact of non-linear operations such as Layer Normalization, Softmax, and GELU, as well as linear operations, on hardware design; (iii) approaches for optimizing a fixed Transformer architecture; (iv) challenges in finding the right mapping and scheduling of operations for Transformer models; and (v) approaches for optimizing Transformer models by adapting the architecture using neural architecture search. Finally, we perform a case study by applying the surveyed optimizations on Gemmini, the open-source, full-stack DNN accelerator generator, and we show how each of these approaches can yield improvements, compared to previous benchmark results on Gemmini. Among other things, we find that a full-stack co-design approach with the aforementioned methods can result in up to 88.7x speedup with a minimal performance degradation for Transformer inference

Keystone microalgae species determine the removal efficiency of sulfamethoxazole: a case study of Chlorella pyrenoidosa and microalgae consortia

In recent years, antibiotics pollution has caused serious harm to the aquatic environment, and microalgae mediated degradation of antibiotics has attracted increasing attention. However, the potential toxicity of antibiotics to keystone microalgae species or their microalgae consortia, and the impact of microalgal diversity on antibiotic removal need to be further studied. In this study, we investigated the removal efficiency and tolerance of five freshwater microalgae (Chlorella pyrenoidosa, Scenedesmus quadricauda, Dictyosphaerium sp., Haematoccocus pluvialis, and Botryococcus braunii) and their microalgae consortia to sulfamethoxazole (SMX). We found that the removal efficiency of SMX by C. pyrenoidosa reached 49%, while the other four microalgae ranged between 9% and 16%. In addition, C. pyrenoidosa, S. quadricauda, and Dictyosphaerium sp. had better tolerance to SMX than H. pluvialis, and their growth and photosynthesis were less affected. At 10 and 50 mg/L SMX, the removal capacity of SMX by mixed microalgae consortia was lower than that of C. pyrenoidos except for the consortium with C. pyrenoidos and S. quadricauda. The consortia generally showed higher sensitivity towards SMX than the individual species, and the biochemical characteristics (photosynthetic pigment, chlorophyll fluorescence parameters, superoxide anion (O2-), superoxide dismutase activity (SOD), malondialdehyde (MDA) and extracellular enzymes) were significantly influenced by SMX stress. Therefore, the removal of antibiotics by microalgae consortia did not increase with the number of microalgae species. Our study provides a new perspective for the selection of microalgal consortia to degrade antibiotics

Natural Coevolution of Tumor and Immunoenvironment in Glioblastoma.

Isocitrate dehydrogenase (IDH) wild-type glioblastoma (GBM) has a dismal prognosis. A better understanding of tumor evolution holds the key to developing more effective treatment. Here we study GBM\u27s natural evolutionary trajectory by using rare multifocal samples. We sequenced 61,062 single cells from eight multifocal IDH wild-type primary GBMs and defined a natural evolution signature (NES) of the tumor. We show that the NES significantly associates with the activation of transcription factors that regulate brain development, including MYBL2 and FOSL2. Hypoxia is involved in inducing NES transition potentially via activation of the HIF1A-FOSL2 axis. High-NES tumor cells could recruit and polarize bone marrow-derived macrophages through activation of the FOSL2-ANXA1-FPR1/3 axis. These polarized macrophages can efficiently suppress T-cell activity and accelerate NES transition in tumor cells. Moreover, the polarized macrophages could upregulate CCL2 to induce tumor cell migration. SIGNIFICANCE: GBM progression could be induced by hypoxia via the HIF1A-FOSL2 axis. Tumor-derived ANXA1 is associated with recruitment and polarization of bone marrow-derived macrophages to suppress the immunoenvironment. The polarized macrophages promote tumor cell NES transition and migration. This article is highlighted in the In This Issue feature, p. 2711

Aggregation-Induced Emission (AIE), Life and Health

Light has profoundly impacted modern medicine and healthcare, with numerous luminescent agents and imaging techniques currently being used to assess health and treat diseases. As an emerging concept in luminescence, aggregation-induced emission (AIE) has shown great potential in biological applications due to its advantages in terms of brightness, biocompatibility, photostability, and positive correlation with concentration. This review provides a comprehensive summary of AIE luminogens applied in imaging of biological structure and dynamic physiological processes, disease diagnosis and treatment, and detection and monitoring of specific analytes, followed by representative works. Discussions on critical issues and perspectives on future directions are also included. This review aims to stimulate the interest of researchers from different fields, including chemistry, biology, materials science, medicine, etc., thus promoting the development of AIE in the fields of life and health

Phase-rotation-aided relay selection in two-way decode-and-forward relay networks

This paper proposes a relay selection scheme that aims to improve the end-to-end symbol error rate (SER) performance of a two-way relay network (TWRN). The TWRN consists of two single-antenna sources and multiple relays employing decode-and-forward (DF) protocol. It is shown that the SER performance is determined by the minimum decision distance (DD) observed in the TWRN. However, the minimum DD is likely to be made arbitrarily small by channel fading. To tackle this problem, a phase rotation (PR) aided relay selection (RS) scheme is proposed to enlarge the minium DD, which in turn improves the SER performance. The proposed PR based scheme rotates the phases of the transmitted symbols of one source and of the selected relay according to the channel state information, aiming for increasing all DDs to be above a desired bound. The lower bound is further optimized by using a MaxMin-RS criterion associated with the channel gains. It is demonstrated that the PR aided MaxMin-RS approach achieves full diversity gain and an improved array gain. Furthermore, compared with the existing DF based schemes, the proposed scheme allows more flexible relay antenna configurations

Task modulation of disyllabic spoken word recognition in Mandarin Chinese: a unimodal ERP study

Using unimodal auditory tasks of word-matching and meaning-matching, this study investigated how the phonological and semantic processes in Chinese disyllabic spoken word recognition are modulated by top-down mechanism induced by experimental tasks. Both semantic similarity and word-initial phonological similarity between the primes and targets were manipulated. Results showed that at early stage of recognition (similar to 150-250 ms), an enhanced P2 was elicited by the word-initial phonological mismatch in both tasks. In similar to 300-500 ms, a fronto-central negative component was elicited by word-initial phonological similarities in the word-matching task, while a parietal negativity was elicited by semantically unrelated primes in the meaning-matching task, indicating that both the semantic and phonological processes can be involved in this time window, depending on the task requirements. In the late stage (similar to 500-700 ms), a centro-parietal Late N400 was elicited in both tasks, but with a larger effect in the meaning-matching task than in the word-matching task. This finding suggests that the semantic representation of the spoken words can be activated automatically in the late stage of recognition, even when semantic processing is not required. However, the magnitude of the semantic activation is modulated by task requirements

Effects of Polyester Microfibers on the Growth and Toxicity Production of Bloom-Forming Cyanobacterium <i>Microcystis aeruginosa</i>

The global pollution of microplastics (MPs) has attracted wide attention, and many studies have been conducted on the effects of MP qualities or types and particle sizes on aquatic organisms. However, few studies on the impact of polyethylene terephthalate microplastic (mPET) with different colors on phytoplankton in aquatic ecosystems have been carried out. In this study, mPET of three common colors (green, black, and white) in different concentrations (0, 10, 50, 100, and 200 mg/L) were selected to explore effects on a bloom-forming cyanobacterium Microcystis aeruginosa. The growth, photosynthesis, the number and size of colony, and MC-LR production of M. aeruginosa were studied within a 25-days exposure experiment. The results showed that colors of mPET had significant effects on the growth and photosynthesis of this species but the concentration of mPET had no significant effect. The low concentration of green mPET group promoted algal growth, photosynthesis, and the M. aeruginosa exposed to it was easier to agglomerate into colonies. Moreover, both mPET colors and concentrations have a significant impact on the microcystin production of M. aeruginosa. The low concentration of the green mPET group significantly inhibited the production throughout the experiment, while the white and black mPET significantly increased the concentration of extracellular microcystin (MC-LR). Our results provided new insights into the effects of MPs with different colors and concentrations on the growth and physiology of cyanobacteria and provide basic data for the ecological risk assessment and pollution prevention of MPs

Metabolic signatures and potential biomarkers for the diagnosis and treatment of colon cancer cachexia

Cancer cachexia (CAC) is a debilitating condition that often arises from noncachexia cancer (NCAC), with distinct metabolic characteristics and medical treatments. However, the metabolic changes and underlying molecular mechanisms during cachexia progression remain poorly understood. Understanding the progression of CAC is crucial for developing diagnostic approaches to distinguish between CAC and NCAC stages, facilitating appropriate treatment for cancer patients. In this study, we establish a mouse model of colon CAC and categorize the mice into three groups: CAC, NCAC and normal control (NOR). By performing nuclear magnetic resonance (NMR)-based metabolomic profiling on mouse sera, we elucidate the metabolic properties of these groups. Our findings unveil significant differences in the metabolic profiles among the CAC, NCAC and NOR groups, highlighting significant impairments in energy metabolism and amino acid metabolism during cachexia progression. Additionally, we observe the elevated serum levels of lysine and acetate during the transition from the NCAC to CAC stages. Using multivariate ROC analysis, we identify lysine and acetate as potential biomarkers for distinguishing between CAC and NCAC stages. These biomarkers hold promise for the diagnosis of CAC from noncachexia cancer. Our study provides novel insights into the metabolic mechanisms underlying cachexia progression and offers valuable avenues for the diagnosis and treatment of CAC in clinical settings

NMR-Based Metabolomic Analysis for the Effects of Trimethylamine N-Oxide Treatment on C2C12 Myoblasts under Oxidative Stress

The gut microbial metabolite trimethylamine N-oxide (TMAO) has received increased attention due to its close relationship with cardiovascular disease and type 2 diabetes. In previous studies, TMAO has shown both harmful and beneficial effects on various tissues, but the underlying molecular mechanisms remain to be clarified. Here, we explored the effects of TMAO treatment on H2O2-impaired C2C12 myoblasts, analyzed metabolic changes and identified significantly altered metabolic pathways through nuclear magnetic resonance-based (NMR-based) metabolomic profiling. The results exhibit that TMAO treatment partly alleviated the H2O2-induced oxidative stress damage of cells and protected C2C12 myoblasts by improving cell viability, increasing cellular total superoxide dismutase capacity, improving the protein expression of catalase, and reducing the level of malondialdehyde. We further showed that H2O2 treatment decreased levels of branched-chain amino acids (isoleucine, leucine and valine) and several amino acids including alanine, glycine, threonine, phenylalanine and histidine, and increased the level of phosphocholine related to cell membrane structure, while the TMAO treatment partially reversed the changing trends of these metabolite levels by improving the integrity of the cell membranes. This study indicates that the TMAO treatment may be a promising strategy to alleviate oxidative stress damage in skeletal muscle