3D Model-based Zero-Shot Pose Estimation Pipeline

Most existing learning-based pose estimation methods are typically developed for non-zero-shot scenarios, where they can only estimate the poses of objects present in the training dataset. This setting restricts their applicability to unseen objects in the training phase. In this paper, we introduce a fully zero-shot pose estimation pipeline that leverages the 3D models of objects as clues. Specifically, we design a two-step pipeline consisting of 3D model-based zero-shot instance segmentation and a zero-shot pose estimator. For the first step, there is a novel way to perform zero-shot instance segmentation based on the 3D models instead of text descriptions, which can handle complex properties of unseen objects. For the second step, we utilize a hierarchical geometric structure matching mechanism to perform zero-shot pose estimation which is 10 times faster than the current render-based method. Extensive experimental results on the seven core datasets on the BOP challenge show that the proposed method outperforms the zero-shot state-of-the-art method with higher speed and lower computation cost

MIAD: A Maintenance Inspection Dataset for Unsupervised Anomaly Detection

Visual anomaly detection plays a crucial role in not only manufacturing inspection to find defects of products during manufacturing processes, but also maintenance inspection to keep equipment in optimum working condition particularly outdoors. Due to the scarcity of the defective samples, unsupervised anomaly detection has attracted great attention in recent years. However, existing datasets for unsupervised anomaly detection are biased towards manufacturing inspection, not considering maintenance inspection which is usually conducted under outdoor uncontrolled environment such as varying camera viewpoints, messy background and degradation of object surface after long-term working. We focus on outdoor maintenance inspection and contribute a comprehensive Maintenance Inspection Anomaly Detection (MIAD) dataset which contains more than 100K high-resolution color images in various outdoor industrial scenarios. This dataset is generated by a 3D graphics software and covers both surface and logical anomalies with pixel-precise ground truth. Extensive evaluations of representative algorithms for unsupervised anomaly detection are conducted, and we expect MIAD and corresponding experimental results can inspire research community in outdoor unsupervised anomaly detection tasks. Worthwhile and related future work can be spawned from our new dataset

Balancing Logit Variation for Long-tailed Semantic Segmentation

Semantic segmentation usually suffers from a long-tail data distribution. Due to the imbalanced number of samples across categories, the features of those tail classes may get squeezed into a narrow area in the feature space. Towards a balanced feature distribution, we introduce category-wise variation into the network predictions in the training phase such that an instance is no longer projected to a feature point, but a small region instead. Such a perturbation is highly dependent on the category scale, which appears as assigning smaller variation to head classes and larger variation to tail classes. In this way, we manage to close the gap between the feature areas of different categories, resulting in a more balanced representation. It is noteworthy that the introduced variation is discarded at the inference stage to facilitate a confident prediction. Although with an embarrassingly simple implementation, our method manifests itself in strong generalizability to various datasets and task settings. Extensive experiments suggest that our plug-in design lends itself well to a range of state-of-the-art approaches and boosts the performance on top of them

Geo6D: Geometric Constraints Learning for 6D Pose Estimation

Numerous 6D pose estimation methods have been proposed that employ end-to-end regression to directly estimate the target pose parameters. Since the visible features of objects are implicitly influenced by their poses, the network allows inferring the pose by analyzing the differences in features in the visible region. However, due to the unpredictable and unrestricted range of pose variations, the implicitly learned visible feature-pose constraints are insufficiently covered by the training samples, making the network vulnerable to unseen object poses. To tackle these challenges, we proposed a novel geometric constraints learning approach called Geo6D for direct regression 6D pose estimation methods. It introduces a pose transformation formula expressed in relative offset representation, which is leveraged as geometric constraints to reconstruct the input and output targets of the network. These reconstructed data enable the network to estimate the pose based on explicit geometric constraints and relative offset representation mitigates the issue of the pose distribution gap. Extensive experimental results show that when equipped with Geo6D, the direct 6D methods achieve state-of-the-art performance on multiple datasets and demonstrate significant effectiveness, even with only 10% amount of data

TPTU-v2: Boosting Task Planning and Tool Usage of Large Language Model-based Agents in Real-world Systems

Large Language Models (LLMs) have demonstrated proficiency in addressing tasks that necessitate a combination of task planning and the usage of external tools that require a blend of task planning and the utilization of external tools, such as APIs. However, real-world complex systems present three prevalent challenges concerning task planning and tool usage: (1) The real system usually has a vast array of APIs, so it is impossible to feed the descriptions of all APIs to the prompt of LLMs as the token length is limited; (2) the real system is designed for handling complex tasks, and the base LLMs can hardly plan a correct sub-task order and API-calling order for such tasks; (3) Similar semantics and functionalities among APIs in real systems create challenges for both LLMs and even humans in distinguishing between them. In response, this paper introduces a comprehensive framework aimed at enhancing the Task Planning and Tool Usage (TPTU) abilities of LLM-based agents operating within real-world systems. Our framework comprises three key components designed to address these challenges: (1) the API Retriever selects the most pertinent APIs for the user task among the extensive array available; (2) LLM Finetuner tunes a base LLM so that the finetuned LLM can be more capable for task planning and API calling; (3) the Demo Selector adaptively retrieves different demonstrations related to hard-to-distinguish APIs, which is further used for in-context learning to boost the final performance. We validate our methods using a real-world commercial system as well as an open-sourced academic dataset, and the outcomes clearly showcase the efficacy of each individual component as well as the integrated framework

Seawater carbonate chemistry and competition for growth, photosynthetic performance and biochemical composition in Neopyropia yezoensis and Ulva prolifera

The occurrence of various marine macroalgae in the same niche will inevitably lead to interspecific competition due to similar environmental requirements. With the increasing global atmospheric CO2 concentration, the resulting ocean acidification can potentially influence competition among macroalgae in the future. Neopyropia yezoensis (Rhodophyta, formerly Pyropia yezoensis) and the epiphytic alga Ulva prolifera (Chlorophyta) were selected for investigating competition among macroalgae grown under different CO2 conditions. The results showed that when cultured with U. prolifera, N. yezoensis' growth rate was significantly inhibited along with a sharp decrease in net photosynthetic rate. Although CO2 decreased the growth rate of N. yezoensis, it enhanced the resistance of the alga to the allelopathic effect of U. prolifera. While no difference was found between U. prolifera grown in monoculture and biculture, strong competitive ability was observed. CO2 could enhance this ability with higher net photosynthetic rate. However, CO2 significantly inhibited the carotenoid synthesis in both plants. This inhibition in N. yezoensis was more pronounced in the presence of U. prolifera. Biculture promoted the accumulation of soluble protein in N. yezoensis while it inhibited the process in U. prolifera. In addition, it enhanced the inhibitory effect of acidification on soluble carbohydrates of both plants. Elevated CO2 levels alleviated the competition between N. yezoensis and U. prolifera, but the latter can become the more competitive epiphytic alga which can impact the future of nori culture

Seawater carbonate chemistry and growth, net photosynthesis rate, Chlorophyll fluorescence parameter, soluble protein, photosynthetic pigments of red algae Pyropia yezoensis

Increasing CO2 levels in the surface water of oceans are expected to decrease oceanic pH and lead to seawater acidification. The responses of macroalgaea to this acidification of coastal waters have been studied in detail; however, most reports have focused on the adult stage only, while ignoring other life cycle stages. In this study, the economically important seaweed species Pyropia yezoensis was cultured under two CO2 concentrations (ambient CO2: 400 μatm; elevated CO2: 1000 μatm) and two light intensities (low light intensity: 80 μmol photons/m**2 /s; and high light intensity: 240 μmol photons/m**2 /s). The effects on the growth and photosynthetic performance of P. yezoensis were explored at different life cycle stages. Relative growth rates were significantly elevated at the conchocelis stage under high light intensity and elevated CO2 concentration. Moreover, the Pmax of P. yezoensis was also increased under high light intensity. However, this positive effect inversed at the thallus stage. The relative growth rate, relative electron transport rate (rETR), and net photosynthetic rate decreased at the thallus stage in response to high CO2 concentration. Under low light intensity, elevated CO2 concentration significantly increased the relative growth rates of conchocelis and thallus stages. These were 269% and 45% higher at elevated CO2 concentration compared with ambient CO2 concentrations, respectively. The Chl a and phycoerythrin levels were also higher under elevated CO2 level at the conchocelis stage. However, the rETR for the thallus stage was elevated under low light. This suggests that seawater acidification could positively affect algae at low light conditions (especially at the conchocelis stage). Different growth stages of P. yezoensis may respond differently to seawater acidification and changes of light intensity. Thalli growth stage, stocking density, and seawater depth should be considered in different areas to optimize the primary production of macroalgae

Isolation and characterization of a novel temperate bacteriophage infecting Aeromonas hydrophila isolated from a Macrobrachium rosenbergii larvae pond

Aeromonas hydrophila is an opportunistic pathogen that frequently leads to significant mortality in various commercially cultured aquatic species. Bacteriophages offer an alternative strategy for pathogens elimination. In this study, we isolated, identified, and characterized a novel temperate A. hydrophila phage, designated as P05B. The bacteriophage P05B is a myovirus based on its morphological features, and possesses the capability to lyse A. hydrophila strains isolated from shrimp. The optimal multiplicity of infection (MOI), adsorption rate, latent period, and burst size for phage P05B were determined to be 0.001, 91.7 %, 20 min, and 483 PFU/cell, respectively. Phage P05B displayed stability across a range of temperatures (28–50 °C) and pH values (4.0–10.0). Sequence analysis unveiled that the genome of phage P05B comprises 32,302 base pairs with an average G + C content of 59.4 %. A total of 40 open reading frames (ORF) were encoded within the phage P05B genome. The comparative genomic analyses clearly implied that P05B might represent a novel species of the genus Bielevirus under Peduoviridae family. A phylogenetic tree was reconstructed, demonstrating that P05B shares a close evolutionary relationship with other Aeromonas and Aeromonas phages. In conclusion, this study increased our knowledge about a new temperate phage of A. hydrophila with strong lytic ability