Customizing Segmentation Foundation Model via Prompt Learning for Instance Segmentation

Recently, foundation models trained on massive datasets to adapt to a wide range of domains have attracted considerable attention and are actively being explored within the computer vision community. Among these, the Segment Anything Model (SAM) stands out for its remarkable progress in generalizability and flexibility for image segmentation tasks, achieved through prompt-based object mask generation. However, despite its strength, SAM faces two key limitations when applied to customized instance segmentation that segments specific objects or those in unique environments not typically present in the training data: 1) the ambiguity inherent in input prompts and 2) the necessity for extensive additional training to achieve optimal segmentation. To address these challenges, we propose a novel method, customized instance segmentation via prompt learning tailored to SAM. Our method involves a prompt learning module (PLM), which adjusts input prompts into the embedding space to better align with user intentions, thereby enabling more efficient training. Furthermore, we introduce a point matching module (PMM) to enhance the feature representation for finer segmentation by ensuring detailed alignment with ground truth boundaries. Experimental results on various customized instance segmentation scenarios demonstrate the effectiveness of the proposed method.Comment: 11 pages, 10 figure

Topological band-order transition and quantum spin Hall edge engineering in functionalized X-Bi(111) (X = Ga, In, and Tl) bilayer

Functionalized X-Bi bilayers (X = Ga, In, and Tl) with halogens bonded on their both sides have been recently claimed to be the giant topological insulators due to the strong band inversion strengths. Employing the first-principles electronic structure calculation, we find the topological band order transition from the order p-p-s of the X-Bi bilayers with halogens on their both sides to the new order p-s-p of the bilayers (especially for X = Ga and In) with halogen on one side and hydrogen on the other side, where the asymmetric hydrogen bonding simulates the substrate. We further find that the p-s bulk band gap of the bilayer bearing the new order p-s-p sensitively depends on the electric field, which enables a meaningful engineering of the quantum spin Hall edge state by controlling the external electric field. © 2016 The Author(s).1

Automatic segmentation of wrist bone fracture area by K-means pixel clustering from X-ray image

Early detection of subtle fracture is important particularly for the senior citizens’ quality of life. Naked eye examination from X-ray image may cause false negatives due to operator subjectivity thus computer vision based automatic detection software is much needed in practice.  In this paper, we propose an automatic extraction method for suspisious wrist fracture regions. We apply K-means in pixel clustering to form the candidate part of possible fracture from wrist X-ray image automatically. This method can recover previously detected patterned false cases with edge detection method after fuzzy stretching. The proposed method is successful in 16 out of 20 tested cases in experiment