Object-Centric Open-Vocabulary Image-Retrieval with Aggregated Features

The task of open-vocabulary object-centric image retrieval involves the retrieval of images containing a specified object of interest, delineated by an open-set text query. As working on large image datasets becomes standard, solving this task efficiently has gained significant practical importance. Applications include targeted performance analysis of retrieved images using ad-hoc queries and hard example mining during training. Recent advancements in contrastive-based open vocabulary systems have yielded remarkable breakthroughs, facilitating large-scale open vocabulary image retrieval. However, these approaches use a single global embedding per image, thereby constraining the system's ability to retrieve images containing relatively small object instances. Alternatively, incorporating local embeddings from detection pipelines faces scalability challenges, making it unsuitable for retrieval from large databases. In this work, we present a simple yet effective approach to object-centric open-vocabulary image retrieval. Our approach aggregates dense embeddings extracted from CLIP into a compact representation, essentially combining the scalability of image retrieval pipelines with the object identification capabilities of dense detection methods. We show the effectiveness of our scheme to the task by achieving significantly better results than global feature approaches on three datasets, increasing accuracy by up to 15 mAP points. We further integrate our scheme into a large scale retrieval framework and demonstrate our method's advantages in terms of scalability and interpretability.Comment: BMVC 202

Deep Multi-Spectral Registration Using Invariant Descriptor Learning

In this paper, we introduce a novel deep-learning method to align cross-spectral images. Our approach relies on a learned descriptor which is invariant to different spectra. Multi-modal images of the same scene capture different signals and therefore their registration is challenging and it is not solved by classic approaches. To that end, we developed a feature-based approach that solves the visible (VIS) to Near-Infra-Red (NIR) registration problem. Our algorithm detects corners by Harris and matches them by a patch-metric learned on top of CIFAR-10 network descriptor. As our experiments demonstrate we achieve a high-quality alignment of cross-spectral images with a sub-pixel accuracy. Comparing to other existing methods, our approach is more accurate in the task of VIS to NIR registration

The Neurospora crassa PP2A Regulatory Subunits RGB1 and B56 Are Required for Proper Growth and Development and Interact with the NDR Kinase COT1

COT1 is the founding member of the highly conserved nuclear Dbf2-related (NDR) Ser/Thr kinase family and plays a role in the regulation of polar growth and development in Neurospora crassa and other fungi. Changes in COT1 phosphorylation state have been shown to affect hyphal elongation, branching, and conidiation. The function of NDR protein kinases has been shown to be regulated by type 2A protein phosphatases (PP2As). PP2As are heterotrimers comprised of a catalytic and scaffolding protein along with an interchangeable regulatory subunit involved in determining substrate specificity. Inactivation of the N. crassa PP2A regulatory subunits rgb-1 and b56 conferred severe hyphal growth defects. Partial suppression of defects observed in the rgb-1RIP strain (but not in the Δb56 mutant) was observed in cot-1 phosphomimetic mutants, demonstrating that altering COT1 phosphorylation state can bypass, at least in part, the requirement of a functional RGB1 subunit. The functional fusion proteins RGB1::GFP and B56::GFP predominantly localized to hyphal tips and septa, respectively, indicating that their primary activity is in different cellular locations. COT1 protein forms exhibited a hyperphosphorylated gel migration pattern in an rgb-1RIP mutant background, similar to that observed when the fungus was cultured in the presence of the PP2A inhibitor cantharidin. COT1 was hypophosphorylated in a Δb56 mutant background, suggesting that this regulatory subunit may be involved in determining COT1 phosphorylation state, yet in an indirect manner. Reciprocal co-immunoprecipitation analyses, using tagged COT1, PPH1, RGB1, and B56 subunits established that these proteins physically interact. Taken together, our data determine the presence of a functional and physical link between PP2A and COT1 and show that two of the PP2A regulatory subunits interact with the kinase and determine COT1 phosphorylation state

Inhibition of muscle fibrosis and improvement of muscle histopathology in dysferlin knock-out mice treated with halofuginone

Absence of, or loss-of-function mutations in the dysferlin gene (dysf) result in dysferlinopathy, characterized by increased muscle inflammation, collagen deposition and deterioration in muscle function. We evaluated halofuginone efficacy in improving muscle histopathology in mice with deleted dysf transmembrane domain. Quadriceps sublumbar and longissimus muscles of 9-month-old dysf-/- mice treated with halofuginone for 4 months exhibited a reduction in centrally-nucleated myofibers, inflammatory infiltrates and collagen content. Late onset of dysferlinopathy makes it ideal for evaluating the efficacy of early treatments on late outcome. The dysf-/- mice were treated with halofuginone for 3 to 4 months starting at 1, 5 or 9 months of age, and quadricep muscle histopathology was evaluated at 12 months. Collagen content and number of centrally nucleated myofibers decreased after early halofuginone treatment, administered when myofibers with central nuclei and inflammatory infiltrates are evident, but there was almost no fibrosis. When administered at the beginning of fibrosis it resulted in a further decrease in the number of centrally-nucleated myofibers with no additional decrease in collagen levels. Cardiac fibrosis was almost completely abolished following early halofuginone treatment. Halofuginone inhibited Smad3 phosphorylation and its translocation to the nucleus and increased the activity of matrix metalloproteinases 9 and 2 responsible for resolution of pre-existing collagen. Macrophage and myofibroblast invasion into the dystrophic muscle at the site of myofibers with central nuclei was inhibited by halofuginone. These results suggest that early halofuginone treatment can prevent the late outcome of dysferlinopathy and can cause resolution of the established fibrosis when administered at later stages

Image interpretation by iterative bottom-up top-down processing

Scene understanding requires the extraction and representation of scene components, such as objects and their parts, people, and places, together with their individual properties, as well as relations and interactions between them. We describe a model in which meaningful scene structures are extracted from the image by an iterative process, combining bottom-up (BU) and top-down (TD) networks, interacting through a symmetric bi-directional communication between them (‘counter-streams’ structure). The BU- TD model extracts and recognizes scene constituents with their selected properties and relations, and uses them to describe and understand the image. The scene representation is constructed by the iterative use of three components. The first model component is a bottom-up stream that extracts selected scene elements, properties and relations. The second component (‘cognitive augmentation’) augments the extracted visual representation based on relevant non-visual stored representations. It also provides input to the third component, the top-down stream, in the form of a TD instruction, instructing the model what task to perform next. The top-down stream then guides the BU visual stream to perform the selected task in the next cycle. During this process, the visual representations extracted from the image can be combined with relevant non- visual representations, so that the final scene representation is based on both visual information extracted from the scene and relevant stored knowledge of the world. We show how the BU-TD model composes complex visual tasks from sequences of steps, invoked by individual TD instructions. In particular, we describe how a sequence of TD-instructions is used to extract from the scene structures of interest, including an algorithm to automatically select the next TD- instruction in the sequence. The selection of TD instruction depends in general on the goal, the image, and on information already extracted from the image in previous steps. The TD-instructions sequence is therefore not a fixed sequence determined at the start, but an evolving program (or ‘visual routine’) that depends on the goal and the image. The extraction process is shown to have favourable properties in terms of combinatorial generalization, generalizing well to novel scene structures and new combinations of objects, properties and relations not seen during training. Finally, we compare the model with relevant aspects of the human vision, and suggest directions for using the BU-TD scheme for integrating visual and cognitive components in the process of scene understanding.This work was supported by the Center for Brains, Minds and Machines (CBMM), funded by NSF STC award CCF-1231216

Epigenetic Optical Mapping of 5‑Hydroxymethylcytosine in Nanochannel Arrays

The epigenetic mark 5-hydroxymethylcytosine (5-hmC) is a distinct product of active DNA demethylation that is linked to gene regulation, development, and disease. In particular, 5-hmC levels dramatically decline in many cancers, potentially serving as an epigenetic biomarker. The noise associated with next-generation 5-hmC sequencing hinders reliable analysis of low 5-hmC containing tissues such as blood and malignant tumors. Additionally, genome-wide 5-hmC profiles generated by short-read sequencing are limited in providing long-range epigenetic information relevant to highly variable genomic regions, such as the 3.7 Mbp disease-related Human Leukocyte Antigen (HLA) region. We present a long-read, highly sensitive single-molecule mapping technology that generates hybrid genetic/epigenetic profiles of native chromosomal DNA. The genome-wide distribution of 5-hmC in human peripheral blood cells correlates well with 5-hmC DNA immunoprecipitation (hMeDIP) sequencing. However, the long single-molecule read-length of 100 kbp to 1 Mbp produces 5-hmC profiles across variable genomic regions that failed to show up in the sequencing data. In addition, optical 5-hmC mapping shows a strong correlation between the 5-hmC density in gene bodies and the corresponding level of gene expression. The single-molecule concept provides information on the distribution and coexistence of 5-hmC signals at multiple genomic loci on the same genomic DNA molecule, revealing long-range correlations and cell-to-cell epigenetic variation

Current status of liver surgery for non-colorectal non-neuroendocrine liver metastases: the NON.LI.MET. Italian Society for Endoscopic Surgery and New Technologies (SICE) and Association of Italian Surgeons in Europe (ACIE) collaborative international survey

Despite the increasing trend in liver resections for non-colorectal non-neuroendocrine liver metastases (NCNNLM), the role of surgery for these liver malignancies is still debated. Registries are an essential, reliable tool for assessing epidemiology, diagnosis, and therapeutic approach in a single hub, especially when data are dispersive and inconclusive, as in our case. The dissemination of this preliminary survey would allow us to understand if the creation of an International Registry is a viable option, while still offering a snapshot on this issue, investigating clinical practices worldwide. The steering committee designed an online questionnaire with Google Forms, which consisted of 37 questions, and was open from October 5th, 2022, to November 30th, 2022. It was disseminated using social media and mailing lists of the Italian Society of Endoscopic Surgery and New Technologies (SICE), the Association of Italian Surgeons in Europe (ACIE), and the Spanish Chapter of the American College of Surgeons (ACS). Overall, 141 surgeons (approximately 18% of the total invitations sent) from 27 countries on four continents participated in the survey. Most respondents worked in general surgery units (62%), performing less than 50 liver resections/year (57%). A multidisciplinary discussion was currently performed to validate surgical indications for NCNNLM in 96% of respondents. The most commonly adopted selection criteria were liver resectability, RECIST criteria, and absence of extrahepatic disease. Primary tumors were generally of gastrointestinal (42%), breast (31%), and pancreaticobiliary origin (13%). The most common interventions were parenchymal-sparing resections (51% of respondents) of metachronous metastases with an open approach. Major post-operative complications (Clavien-Dindo > 2) occurred in up to 20% of the procedures, according to 44% of respondents. A subset analysis of data from high-volume centers (> 100 cases/year) showed lower post-operative complications and better survival. The present survey shows that NCNNLM patients are frequently treated by surgeons in low-volume hospitals for liver surgery. Selection criteria are usually based on common sense. Liver resections are performed mainly with an open approach, possibly carrying a high burden of major post-operative complications. International guidelines and a specific consensus on this field are desirable, as well as strategies for collaboration between high-volume and low-volume centers. The present study can guide the elaboration of a multi-institutional document on the optimal pathway in the management of patients with NCNNLM