Zero-shot visual reasoning through probabilistic analogical mapping

Human reasoning is grounded in an ability to identify highly abstract commonalities governing superficially dissimilar visual inputs. Recent efforts to develop algorithms with this capacity have largely focused on approaches that require extensive direct training on visual reasoning tasks, and yield limited generalization to problems with novel content. In contrast, a long tradition of research in cognitive science has focused on elucidating the computational principles underlying human analogical reasoning; however, this work has generally relied on manually constructed representations. Here we present visiPAM (visual Probabilistic Analogical Mapping), a model of visual reasoning that synthesizes these two approaches. VisiPAM employs learned representations derived directly from naturalistic visual inputs, coupled with a similarity-based mapping operation derived from cognitive theories of human reasoning. We show that without any direct training, visiPAM outperforms a state-of-the-art deep learning model on an analogical mapping task. In addition, visiPAM closely matches the pattern of human performance on a novel task involving mapping of 3D objects across disparate categories

RIS-based IMT-2030 Testbed for MmWave Multi-stream Ultra-massive MIMO Communications

As one enabling technique of the future sixth generation (6G) network, ultra-massive multiple-input-multiple-output (MIMO) can support high-speed data transmissions and cell coverage extension. However, it is hard to realize the ultra-massive MIMO via traditional phased arrays due to unacceptable power consumption. To address this issue, reconfigurable intelligent surface-based (RIS-based) antennas are an energy-efficient enabler of the ultra-massive MIMO, since they are free of energy-hungry phase shifters. In this article, we report the performances of the RIS-enabled ultra-massive MIMO via a project called Verification of MmWave Multi-stream Transmissions Enabled by RIS-based Ultra-massive MIMO for 6G (V4M), which was proposed to promote the evolution towards IMT-2030. In the V4M project, we manufacture RIS-based antennas with 1024 one-bit elements working at 26 GHz, based on which an mmWave dual-stream ultra-massive MIMO prototype is implemented for the first time. To approach practical settings, the Tx and Rx of the prototype are implemented by one commercial new radio base station and one off-the-shelf user equipment, respectively. The measured data rate of the dual-stream prototype approaches the theoretical peak rate. Our contributions to the V4M project are also discussed by presenting technological challenges and corresponding solutions.Comment: 8 pages, 5 figures, to be published in IEEE Wireless Communication

Engineering zinc oxide hybrid selenium nanoparticles for synergetic anti-tuberculosis treatment by combining Mycobacterium tuberculosis killings and host cell immunological inhibition

IntroductionAs a deadly disease induced by Mycobacterium tuberculosis (Mtb), tuberculosis remains one of the top killers among infectious diseases. The low intracellular Mtb killing efficiency of current antibiotics introduced the long duration anti-TB therapy in clinic with strong side effects and increased drug-resistant mutants. Therefore, the exploration of novel anti-TB agents with potent anti-TB efficiency becomes one of the most urgent issues for TB therapies. MethodsHere, we firstly introduced a novel method for the preparation of zinc oxide-selenium nanoparticles (ZnO-Se NPs) by the hybridization of zinc oxide and selenium to combine the anti-TB activities of zinc oxide nanoparticles and selenium nanoparticles. We characterized the ZnO-Se NPs by dynamic laser light scattering and transmission electron microscopy, and then tested the inhibition effects of ZnO-Se NPs on extracellular Mtb by colony-forming units (CFU) counting, bacterial ATP analysis, bacterial membrane potential analysis and scanning electron microscopy imaging. We also analyzed the effects of ZnO-Se NPs on the ROS production, mitochondrial membrane potential, apoptosis, autophagy, polarization and PI3K/Akt/mTOR signaling pathway of Mtb infected THP-1 macrophages. At last, we also tested the effects of ZnO-Se NPs on intracellular Mtb in THP-1 cells by colony-forming units (CFU) counting. ResultsThe obtained spherical core-shell ZnO-Se NPs with average diameters of 90 nm showed strong killing effects against extracellular Mtb, including BCG and the virulent H37Rv, by disrupting the ATP production, increasing the intracellular ROS level and destroying the membrane structures. More importantly, ZnO-Se NPs could also inhibit intracellular Mtb growth by promoting M1 polarization to increase the production of antiseptic nitric oxide and also promote apoptosis and autophagy of Mtb infected macrophages by increasing the intracellular ROS, disrupting mitochondria membrane potential and inhibiting PI3K/Akt/mTOR signaling pathway. DiscussionThese ZnO-Se NPs with synergetic anti-TB efficiency by combining the Mtb killing effects and host cell immunological inhibition effects were expected to serve as novel anti-TB agents for the development of more effective anti-TB strategy

From Vision to Reasoning: Probabilistic Analogical Mapping Between 3D Objects

We see the external world as consisting not only of objects and their parts, but also of relations that hold between them. Visual analogy, which depends on similarities between relations, provides a clear example of how perception supports reasoning. Here we report an experiment in which we quantitatively measured the human ability to find analogical mappings between parts of different objects, where the objects to be compared were drawn either from the same category (e.g., images of two mammals, such as a dog and a horse), or from two dissimilar categories (e.g., a chair image mapped to a cat image). Humans showed systematic mapping patterns, but with greater variability in mapping responses when objects were drawn from dissimilar categories. We simulated the human response of analogical mapping using a computational model of mapping between 3D objects, visiPAM (visual Probabilistic Analogical Mapping). VisiPAM takes point-cloud representations of two 3D objects as inputs, and outputs the mapping between analogous parts of the two objects. VisiPAM consists of a visual module that constructs structural representations of individual objects, and a reasoning module that identifies a probabilistic mapping between parts of the two 3D objects. Model simulations not only capture the qualitative pattern of human mapping performance cross conditions, but also approach human-level reliability in solving visual analogy problems

Human similarity judgments of emojis support alignment of conceptual systems across modalities

Humans can readily generalize their learning to new visual concepts, and infer their associated meanings. How do people align the different conceptual systems learned from different modalities? In the present paper, we examine emojis— pictographs uniquely situated between visual and linguistic modalities—to explore the role of alignment and multimodality in visual and linguistic semantics. Simulation experiments show that relational structures of emojis captured in visual and linguistic conceptual systems can be aligned, and that the ease of alignment increases as the number of emojis increases. We also found that emojis with subjective impressions of high popularity are easier to align between their visual and linguistic representations. A behavioral experiment was conducted to measure similarity patterns between 48 emojis, and to compare human similarity judgments with three models based on visual, semantic and multimodal-joint representations of emojis. We found that the model trained with multimodal data by aligning visual and semantic spaces best accounts for human judgments

Causal versus Associative Relations: Do Humans Perceive and Represent Them Differently?

Research has shown that visual diagrams facilitate people’s understanding of and communication about abstract relations. In addition, the distinction between causal versus associative relations is important in human reasoning However, previous research has not directly compared how humans represent these two types of relations through visual diagrams. The current study examined whether causal and associative relations differ with respect to how people cognitively represent and interpret them in a spatial context using diagrams. We found that participants perceived relatedness of causal relationships to be stronger than that of associative relationships. This difference was reflected in their drawing of diagrams. Participants connected variables that shared a causal relationship with a shorter line than they did with variables that shared an associative relationship. The results shed light on the difference between causal and associative relations, and suggest new directions for future research to explore the spatial component of causal reasoning

Visual Analogy: Deep Learning Versus Compositional Models

Is analogical reasoning a task that must be learned to solve from scratch by applying deep learning models to massive numbers of reasoning problems? Or are analogies solved by computing similarities between structured representations of analogs? We address this question by comparing human performance on visual analogies created using images of familiar three-dimensional objects (cars and their subregions) with the performance of alternative computational models. Human reasoners achieved above-chance accuracy for all problem types, but made more errors in several conditions (e.g., when relevant subregions were occluded). We compared human performance to that of two recent deep learning models (Siamese Network and Relation Network) directly trained to solve these analogy problems, as well as to that of a compositional model that assesses relational similarity between part-based representations. The compositional model based on part representations, but not the deep learning models, generated qualitative performance similar to that of human reasoners