Retrieving Multimodal Information for Augmented Generation: A Survey

As Large Language Models (LLMs) become popular, there emerged an important trend of using multimodality to augment the LLMs' generation ability, which enables LLMs to better interact with the world. However, there lacks a unified perception of at which stage and how to incorporate different modalities. In this survey, we review methods that assist and augment generative models by retrieving multimodal knowledge, whose formats range from images, codes, tables, graphs, to audio. Such methods offer a promising solution to important concerns such as factuality, reasoning, interpretability, and robustness. By providing an in-depth review, this survey is expected to provide scholars with a deeper understanding of the methods' applications and encourage them to adapt existing techniques to the fast-growing field of LLMs

Atomic-scale observation of localized phonons at FeSe/SrTiO3 interface

In single unit-cell FeSe grown on SrTiO3, the superconductivity transition temperature features a significant enhancement. Local phonon modes at the interface associated with electron-phonon coupling may play an important role in the interface-induced enhancement. However, such phonon modes have eluded direct experimental observations. Indeed, the complicated atomic structure of the interface brings challenges to obtain the accurate structure-phonon relation knowledge from either experiment or theory, thus hindering our understanding of the enhancement mechanism. Here, we achieve direct characterizations of atomic structure and phonon modes at the FeSe/SrTiO3 interface with atomically resolved imaging and electron energy loss spectroscopy in a scanning transmission electron microscope. We find several phonon modes highly localized (~1.3 nm) at the unique double layer Ti-O termination at the interface, one of which (~ 83 meV) engages in strong interactions with the electrons in FeSe based on ab initio calculations. The electron-phonon coupling strength for such a localized interface phonon with short-range interactions is comparable to that of Fuchs-Kliewer (FK) phonon mode with long-rang interactions. Thus, our atomic-scale study provides new insights into understanding the origin of superconductivity enhancement at the FeSe/SrTiO3 interface

Tunable Interband Transitions in Twisted h-BN/Graphene Heterostructures

In twisted h-BN/graphene heterostructures, the complex electronic properties of the fast-traveling electron gas in graphene are usually considered to be fully revealed. However, the randomly twisted heterostructures may also have unexpected transition behaviors, which may influence the device performance. Here, we study the twist angle-dependent coupling effects of h-BN/graphene heterostructures using monochromatic electron energy loss spectroscopy. We find that the moir\'e potentials alter the band structure of graphene, resulting in a redshift of the intralayer transition at the M-point, which becomes more pronounced up to 0.25 eV with increasing twist angle. Furthermore, the twisting of the Brillouin zone of h-BN relative to the graphene M-point leads to tunable vertical transition energies in the range of 5.1-5.6 eV. Our findings indicate that twist-coupling effects of van der Waals heterostructures should be carefully considered in device fabrications, and the continuously tunable interband transitions through the twist angle can serve as a new degree of freedom to design optoelectrical devices

Cholesterol 25-hydroxylase suppresses SARS-CoV-2 replication by blocking membrane fusion

Cholesterol 25-hydroxylase (CH25H) is an interferon (IFN)-stimulated gene that shows broad antiviral activities against a wide range of enveloped viruses. Here, using an IFN-stimulated gene screen against vesicular stomatitis virus (VSV)-SARS-CoV and VSV-SARS-CoV-2 chimeric viruses, we identified CH25H and its enzymatic product 25-hydroxycholesterol (25HC) as potent inhibitors of SARS-CoV-2 replication. Internalized 25HC accumulates in the late endosomes and potentially restricts SARS-CoV-2 spike protein catalyzed membrane fusion via blockade of cholesterol export. Our results highlight one of the possible antiviral mechanisms of 25HC and provide the molecular basis for its therapeutic development

Robust estimation of bacterial cell count from optical density

Optical density (OD) is widely used to estimate the density of cells in liquid culture, but cannot be compared between instruments without a standardized calibration protocol and is challenging to relate to actual cell count. We address this with an interlaboratory study comparing three simple, low-cost, and highly accessible OD calibration protocols across 244 laboratories, applied to eight strains of constitutive GFP-expressing E. coli. Based on our results, we recommend calibrating OD to estimated cell count using serial dilution of silica microspheres, which produces highly precise calibration (95.5% of residuals <1.2-fold), is easily assessed for quality control, also assesses instrument effective linear range, and can be combined with fluorescence calibration to obtain units of Molecules of Equivalent Fluorescein (MEFL) per cell, allowing direct comparison and data fusion with flow cytometry measurements: in our study, fluorescence per cell measurements showed only a 1.07-fold mean difference between plate reader and flow cytometry data

Collaboration and Task Planning of Turtle-Inspired Multiple Amphibious Spherical Robots

Amphibious Spherical Robots (ASRs) use an electric field to communicate and collaborate effectively in a turbid water of confined spaces where other mode communication modalities failed. This paper proposes an embedded architecture formation strategy for a group of turtle-inspired amphibious robots to maintain a long distance-parameterized path based on dynamic visual servoing. Inspired by this biological phenomenon, we design an artificial multi-robot cooperative mode and explore an electronic communication and collaborate devices, the control method is based in particular on underwater environment and also conduct a detailed analysis of control motion module. The objectives of control strategies are divided into four categories: The first strategy is that the leader robot controls the action of the overall robots to maintain collaborate together during motion along a desired geometric path and to follow a timing law that the communication efficiency and the arrival times to assigned sites. Furthermore, we design an adaptive visual servoing controller for trajectory tracking task, taking into account system dynamics with environment interactions. After that, the third strategy is a centralized optimization algorithm for the redistribution of target mission changes. Finally, this paper also proposes a new method of control strategies in order to guarantee that each robot in the team moves together according to the preset target toward its location in the group formation based on communication and stability modules

Multiple Bio-Inspired Father–Son Underwater Robot for Underwater Target Object Acquisition and Identification

Underwater target acquisition and identification performed by manipulators having broad application prospects and value in the field of marine development. Conventional manipulators are too heavy to be used for small target objects and unsuitable for shallow sea working. In this paper, a bio-inspired Father–Son Underwater Robot System (FURS) is designed for underwater target object image acquisition and identification. Our spherical underwater robot (SUR), as the father underwater robot of the FURS, has the ability of strong dynamic balance and good maneuverability, can realize approach the target area quickly, and then cruise and surround the target object. A coiling mechanism was installed on SUR for the recycling and release of the son underwater robot. A Salamandra-inspired son underwater robot is used as the manipulator of the FURS, which is connected to the spherical underwater robot by a tether. The son underwater robot has multiple degrees of freedom and realizes both swimming and walking movement modes. The son underwater robot can move to underwater target objects. The vision system is installed to enable the FURS to acquire the image information of the target object with the aid of the camera, and also to identify the target object. Finally, verification experiments are conducted in an indoor water tank and outdoor swimming pool conditions to verify the effectiveness of the proposed in this paper