Large-scale Multi-Modal Pre-trained Models: A Comprehensive Survey

With the urgent demand for generalized deep models, many pre-trained big models are proposed, such as BERT, ViT, GPT, etc. Inspired by the success of these models in single domains (like computer vision and natural language processing), the multi-modal pre-trained big models have also drawn more and more attention in recent years. In this work, we give a comprehensive survey of these models and hope this paper could provide new insights and helps fresh researchers to track the most cutting-edge works. Specifically, we firstly introduce the background of multi-modal pre-training by reviewing the conventional deep learning, pre-training works in natural language process, computer vision, and speech. Then, we introduce the task definition, key challenges, and advantages of multi-modal pre-training models (MM-PTMs), and discuss the MM-PTMs with a focus on data, objectives, network architectures, and knowledge enhanced pre-training. After that, we introduce the downstream tasks used for the validation of large-scale MM-PTMs, including generative, classification, and regression tasks. We also give visualization and analysis of the model parameters and results on representative downstream tasks. Finally, we point out possible research directions for this topic that may benefit future works. In addition, we maintain a continuously updated paper list for large-scale pre-trained multi-modal big models: https://github.com/wangxiao5791509/MultiModal_BigModels_SurveyComment: Accepted by Machine Intelligence Researc

3D printed milk protein food simulant: improving the printing performance of milk protein concentration by incorporating whey protein isolate

This paper aimed to establish a milk protein based 3D printing food simulant and investigated the effect of whey protein isolate (WPI) concentration on the printing performance of milk protein concentrate (MPC). WPI and MPC powders at different ratios were prepared in paste (35 wt%, total dry matter content). The rheological properties and water distribution of protein matrix prepared with different MPC/WPI ratios were characterized with a rheometer and low field nuclear magnetic resonance (LF-NMR), respectively. Moreover, the variations in the microstructure of printed objects were observed with a scanning electron microscope (SEM). The printed objects showed different appearance and physical properties; the printing fidelity was also evaluated by measuring the geometric accuracy of printed objects. The rheological and texture data showed that the presence of WPI could reduce the apparent viscosity and soften the MPC paste, benefiting the printing process. The results showed that the milk powder paste mixture prepared with MPC/WPI at a ratio of 5/2 was the most desirable material for extrusion-based 3D printing, which could be successfully printed and matched the designed 3D model best. Industrial relevance: 3D printing in food sector has been an attractive and emerging technology owing to its potential advantages, such as customized food designs, personalized and digitalized nutrition, simplifying supply chain and so on. This paper established a high protein food simulant for 3D printing, optimized its printing performance with whey protein isolate, and studied the physicochemical property of prepared protein pastes. The overall results indicated that milk protein powders could be the promising materials for the application in food 3D printing. In flowing studies or practical production, the glycerol could be replaced by ingredients such as syrup, honey etc. This study may give more insights into 3D printing applied in food sector and facilitate the further developments of 3D food printing

Impact of grazing on shaping abundance and composition of active methanotrophs and methane oxidation activity in a grassland soil

The effect of grazing on the abundance, composition, and methane (CH4) uptake of methanotrophs in grasslands has been well documented in the past few decades, but the dominant communities of active methanotrophs responsible for CH4 oxidation activity in grazed soils are still poorly understood. In this study, we characterized the metabolically active, aerobic methanotrophs in grasslands with three different levels of grazing (light, medium, and heavy) by combining DNA-stable isotope probing (SIP) and quantitative PCR (qPCR) for methane monooxygenase (pmoA) gene– and 16S rRNA gene–based amplicon sequencing. The CH4 oxidation potential was as low as 0.51 μmol g dry weight−1 day−1 in the ungrazed control, while it decreased as grazing intensity increased in grazed fields, ranging from 2.25 μmol g dry weight−1 day−1 in light grazed fields to 1.59 in heavily grazed fields. Increased CH4 oxidation activity was paralleled by twofold increases in abundance of pmoA genes and relative abundance of methanotroph-affiliated 16S rRNA genes in the total microbial community in grazed soils. SIP and sequencing revealed that the genera Methylobacter and Methylosarcina (type I; Gammaproteobacteria) and Methylocystis (type II; Alphaproteobacteria) were active methanotrophs responsible for CH4 oxidation in grazed soils. Light and intermediate grazing stimulated the growth and activity of methanotrophs, while heavy grazing decreased the abundance and diversity of the active methanotrophs in the typical steppe. Redundancy and correlation analysis further indicated that the variation of bulk density and soil C and N induced by grazing determined the abundance, diversity of active methanotrophs, and methane oxidation activity in the long-term grazed grassland soil

Multi-level navigation for curriculum planning in intelligent tutoring systems

This paper presents and investigates the application of multi-level planning techniques, together with meta and micro level knowledge architecture model, in building the domain knowledge, planning and navigate the curriculum of an intelligent tutoring system (ITS). Curriculum issues have been important to ITS. In order to truly individualize instruction, ITS must be able to reason about the curriculum, understand its implications, and be able to dynamically redesign the curriculum. This leads to the dramatic increase in the information an planner handles. The multi-level planning, made possible by the meta and micro level knowledge model, efficiently manages large domain knowledge base, in that it largely eases the planning task both in complexity and in storage

Physical and Electrochemical Performances of Cold Sprayed Pb Electrodes

Titanium-based PbO2 electrodes are widely used for chemical industries, such as electrodialysis, electrolysis, and electrodepositing, to improve the mechanical and life cycle properties of Pb metal electrodes. However, PbO2 electrodes are usually electrodeposited onto rigid metals due to its soft characteristic, which results in severe passivation problems requiring thin thickness and high porosity. It is of great importance to develop a rigid Pb metal electrode system since thermal spraying and welding methods fail to manufacture such a promising electrode. In the present work, the cold spraying method was used to deposit a pure Pb metal coating with thickness of above 500 μm on Q235 steel substrate. The coating has good physical performances, the porosity is less than 1%, and the bonding strength ranges from 6.25 to 7.75 MPa. The cross-sectional morphology suggests that no through-thickness pores exist in the coating. The oxygen evolution potential is larger than 1.5 V vs. SCE, which is similar to the potential of the titanium-based PbO2 electrode. Dynamic polarization curves and cyclic voltammetry curves of coated sample in sodium sulfate solution indicate that cold sprayed Pb coating is a good electrode for electrochemical reduction reactions. All our results mean that cold spraying is capable of manufacturing electrode materials for electrochemical industries

CNN Based Vehicle Counting with Virtual Coil in Traffic Surveillance Video

This paper presents an efficient method of vehicle counting based on convolutional neural network (CNN) with virtual coils. Within virtual coils, foreground is obtained by background substraction. Vehicle is then detected by voting of virtual coil sub-regions. To deal with vehicle cross-lane cases, a cascade classifier combining connected component analysis (CCA) and CNN is adopted. Experiments are carried out on seven real traffic videos. The proposed approach works well on recognizing cross-lane vehicles, achieving above 90% accuracy with real-time processing speed.EICPCI-S(ISTP)[email protected]; [email protected]; [email protected]

Numerical Analysis of Factors Affecting the Burden Surface and Porosity Distribution in the Upper Part of the Blast Furnace

A proper burden and porosity distribution of the bed in the upper shaft are important prerequisites for realizing a stable and efficient operation of the ironmaking blast furnace. The discrete element method was used to investigate the effects of the static friction coefficient between burden particles and shaft angle on the burden profile and porosity distribution in the bed formed by charging the burden with a bell-less charging equipment. The results indicate that a large static friction coefficient makes the particles stay closer to the impact point (i.e., where they fall) from the rotating chute. A large mixed region of the burden bed decreases the gas permeability, and an increase in the burden particle roughness will worsen this problem. The burden surface shape becomes flatter with an increase in the shaft angle. These findings explain the effect of particle properties and wall geometry on the inner structure of the burden bed

Numerical Analysis of the Relationship between Friction Coefficient and Repose Angle of Blast Furnace Raw Materials by Discrete Element Method

In recent years, the discrete element method (DEM) has been widely used to study the factors affecting the repose angle and calibrate particle parameters for simulations. In this paper, DEM is used to study the effects of the coefficient of rolling and static friction of pellet, sinter and coke particles on the repose angle. By comparison of the results of simulations and physical experiments, the coefficients of rolling and static friction suitable for simulation work are determined. The results demonstrate that repose angle increases with the coefficient of rolling and static friction, but the rate of increase gradually decays, when the coefficient of rolling friction exceeds 0.4 or the coefficient of static friction exceeds 0.35. The coefficient of static friction has a greater impact on the repose angle than the coefficient of rolling friction. The rougher of the base surface, the larger the repose angle of the formed particle piled. It can be concluded that appropriate coefficient of rolling and static friction for simulations can be obtained by the outlined procedure

Multi-view gait recognition with incomplete training data

Changes in the viewing angles pose a major challenge for gait recognition because the human gait silhouettes can be different under the various viewing angles. Recently, View Transformation Model (VTM) was proposed to tackle this problem by transforming gait features from across views to a common viewing angle. However, VTM must use the data of subjects crossing all views to train the pre-constructed model, which might be unsuitable for the real applications. To address this problem, this paper proposes a View Feature Recovering Model (VFRM) to generate the VTM with incomplete training data. In our algorithm, if the gait signature of a pedestrian is missing under a view, it can be recovered from the K-nearest pedestrians whose gait features are available in the same view. Moreover, the Geodesic distance based K-Nearest Neighbor (GKNN) algorithm is adopted in our algorithm to better measure the neighborhood between two pedestrians. Experimental results on a benchmark database has demonstrated the effectiveness of our method.EICPCI-S(ISTP)[email protected]