EVA-CLIP: Improved Training Techniques for CLIP at Scale

Contrastive language-image pre-training, CLIP for short, has gained increasing attention for its potential in various scenarios. In this paper, we propose EVA-CLIP, a series of models that significantly improve the efficiency and effectiveness of CLIP training. Our approach incorporates new techniques for representation learning, optimization, and augmentation, enabling EVA-CLIP to achieve superior performance compared to previous CLIP models with the same number of parameters but significantly smaller training costs. Notably, our largest 5.0B-parameter EVA-02-CLIP-E/14+ with only 9 billion seen samples achieves 82.0 zero-shot top-1 accuracy on ImageNet-1K val. A smaller EVA-02-CLIP-L/14+ with only 430 million parameters and 6 billion seen samples achieves 80.4 zero-shot top-1 accuracy on ImageNet-1K val. To facilitate open access and open research, we release the complete suite of EVA-CLIP to the community at https://github.com/baaivision/EVA/tree/master/EVA-CLIP.Comment: To Rei and the moon. Code & Models: https://github.com/baaivision/EVA/tree/master/EVA-CLI

Characterization of marine-terrigenous transitional Taiyuan formation shale reservoirs in Hedong coal field, China

To better understand the basic characteristics of Marine-terrigenous Transitional Taiyuan formation shale (TYS) reservoirs in Hedong coal field, a series of reservoir evaluation experiments were conducted on 33 core samples, which were collected from an exploration shale gas well (SL-1). The results show that organic matters in TYS are Type III gas prone kerogen and are in the high-maturity stage with an average Ro value of 1.87% (ranging from 1.71% to 2.10%). The total organic carbon (TOC) is ranging from 0.29% to 11.87% with an average value of 2.91% and gas content is from 0.41% to 2.96 ml/g, which indicates that TYS still has certain hydrocarbon generation potential despite a mass generation of hydrocarbons occurred during the geological history. X-ray diffraction analysis shows that TYS is composed mainly of quartz minerals and clay minerals with an average brittleness index of 46.5%, which is relatively favorable for hydraulic fracture. Pore size of TYS ranges from a few nanometers to hundreds of nanometers. The permeability is irrelevant with porosity and its values are all lower than 0.1 md. The average value of Brunauer- Emmett-Teller surface area and Barrett-Joyner-Halendar volumes are 8.57 m2 /g and 1.84 cm3 /100g, respectively. Similar to previous studies, TOC content is a decisive control on gas adsorption capacity in this study.Cited as: Li, K., Chen, G., Li, W., Wu, X., Tan, J., Qu, J. Characterization of marine-terrigenous transitional Taiyuan formation shale reservoirs in Hedong coal field, China. Advances in Geo-Energy Research, 2018, 2(1): 72-85, doi: 10.26804/ager.2018.01.0

Preliminary study of improving immune tolerance in vivo of bioprosthetic heart valves through a novel antigenic removal method

The durability of bioprosthetic heart valves is always compromised by the inherent antigenicity of biomaterials. Decellularization has been a promising approach to reducing the immunogenicity of biological valves. However, current methods are insufficient in eliminating all immunogenicity from the biomaterials, necessitating the exploration of novel techniques. In this study, we investigated using a novel detergent, fatty alcohol polyoxyethylene ether sodium sulfate (AES), to remove antigens from bovine pericardium. Our results demonstrated that AES treatment achieved a higher pericardial antigen removal rate than traditional detergent treatments while preserving the mechanical properties and biocompatibility of the biomaterials. Moreover, we observed excellent immune tolerance in the in vivo rat model. Overall, our findings suggest that AES treatment is a promising method for preparing biological valves with ideal clinical application prospects

Research progress of 3D printed poly (ether ether ketone) in the reconstruction of craniomaxillofacial bone defects

The clinical challenge of bone defects in the craniomaxillofacial region, which can lead to significant physiological dysfunction and psychological distress, persists due to the complex and unique anatomy of craniomaxillofacial bones. These critical-sized defects require the use of bone grafts or substitutes for effective reconstruction. However, current biomaterials and methods have specific limitations in meeting the clinical demands for structural reinforcement, mechanical support, exceptional biological performance, and aesthetically pleasing reconstruction of the facial structure. These drawbacks have led to a growing need for novel materials and technologies. The growing development of 3D printing can offer significant advantages to address these issues, as demonstrated by the fabrication of patient-specific bioactive constructs with controlled structural design for complex bone defects in medical applications using this technology. Poly (ether ether ketone) (PEEK), among a number of materials used, is gaining recognition as a feasible substitute for a customized structure that closely resembles natural bone. It has proven to be an excellent, conformable, and 3D-printable material with the potential to replace traditional autografts and titanium implants. However, its biological inertness poses certain limitations. Therefore, this review summarizes the distinctive features of craniomaxillofacial bones and current methods for bone reconstruction, and then focuses on the increasingly applied 3D printed PEEK constructs in this field and an update on the advanced modifications for improved mechanical properties, biological performance, and antibacterial capacity. Exploring the potential of 3D printed PEEK is expected to lead to more cost-effective, biocompatible, and personalized treatment of craniomaxillofacial bone defects in clinical applications

Integral convergence of the higher-order theory for solitary waves

An exact analytic solution for a solitary wave of arbitrary height is attained by series expansions of flow variables based on parameter ε = k²h², (k being the wave number of the solitary wave on water of uniform depth h) by orders in O(ε^n) up to n = 25. Its convergence behavior is found first to yield a set of asymptotic representations for all the flow variables, each and every becoming highest in accuracy at O(ε^(17). For n > 17, the field variables and wave parameters, e.g., wave amplitude, have their errors continue increasing with n, but, in sharp contrast, all the wave integral properties including the excess mass first undergo finite fluctuations from O(ε^(17) to O(ε^(20), then all converge uniformly beyond in a group of tight bundle within the range 0 < ε < 0.283, with ε = 0.283 corresponding to the highest solitary wave with a 120° vertex angle. This remarkable behavior of series convergence seems to have no precedent, and furthermore, is unique in ε, not shared by the exact solutions based on all other parameters examined here

Integral convergence of the higher-order theory for solitary waves

An exact analytic solution for a solitary wave of arbitrary height is attained by series expansions of flow variables based on parameter ε = k²h², (k being the wave number of the solitary wave on water of uniform depth h) by orders in O(ε^n) up to n = 25. Its convergence behavior is found first to yield a set of asymptotic representations for all the flow variables, each and every becoming highest in accuracy at O(ε^(17). For n > 17, the field variables and wave parameters, e.g., wave amplitude, have their errors continue increasing with n, but, in sharp contrast, all the wave integral properties including the excess mass first undergo finite fluctuations from O(ε^(17) to O(ε^(20), then all converge uniformly beyond in a group of tight bundle within the range 0 < ε < 0.283, with ε = 0.283 corresponding to the highest solitary wave with a 120° vertex angle. This remarkable behavior of series convergence seems to have no precedent, and furthermore, is unique in ε, not shared by the exact solutions based on all other parameters examined here