Pre-training Contextualized World Models with In-the-wild Videos for Reinforcement Learning

Unsupervised pre-training methods utilizing large and diverse datasets have achieved tremendous success across a range of domains. Recent work has investigated such unsupervised pre-training methods for model-based reinforcement learning (MBRL) but is limited to domain-specific or simulated data. In this paper, we study the problem of pre-training world models with abundant in-the-wild videos for efficient learning of downstream visual control tasks. However, in-the-wild videos are complicated with various contextual factors, such as intricate backgrounds and textured appearance, which precludes a world model from extracting shared world knowledge to generalize better. To tackle this issue, we introduce Contextualized World Models (ContextWM) that explicitly model both the context and dynamics to overcome the complexity and diversity of in-the-wild videos and facilitate knowledge transfer between distinct scenes. Specifically, a contextualized extension of the latent dynamics model is elaborately realized by incorporating a context encoder to retain contextual information and empower the image decoder, which allows the latent dynamics model to concentrate on essential temporal variations. Our experiments show that in-the-wild video pre-training equipped with ContextWM can significantly improve the sample-efficiency of MBRL in various domains, including robotic manipulation, locomotion, and autonomous driving

Posterior hybrid surgery for atlantoaxial dislocation coexisting with multilevel cervical spondylotic myelopathy

BackgroundTo introduce a hybrid surgery of posterior craniovertebral fusion plus subaxial laminoplasty for atlantoaxial dislocation (AAD) coexisting with multilevel cervical spondylotic myelopathy (CSM).MethodsA retrospective study was performed by reviewing data from 23 patients with the coexistence of AAD and CSM who underwent the hybrid technique (n = 23). Clinical outcomes, including visual analogue scale (VAS), Japanese Orthopaedic Association (JOA), and neck disability index (NDI) score, and radiological cervical alignment parameters including C0–2 and C2–7 Cobb angle and range of motion (ROM) were analyzed. The operation time, blood loss, surgical levels, and complications were recorded.ResultsThe included patients were followed up with an average of 20.91 months (range, 12–36 months). Clinical outcomes including JOA, NDI, and VAS scores were significantly improved at different postoperative follow-up points. C0–2 Cobb angle, C2–7 Cobb angle, and ROM showed a stable tendency after 1-year follow-up. No major perioperative complications occurred.ConclusionThis study underlined the importance of pathologic condition of AAD coexisting with CSM and presented a novel hybrid approach of posterior craniovertebral fusion plus subaxial laminoplasty. This hybrid surgery was effective in achieving the desired clinical outcomes and better maintaining cervical alignment, proving its value and safety as an alternative technique

iTransformer: Inverted Transformers Are Effective for Time Series Forecasting

The recent boom of linear forecasting models questions the ongoing passion for architectural modifications of Transformer-based forecasters. These forecasters leverage Transformers to model the global dependencies over temporal tokens of time series, with each token formed by multiple variates of the same timestamp. However, Transformers are challenged in forecasting series with larger lookback windows due to performance degradation and computation explosion. Besides, the embedding for each temporal token fuses multiple variates that represent potential delayed events and distinct physical measurements, which may fail in learning variate-centric representations and result in meaningless attention maps. In this work, we reflect on the competent duties of Transformer components and repurpose the Transformer architecture without any modification to the basic components. We propose iTransformer that simply applies the attention and feed-forward network on the inverted dimensions. Specifically, the time points of individual series are embedded into variate tokens which are utilized by the attention mechanism to capture multivariate correlations; meanwhile, the feed-forward network is applied for each variate token to learn nonlinear representations. The iTransformer model achieves state-of-the-art on challenging real-world datasets, which further empowers the Transformer family with promoted performance, generalization ability across different variates, and better utilization of arbitrary lookback windows, making it a nice alternative as the fundamental backbone of time series forecasting. Code is available at this repository: https://github.com/thuml/iTransformer

Ka-Band LTCC Stacked Substrate Integrated Waveguide Bandpass Filter

A Ka-band substrate integrated waveguide bandpass filter has been designed and fabricated using low temperature co-fired ceramic (LTCC) technology. The in-house developed SICCAS-K5F3 material with a permittivity of 6.2 and a loss tangent of 0.002 was used. The size and surface area of the proposed bandpass filter are reduced by exploiting vertical coupling in vertically laminated three-dimensional structures. The coupling between adjacent cavities is realized by a narrow slot. A vertical transition structure between the coplanar-waveguide feed line and the substrate integrated waveguide is adopted to facilitate the internal signal connection. The demonstrated third-order filter has a compact size of 6.79 mm×4.13 mm×1.34 mm (0.63λ0  × 0.38λ0  × 0.12λ0) and exhibits good performance with a low insertion loss of 1.74 dB at 27.73 GHz and a 3 dB fractional bandwidth of 10 %

Comprehensive summary: the role of PBX1 in development and cancers

PBX1 is a transcription factor that can promote the occurrence of various tumors and play a reg-ulatory role in tumor growth, metastasis, invasion, and drug resistance. Furthermore, a variant generated by fusion of E2A and PBX1, E2A-PBX1, has been found in 25% of patients with childhood acute lymphoblastic leukemia. Thus, PBX1 is a potential therapeutic target for many cancers. Here, we describe the structure of PBX1 and E2A-PBX1 as well as the molecular mecha-nisms whereby these proteins promote tumorigenesis to provide future research directions for developing new treatments. We show that PBX1 and E2A-PBX1 induce the development of highly malignant and difficult-to-treat solid and blood tumors. The development of specific drugs against their targets may be a good therapeutic strategy for PBX1-related cancers. Furthermore, we strongly recommend E2A-PBX1 as one of the genes for prenatal screening to reduce the incidence of childhood hematological malignancies

Case report: Multiple arterial stenoses induced by autosomal-recessive hypophosphatemic rickets type 2 associated with mutation of ENPP1: a case study

Ectonucleotide pyrophosphatase/phosphodiesterase 1 (ENPP1)-related multiple arterial stenoses is a rare clinical syndrome in which global arterial calcification begins in infancy, with a high probability of early mortality, and hypophosphatemic rickets develops later in childhood. The vascular status of an ENPP1-mutated patient when they enter the rickets phase has not been thoroughly explored. In this study, we presented a case of an adolescent with an ENPP1 mutation who complained of uncontrolled hypertension. Systematic radiography showed renal, carotid, cranial, and aortic stenoses as well as random calcification foci on arterial walls. The patient was incorrectly diagnosed with Takayasu’s arteritis, and cortisol therapy had little effect on reducing the vascular stenosis. As a result, phosphate replacement, calcitriol substitution, and antihypertensive medication were prescribed, and the patient was discharged for further examination. This research presented the vascular alterations of an ENPP1-mutanted patient, and while there is less calcification, intimal thickening may be the primary cause of arterial stenosis

Single domain antibody multimers confer protection against rabies infection

Post-exposure prophylactic (PEP) neutralizing antibodies against Rabies are the most effective way to prevent infection-related fatality. The outer envelope glycoprotein of the Rabies virus (RABV) is the most significant surface antigen for generating virus-neutralizing antibodies. The small size and uncompromised functional specificity of single domain antibodies (sdAbs) can be exploited in the fields of experimental therapeutic applications for infectious diseases through formatting flexibilities to increase their avidity towards target antigens. In this study, we used phage display technique to select and identify sdAbs that were specific for the RABV glycoprotein from a naïve llama-derived antibody library. To increase their neutralizing potencies, the sdAbs were fused with a coiled-coil peptide derived from the human cartilage oligomeric matrix protein (COMP48) to form homogenous pentavalent multimers, known as combodies. Compared to monovalent sdAbs, the combodies, namely 26424 and 26434, exhibited high avidity and were able to neutralize 85-fold higher input of RABV (CVS-11 strain) pseudotypes in vitro, as a result of multimerization, while retaining their specificities for target antigen. 26424 and 26434 were capable of neutralizing CVS-11 pseudotypes in vitro by 90–95% as compared to human rabies immunoglobulin (HRIG), currently used for PEP in Rabies. The multimeric sdAbs were also demonstrated to be partially protective for mice that were infected with lethal doses of rabies virus in vivo. The results demonstrate that the combodies could be valuable tools in understanding viral mechanisms, diagnosis and possible anti-viral candidate for RABV infection

Laser generated electron transport experiment in a novel wire nail target

The transport of high intensity (2x1020 W/cm2) laser generated relativistic electrons with a solid target has been studied in a novel geometry. The targets were 20 um diameter solid copper wires, coated with ~ 2um of titanium, with an 80 um diameter hemispherical termination. They were illuminated by an ~500fs, ~200J pulse of 1.053um laser light focused to a ~ 20um diameter spot centered on the flat face of the hemisphere. K-alpha fluorescence from the Cu and Ti regions was imaged together with extreme ultraviolet (X-UV) emission at 68 and 256eV. Results showed a quasi exponential decline in K-alpha emission along the wire over a distance of a few hundred microns from the laser focus, consistent with bulk Ohmic inhibition of the relativistic electron transport. Weaker Ka and X-UV emission on a longer scale length showed limb brightening suggesting a transition to enhanced transport at the surface of the wire

A Highly Conserved, Small LTR Retrotransposon that Preferentially Targets Genes in Grass Genomes

LTR retrotransposons are often the most abundant components of plant genomes and can impact gene and genome evolution. Most reported LTR retrotransposons are large elements (>4 kb) and are most often found in heterochromatic (gene poor) regions. We report the smallest LTR retrotransposon found to date, only 292 bp. The element is found in rice, maize, sorghum and other grass genomes, which indicates that it was present in the ancestor of grass species, at least 50–80 MYA. Estimated insertion times, comparisons between sequenced rice lines, and mRNA data indicate that this element may still be active in some genomes. Unlike other LTR retrotransposons, the small LTR retrotransposons (SMARTs) are distributed throughout the genomes and are often located within or near genes with insertion patterns similar to MITEs (miniature inverted repeat transposable elements). Our data suggests that insertions of SMARTs into or near genes can, in a few instances, alter both gene structures and gene expression. Further evidence for a role in regulating gene expression, SMART-specific small RNAs (sRNAs) were identified that may be involved in gene regulation. Thus, SMARTs may have played an important role in genome evolution and genic innovation and may provide a valuable tool for gene tagging systems in grass