Scaling Laws of RoPE-based Extrapolation

The extrapolation capability of Large Language Models (LLMs) based on Rotary Position Embedding is currently a topic of considerable interest. The mainstream approach to addressing extrapolation with LLMs involves modifying RoPE by replacing 10000, the rotary base of $\theta_n={10000}^{-2n/d}$ in the original RoPE, with a larger value and providing longer fine-tuning text. In this work, we first observe that fine-tuning a RoPE-based LLM with either a smaller or larger base in pre-training context length could significantly enhance its extrapolation performance. After that, we propose \textbf{\textit{Scaling Laws of RoPE-based Extrapolation}}, a unified framework from the periodic perspective, to describe the relationship between the extrapolation performance and base value as well as tuning context length. In this process, we also explain the origin of the RoPE-based extrapolation issue by \textbf{\textit{critical dimension for extrapolation}}. Besides these observations and analyses, we achieve extrapolation up to 1 million context length within only 16K training length on LLaMA2 7B and 13B.Comment: 26 pages, 12 figures, Accepted by ICLR 202

Interface-engineered hole doping in Sr2IrO4/LaNiO3 heterostructure

The relativistic Mott insulator Sr2IrO4 driven by large spin-orbit interaction is known for the Jeff = 1/2 antiferromagnetic state which closely resembles the electronic structure of parent compounds of superconducting cuprates. Here, we report the realization of hole-doped Sr2IrO4 by means of interfacial charge transfer in Sr2IrO4/LaNiO3 heterostructures. X-ray photoelectron spectroscopy on Ir 4f edge along with the X-ray absorption spectroscopy at Ni L2 edge confirmed that 5d electrons from Ir sites are transferred onto Ni sites, leading to markedly electronic reconstruction at the interface. Although the Sr2IrO4/LaNiO3 heterostructure remains non-metallic, we reveal that the transport behavior is no longer described by the Mott variable range hopping mode, but by the Efros-Shklovskii model. These findings highlight a powerful utility of interfaces to realize emerging electronic states of the Ruddlesden-Popper phases of Ir-based oxides.Comment: 9 pages including 3 figures and reference

Review of Terahertz Pulsed Imaging for Pharmaceutical Film Coating Analysis.

Terahertz pulsed imaging (TPI) was introduced approximately fifteen years ago and has attracted a lot of interest in the pharmaceutical industry as a fast, non-destructive modality for quantifying film coatings on pharmaceutical dosage forms. In this topical review, we look back at the use of TPI for analysing pharmaceutical film coatings, highlighting the main contributions made and outlining the key challenges ahead

Studying pharmaceutical tablets mixing process inside a perforated pan-coater using in-line terahertz sensing

In-line terahertz sensing has been demonstrated to measure the coating thickness of individual pharmaceutical tablets during coating operation. As coating uniformity is highly dependent on tablet mixing, this study presents our research progress to date on using in-line terahertz sensing to investigate the effects of baffle design, drum rotational speed and batch size, on tablet mixing inside a laboratory-sized perforated pan coater

Extracting the dielectric relaxation of water in thin Nafion membranes by terahertz spectroscopy

Terahertz time domain spectroscopy (THz-TDS) has been demonstrated to be capable of quantifying water uptake and retention properties of Nafion proton exchange membranes (PEMs). With a growing interest in thinner membranes, we analysis thin membranes to reveal its water uptake and retention propertie

Integrated line-field optical coherence tomography and scheimpflug imaging for corneal imaging

Previously we demonstrated a combined scanning-point optical coherence tomography (OCT) and Scheimpflug Imaging (SI) system, to provide cross-section images of the corneal layers and the whole anterior segment of the eye simultaneously. However, to fully realise the benefits of this dual imaging modality concept towards clinical practice, it is required that the OCT part of the system can achieve ultra-high axial resolution (<3 µm in air) and faster imaging speeds, without using prohibitively expensive components. To achieve this, a new imaging device integrating line-field OCT and SI, using a supercontinuum light source, is demonstrated. This line-field configuration enables both OCT and SI B-scan to be captured using the same light illumination in a single shot, which is the most significant improvement over single point scanning. This achieves an OCT axial resolution down to 2.1 µm in air (1.6 µm in corneal tissue) and an imaging speed up to 213 kA-Scans/s. Since the OCT and SI capture the identical corneal position and serve as cross-validation, this technique offers a great method for accurately determining the individual corneal thickness and refractive index, thereby minimizing individual variations, which can help determine the extent of cutting or correction needed in vision correction. Moreover, the precise measurement of corneal thickness provided by this technique allows for a better understanding of the biomechanical properties of the cornea. We demonstrate the efficacy of the proposed system by evaluating porcine and bovine eyes ex-vivo for determining the corneal refractive index and thickness

Synthesis of epitaxial magnetic pyrochlore heterojunctions

The synthesis of stoichiometric and epitaxial pyrochlore iridate thin films presents significant challenges yet is critical for unlocking experimental access to novel topological and magnetic states. Towards this goal, we unveil an in-situ two-stage growth mechanism that facilitates the synthesis of high-quality oriented pyrochlore iridate thin films. The growth starts with the deposition of a pyrochlore titanate as an active iso-structural template, followed by the application of an in-situ solid phase epitaxy technique in the second stage to accomplish the formation of single crystalline, large-area films. This novel protocol ensures the preservation of stoichiometry and structural homogeneity, leading to a marked improvement in surface and interface qualities over previously reported methods. The success of this synthesis approach is attributed to the application of directional laser-heat annealing, which effectively reorganizes the continuous random network of ions into a crystalline structure, as evidenced by our comprehensive analysis of the growth kinetics. This new synthesis approach advances our understanding of pyrochlore iridate film fabrication and opens a new perspective for investigating their unique physical properties.Comment: 11 pages, 4 figures; supplementary materials (1 table, 6 figures