Rethinking Query, Key, and Value Embedding in Vision Transformer under Tiny Model Constraints

A vision transformer (ViT) is the dominant model in the computer vision field. Despite numerous studies that mainly focus on dealing with inductive bias and complexity, there remains the problem of finding better transformer networks. For example, conventional transformer-based models usually use a projection layer for each query (Q), key (K), and value (V) embedding before multi-head self-attention. Insufficient consideration of semantic $Q, K$, and $V$ embedding may lead to a performance drop. In this paper, we propose three types of structures for $Q$, $K$, and $V$ embedding. The first structure utilizes two layers with ReLU, which is a non-linear embedding for $Q, K$, and $V$. The second involves sharing one of the non-linear layers to share knowledge among $Q, K$, and $V$. The third proposed structure shares all non-linear layers with code parameters. The codes are trainable, and the values determine the embedding process to be performed among $Q$, $K$, and $V$. Hence, we demonstrate the superior image classification performance of the proposed approaches in experiments compared to several state-of-the-art approaches. The proposed method achieved $71.4\%$ with a few parameters (of $3.1M$) on the ImageNet-1k dataset compared to that required by the original transformer model of XCiT-N12 ($69.9\%$). Additionally, the method achieved $93.3\%$ with only $2.9M$ parameters in transfer learning on average for the CIFAR-10, CIFAR-100, Stanford Cars datasets, and STL-10 datasets, which is better than the accuracy of $92.2\%$ obtained via the original XCiT-N12 model

Rapid Changes of Photospheric Magnetic Field after Tether-Cutting Reconnection and Magnetic Implosion

The rapid, irreversible change of the photospheric magnetic field has been recognized as an important element of the solar flare process. This Letter reports such a rapid change of magnetic fields during the 2011 February 13 M6.6 flare in NOAA AR 11158 that we found from the vector magnetograms of the Helioseismic and Magnetic Imager with 12-min cadence. High-resolution magnetograms of Hinode that are available at ~-5.5, -1.5, 1.5, and 4 hrs relative to the flare maximum are used to reconstruct three-dimensional coronal magnetic field under the nonlinear force-free field (NLFFF) assumption. UV and hard X-ray images are also used to illuminate the magnetic field evolution and energy release. The rapid change is mainly detected by HMI in a compact region lying in the center of the magnetic sigmoid, where the mean horizontal field strength exhibited a significant increase by 28%. The region lies between the initial strong UV and hard X-ray sources in the chromosphere, which are cospatial with the central feet of the sigmoid according to the NLFFF model. The NLFFF model further shows that strong coronal currents are concentrated immediately above the region, and that more intriguingly, the coronal current system underwent an apparent downward collapse after the sigmoid eruption. These results are discussed in favor of both the tether-cutting reconnection producing the flare and the ensuing implosion of the coronal field resulting from the energy release.Comment: 7 pages, 5 figures, accepted to the Astrophysical Journal Letter

Semantic Map Guided Synthesis of Wireless Capsule Endoscopy Images using Diffusion Models

Wireless capsule endoscopy (WCE) is a non-invasive method for visualizing the gastrointestinal (GI) tract, crucial for diagnosing GI tract diseases. However, interpreting WCE results can be time-consuming and tiring. Existing studies have employed deep neural networks (DNNs) for automatic GI tract lesion detection, but acquiring sufficient training examples, particularly due to privacy concerns, remains a challenge. Public WCE databases lack diversity and quantity. To address this, we propose a novel approach leveraging generative models, specifically the diffusion model (DM), for generating diverse WCE images. Our model incorporates semantic map resulted from visualization scale (VS) engine, enhancing the controllability and diversity of generated images. We evaluate our approach using visual inspection and visual Turing tests, demonstrating its effectiveness in generating realistic and diverse WCE images

Flare differentially rotates sunspot on Sun's surface

Sunspots are concentrations of magnetic field visible on the solar surface (photosphere). It was considered implausible that solar flares, as resulted from magnetic reconnection in the tenuous corona, would cause a direct perturbation of the dense photosphere involving bulk motion. Here we report the sudden flare-induced rotation of a sunspot using the unprecedented spatiotemporal resolution of the 1.6 m New Solar Telescope, supplemented by magnetic data from the Solar Dynamics Observatory. It is clearly observed that the rotation is non-uniform over the sunspot: as the flare ribbon sweeps across, its different portions accelerate (up to ∼50° h−1) at different times corresponding to peaks of flare hard X-ray emission. The rotation may be driven by the surface Lorentz-force change due to the back reaction of coronal magnetic restructuring and is accompanied by a downward Poynting flux. These results have direct consequences for our understanding of energy and momentum transportation in the flare-related phenomena

The variation of relative magnetic helicity around major flares

We have investigated the variation of magnetic helicity over a span of several days around the times of 11 X-class flares which occurred in seven active regions (NOAA 9672, 10030, 10314, 10486, 10564, 10696, and 10720) using the magnetograms taken by the Michelson Doppler Imager (MDI) on board the Solar and Heliospheric Observatory (SOHO). As a major result we found that each of these major flares was preceded by a significant helicity accumulation over a long period (0.5 to a few days). Another finding is that the helicity accumulates at a nearly constant rate and then becomes nearly constant before the flares. This led us to distinguish the helicity variation into two phases: a phase of monotonically increasing helicity and the following phase of relatively constant helicity. As expected, the amount of helicity accumulated shows a modest correlation with time-integrated soft X-ray flux during flares. However, the average helicity change rate in the first phase shows even stronger correlation with the time-integrated soft X-ray flux. We discuss the physical implications of this result and the possibility that this characteristic helicity variation pattern can be used as an early warning sign for solar eruptions