Edge-MoE: Memory-Efficient Multi-Task Vision Transformer Architecture with Task-level Sparsity via Mixture-of-Experts

Computer vision researchers are embracing two promising paradigms: Vision Transformers (ViTs) and Multi-task Learning (MTL), which both show great performance but are computation-intensive, given the quadratic complexity of self-attention in ViT and the need to activate an entire large MTL model for one task. M$^3$ViT is the latest multi-task ViT model that introduces mixture-of-experts (MoE), where only a small portion of subnetworks ("experts") are sparsely and dynamically activated based on the current task. M$^3$ViT achieves better accuracy and over 80% computation reduction but leaves challenges for efficient deployment on FPGA. Our work, dubbed Edge-MoE, solves the challenges to introduce the first end-to-end FPGA accelerator for multi-task ViT with a collection of architectural innovations, including (1) a novel reordering mechanism for self-attention, which requires only constant bandwidth regardless of the target parallelism; (2) a fast single-pass softmax approximation; (3) an accurate and low-cost GELU approximation; (4) a unified and flexible computing unit that is shared by almost all computational layers to maximally reduce resource usage; and (5) uniquely for M$^3$ViT, a novel patch reordering method to eliminate memory access overhead. Edge-MoE achieves 2.24x and 4.90x better energy efficiency comparing with GPU and CPU, respectively. A real-time video demonstration is available online, along with our code written using High-Level Synthesis, which will be open-sourced.Comment: 11 pages, 12 figures. Submitted to ICCAD 202

A Survey on Off-chain Networks: Frameworks, Technologies, Solutions and Challenges

Blockchain has received increasing attention in academia and industry. However, the increasing transaction volumes and limited on-chain storage underscore scalability as a key challenge hindering the widespread adoption of blockchain. Fortunately, off-chain networks that enable transactions outside the blockchain show promising potential to mitigate the scalability challenge. Off-chain solutions that address blockchain scalability hurdles, such as payment channel networks, facilitate secure and fast off-chain transactions, thus relieving the main chain's strain. In this article, we provide a comprehensive review of key technologies, solutions, and challenges of off-chain networks. First, we introduce the background of off-chain networks encompassing design motivation, framework, overview, and application scenarios. We then review the key issues and technologies associated with off-chain networks. Subsequently, we summarize the mainstream solutions for the corresponding key issues. Finally, we discuss some research challenges and open issues in this area.Comment: 30 pages, 5 figure

Pancreatic cancer-initiating cell exosome message transfer into noncancer-initiating cells: the importance of CD44v6 in reprogramming

Background: Cancer-initiating cell (CIC) exosomes (CIC-TEX) are suggested reprogramming Non-CIC. Mode of message transfer and engagement of CIC-markers being disputed, we elaborated the impact of CD44v6 and Tspan8 on the response of Non-CIC. Methods: Non-metastasizing CD44v6- and Tspan8-knockdown (kd) pancreatic cancer cells served as Non-CIC. CIC-TEX coculture-induced changes were evaluated by deep-sequencing and functional assays. Tumor progression was surveyed during in vivo CIC-TEX treatment. Results: Deep-sequencing of CIC-TEX-cocultured CD44v6kd-Non-CIC revealed pronounced mRNA changes in signaling, transport, transcription and translation; altered miRNA affected metabolism, signaling and transcription. CIC-TEX coculture-induced changes in Tspan8kd-Non-CIC mostly relied on CIC-TEX-Tspan8 being required for targeting. CIC-TEX transfer supported apoptosis resistance and significantly promoted epithelial mesenchymal transition, migration, invasion and (lymph)angiogenesis of the kd Non-CIC in vitro and in vivo, deep-sequencing allowing individual mRNA and miRNA assignment to altered functions. Importantly, CIC-TEX act as a hub, initiated by CD44v6-dependent RTK, GPCR and integrin activation and involving CD44v6-assisted transcription and RNA processing. Accordingly, a kinase inhibitor hampered CIC-TEX-fostered tumor progression, which was backed by an anti-Tspan8 blockade of CIC-TEX binding. Conclusions: This in depth report on the in vitro and in vivo impact of CIC-TEX on CD44v6kd and Tspan8kd Non-CIC unravels hub CIC-TEX activity, highlighting a prominent contribution of the CIC-markers CD44v6 to signaling cascade activation, transcription, translation and miRNA processing in Non-CIC and of Tspan8 to CIC-TEX targeting. Blocking CIC-TEX binding/uptake and uptake-initiated target cell activation significantly mitigated the deleterious CIC-TEX impact on CD44v6kd and Tspan8kd Non-CIC

The mediating role of negative symptoms in “secondary factors” determining social functioning in chronic schizophrenia

BackgroundChronic schizophrenia is significantly influenced by negative symptoms, with several known contributors to secondary negative symptoms. However, the impact of these factors and negative symptoms on social functioning warrants further exploration.MethodsWe assessed the clinical symptoms, antipsychotic adverse reactions, and social functioning of 283 hospitalized patients with chronic schizophrenia using various standardized interviews and scales. We conducted multiple regression and mediation analyses to elucidate the impact of secondary factors on negative symptoms, and the relationship among these “secondary factors,” negative symptoms, and social functioning.ResultsOur findings identified depressive symptoms, extrapyramidal symptoms, and positive symptoms as significant contributors to secondary negative symptoms. We found that negative symptoms play a notable mediating role in the effect of depressive and positive symptoms on social functioning. However, the relationship between positive symptoms, negative symptoms, and social functioning proved to be intricate.ConclusionOur findings propose that negative symptoms act as pivotal mediators in the correlation between “secondary factors” (including the depressive symptoms and positive symptoms) and social functioning. The treatment of chronic schizophrenia necessitates focusing on key factors such as depressive and positive symptoms, which might significantly contribute to the development of secondary negative symptoms. Further research is essential to clarify the complex relationship among positive symptoms, negative symptoms, and social functioning in schizophrenia

Improved ground-state modulation characteristics in 1.3 μm InAs/GaAs quantum dot lasers by rapid thermal annealing

We investigated the ground-state (GS) modulation characteristics of 1.3 μm InAs/GaAs quantum dot (QD) lasers that consist of either as-grown or annealed QDs. The choice of annealing conditions was determined from our recently reported results. With reference to the as-grown QD lasers, one obtains approximately 18% improvement in the modulation bandwidth from the annealed QD lasers. In addition, the modulation efficiency of the annealed QD lasers improves by approximately 45% as compared to the as-grown ones. The observed improvements are due to (1) the removal of defects which act as nonradiative recombination centers in the QD structure and (2) the reduction in the Auger-related recombination processes upon annealing

Nanocomposite of Graphene Oxide Encapsulated in Polymethylmethacrylate (PMMA): Pre-Modification, Synthesis, and Latex Stability

The compatibility of graphene oxide with its dispersion medium (polymer) plays a critical role in the formation of nanocomposite materials with significant property improvements. Environmentally friendly miniemulsion polymerization, which allows a formation of nanoencapsulation in an aqueous phase and high molecular weight polymer/composite production is one promising method. In this study, we screened a series of amphiphilic modifiers and found that the quaternary ammonium (ar-vinyl benzyl) trimethyl ammonium chloride (VBTAC) pending carbon double bonds could effectively modify the graphene oxide (GO) to be compatible with the organophilic monomer. After that, free radical miniemulsion polymerization successfully synthesized stable latex of exfoliated poly (methyl methacrylate) (PMMA)/ GO nanocomposite. The final latex had an extended storage life and a relatively uniform particle size distribution. X-ray powder diffraction (XRD), transmission electron microscopy (TEM), and scanning electron microscopy (SEM) analysis of this latex and its films indicated successful encapsulation of exfoliated nano-dimensional graphene oxide inside a polymer matrix

The effects of thermal annealing on the photoluminescence and DC characteristics of 1.3 µm p-doped InAs/InGaAs/GaAs quantum dot lasers

In this work, we investigated the optical characteristics of 1.3 µm p -doped InAs/InGaAs/GaAs quantum dot (QD) lasers that consist of either as-grown or annealed QDs. With reference to the as-grown QD lasers, it is found that 1) the integrated intensity of the photoluminescence could be increased by 63%, and 2) the onset of excited state lasing can be significantly delayed to a higher injection current with optimum annealing conditions. In addition, the internal quantum efficiency and internal optical loss of the annealed QD lasers has been improved. The observed improvements could be attributed to the removal of defects which act as nonradiative recombination centers in the QD structure and the reduction in the Auger-related recombination processes upon annealing

Direct Observation of Filling Process and Porosity Prediction in High Pressure Die Casting

Although numerical simulation accuracy makes progress rapidly, it is in an insufficient phase because of complicated phenomena of the filling process and difficulty of experimental verification in high pressure die casting (HPDC), especially in thin-wall complex die-castings. Therefore, in this paper, a flow visualization experiment is conducted, and the porosity at different locations is predicted under three different fast shot velocities. The differences in flow pattern between the actual filling process and the numerical simulation are compared. It shows that the flow visualization experiment can directly observe the actual and real-time filling process and could be an effective experimental verification method for the accuracy of the flow simulation model in HPDC. Moreover, significant differences start to appear in the flow pattern between the actual experiment and the Anycasting solution after the fragment or atomization formation. Finally, the fast shot velocity would determine the position at which the back flow meets the incoming flow. The junction of two streams of fluid would create more porosity than the other location. There is a transition in flow patterns due to drag crisis under high fast shot velocity around two staggered cylinders, which resulted in the porosity relationship also changing from R1 < R3 < R2 (0.88 m/s) to R1 < R2 < R3 (1.59 and 2.34 m/s)

Topographic patterns of forest decline as detected from tree rings and NDVI

Forest decline is mostly attributed to factors related to climate change and human activities. However, it is not well-known how topographic factors indirectly shape the forest decline. In this study, we analyzed the topographic patterns of growth decline in Pinus densiflora forests in the Mengshan Mountains of eastern China. A forest decline event occurring in 2009-2014 was identified using the Normalized Difference Vegetation Index (NDVI) and tree-ring widths. A decision tree model was developed to extract the topographic pattern of the forest decline. Boosted regression tree model indicates that the relative influence of elevation is 93.6%, and the remaining relative influence includes aspect, slope, and landform. We conclude that topography, elevation in particular, indirectly shapes the vulnerability of sites to forest decline and can thus be used to predict the spatial decline risks. Our findings provide new insights into the role of topography in forest decline and hold implications for predicting decline risk and protecting forests in a rapidly changing world