Tensor-Compressed Back-Propagation-Free Training for (Physics-Informed) Neural Networks

Backward propagation (BP) is widely used to compute the gradients in neural network training. However, it is hard to implement BP on edge devices due to the lack of hardware and software resources to support automatic differentiation. This has tremendously increased the design complexity and time-to-market of on-device training accelerators. This paper presents a completely BP-free framework that only requires forward propagation to train realistic neural networks. Our technical contributions are three-fold. Firstly, we present a tensor-compressed variance reduction approach to greatly improve the scalability of zeroth-order (ZO) optimization, making it feasible to handle a network size that is beyond the capability of previous ZO approaches. Secondly, we present a hybrid gradient evaluation approach to improve the efficiency of ZO training. Finally, we extend our BP-free training framework to physics-informed neural networks (PINNs) by proposing a sparse-grid approach to estimate the derivatives in the loss function without using BP. Our BP-free training only loses little accuracy on the MNIST dataset compared with standard first-order training. We also demonstrate successful results in training a PINN for solving a 20-dim Hamiltonian-Jacobi-Bellman PDE. This memory-efficient and BP-free approach may serve as a foundation for the near-future on-device training on many resource-constraint platforms (e.g., FPGA, ASIC, micro-controllers, and photonic chips)

ESIA: An Efficient and Stable Identity Authentication for Internet of Vehicles

Decentralized, tamper-proof blockchain is regarded as a solution to a challenging authentication issue in the Internet of Vehicles (IoVs). However, the consensus time and communication overhead of blockchain increase significantly as the number of vehicles connected to the blockchain. To address this issue, vehicular fog computing has been introduced to improve efficiency. However, existing studies ignore several key factors such as the number of vehicles in the fog computing system, which can impact the consensus communication overhead. Meanwhile, there is no comprehensive study on the stability of vehicular fog composition. The vehicle movement will lead to dynamic changes in fog. If the composition of vehicular fog is unstable, the blockchain formed by this fog computing system will be unstable, which can affect the consensus efficiency. With the above considerations, we propose an efficient and stable identity authentication (ESIA) empowered by hierarchical blockchain and fog computing. By grouping vehicles efficiently, ESIA has low communication complexity and achieves high stability. Moreover, to enhance the consensus security of the hierarchical blockchain, the consensus process is from the bottom layer to the up layer (bottom-up), which we call B2UHChain. Through theoretical analysis and simulation verification, our scheme achieves the design goals of high efficiency and stability while significantly improving the IoV scalability to the power of 1.5 (^1.5) under similar security to a single-layer blockchain. In addition, ESIA has less communication and computation overhead, lower latency, and higher throughput than other baseline authentication schemes

Prime-Boost Vaccine Regimen for SjTPl and SjC23 Schistosome Vaccines, Increases Efficacy in Water Buffalo in a Field Trial in China

Schistosomiasis remains a serious zoonotic disease in China and the Philippines. Water buffalo and cattle account for the majority of transmission. Vaccination of water buffalo is considered a key strategy to reduce disease prevalence. Previously, we showed that vaccination of water buffalo with SjC23 or SjCTPI plasmid DNA vaccines, induced 50% efficacy to challenge infection. Here, we evaluated several parameters to determine if we can develop a two dose vaccine that maintains the efficacy of the three dose vaccine. We performed four trials evaluating: (1) lab produced vs. GLP grade vaccines, (2) varying the time between prime and boost, (3) the influence of an IL-12 adjuvant, and (4) a two dose heterologous (DNA-protein) prime-boost. We found the source of the DNA vaccines did not matter, nor did increasing the interval between prime and boost. Elimination of the IL-12 plasmid lowered homologous DNA-DNA vaccine efficacy. A major finding was that the heterologous prime boost improved vaccine efficacy, with the prime-boost regimen incorporating both antigens providing a 55% reduction in adult worms and 53% reduction in liver eggs. Vaccinated buffalo produced vaccine-specific antibody responses. These trials suggest that highly effective vaccination against schistosomes can be achieved using a two dose regimen. No adjuvants were used with the protein boost, and the potential that addition of adjuvant to the protein boost to further increase efficacy should be evaluated. These results suggest that use of these two schistosome vaccines can be part of an integrated control strategy to reduce transmission of schistosomiasis in Asia

Human susceptibility to Schistosoma japonicum in China correlates with antibody isotypes to native antigens

Antibody isotypic responses (IgE, IgA, IgG1, IgG2, IgG3 and IgG4) to Schistosoma japonicum antigens—adult worm (AWA), soluble egg (SEA) and the recombinant proteins TEG (22·6-kDa tegumental antigen, Sj22) and PMY (paramyosin, Sj97)—were measured (in 1998) in a cohort of 179 Chinese subjects 2 years post-treatment. Subjects in the highest intensity re-infection group (>100 eggs per gram faeces) had significantly higher levels of IgG1 and IgG4 against AWA. Analysis of IgG4/IgE ratios for AWA and SEA linked IgG4 excess to re-infection and IgE excess to non-re-infection. Two years after chemotherapeutic cure, 29 subjects, who were re-infected or never infected but highly water-exposed, were classified as epidemiologically susceptible (n = 15) or epidemiologically insusceptible to infection (n = 14). IgG4 levels against native antigens (AWA and SEA) were higher in susceptibles and IgE levels were higher in insusceptibles but antibody responses to the recombinant proteins (PMY and TEG) showed no clear pattern or difference between susceptibility groups. These and earlier findings provide evidence that immunity develops against schistosomiasis japonica in China and that susceptibility/resistance correlates with antibody isotypes against native schistosome antigen

Local immune responses of the Chinese water buffalo, Bubalus bubalis, against Schistosoma japonicum larvae: crucial insights for vaccine design

Asian schistosomiasis is a zoonotic parasitic disease infecting up to a million people and threatening tens of millions more. Control of this disease is hindered by the animal reservoirs of the parasite, in particular the water buffalo (Bubalus bubalis), which is responsible for significant levels of human transmission. A transmission-blocking vaccine administered to buffaloes is a realistic option which would aid in the control of schistosomiasis. This will however require a better understanding of the immunobiology of schistosomiasis in naturally exposed buffaloes, particularly the immune response to migrating schistosome larvae, which are the likely targets of an anti-schistosome vaccine. To address this need we investigated the immune response at the major sites of larval migration, the skin and the lungs, in previously exposed and re-challenged water buffaloes. In the skin, a strong allergic-type inflammatory response occurred, characterised by leukocyte and eosinophil infiltration including the formation of granulocytic abscesses. Additionally at the local skin site, interleukin-5 transcript levels were elevated, while interleukin-10 levels decreased. In the skin-draining lymph node (LN) a predominant type-2 profile was seen in stimulated cells, while in contrast a type-1 profile was detected in the lung draining LN, and these responses occurred consecutively, reflecting the timing of parasite migration. The intense type-2 immune response at the site of cercarial penetration is significantly different to that seen in naive and permissive animal models such as mice, and suggests a possible mechanism for immunity. Preliminary data also suggest a reduced and delayed immune response occurred in buffaloes given high cercarial challenge doses compared with moderate infections, particularly in the skin. This study offers a deeper understanding into the immunobiology of schistosomiasis in a natural host, which may aid in the future design of more effective vaccines

Prime-Boost Vaccine Regimen for SjTPI and SjC23 Schistosome Vaccines, Increases Efficacy in Water Buffalo in a Field Trial in China

Schistosomiasis remains a serious zoonotic disease in China and the Philippines. Water buffalo and cattle account for the majority of transmission. Vaccination of water buffalo is considered a key strategy to reduce disease prevalence. Previously, we showed that vaccination of water buffalo with SjC23 or SjCTPI plasmid DNA vaccines, induced 50% efficacy to challenge infection. Here, we evaluated several parameters to determine if we can develop a two dose vaccine that maintains the efficacy of the three dose vaccine. We performed four trials evaluating: (1) lab produced vs. GLP grade vaccines, (2) varying the time between prime and boost, (3) the influence of an IL-12 adjuvant, and (4) a two dose heterologous (DNA-protein) prime-boost. We found the source of the DNA vaccines did not matter, nor did increasing the interval between prime and boost. Elimination of the IL-12 plasmid lowered homologous DNA-DNA vaccine efficacy. A major finding was that the heterologous prime boost improved vaccine efficacy, with the prime-boost regimen incorporating both antigens providing a 55% reduction in adult worms and 53% reduction in liver eggs. Vaccinated buffalo produced vaccine-specific antibody responses. These trials suggest that highly effective vaccination against schistosomes can be achieved using a two dose regimen. No adjuvants were used with the protein boost, and the potential that addition of adjuvant to the protein boost to further increase efficacy should be evaluated. These results suggest that use of these two schistosome vaccines can be part of an integrated control strategy to reduce transmission of schistosomiasis in Asia

TT-PINN: A Tensor-Compressed Neural PDE Solver for Edge Computing

Physics-informed neural networks (PINNs) have been increasingly employed due to their capability of modeling complex physics systems. To achieve better expressiveness, increasingly larger network sizes are required in many problems. This has caused challenges when we need to train PINNs on edge devices with limited memory, computing and energy resources. To enable training PINNs on edge devices, this paper proposes an end-to-end compressed PINN based on Tensor-Train decomposition. In solving a Helmholtz equation, our proposed model significantly outperforms the original PINNs with few parameters and achieves satisfactory prediction with up to 15$\times$ overall parameter reduction