New Interpretations of Normalization Methods in Deep Learning

In recent years, a variety of normalization methods have been proposed to help train neural networks, such as batch normalization (BN), layer normalization (LN), weight normalization (WN), group normalization (GN), etc. However, mathematical tools to analyze all these normalization methods are lacking. In this paper, we first propose a lemma to define some necessary tools. Then, we use these tools to make a deep analysis on popular normalization methods and obtain the following conclusions: 1) Most of the normalization methods can be interpreted in a unified framework, namely normalizing pre-activations or weights onto a sphere; 2) Since most of the existing normalization methods are scaling invariant, we can conduct optimization on a sphere with scaling symmetry removed, which can help stabilize the training of network; 3) We prove that training with these normalization methods can make the norm of weights increase, which could cause adversarial vulnerability as it amplifies the attack. Finally, a series of experiments are conducted to verify these claims.Comment: Accepted by AAAI 202

Compact on-chip power splitter based on topological photonic crystal

We propose and demonstrate an on-chip 1*N power splitter based on topological photonic crystal (TPC) on a monolithic silicon photonic platform. Benefiting from the valley-locked propagation mode at the interface of TPCs with different topological phases, the proposed power splitter has negligible backscattering around the sharp bendings and good robustness to fabrication defects, which therefore enable lower insertion loss, better uniformity, and more compact footprint than the conventional designs. For the fabricated 1*2 (8) power splitter, the uniformity among the output ports is below 0.35 (0.65) dB and the maximum insertion loss is 0.38 (0.58) dB with compact footprint of 5*5 um2 (10*12 um2) within a bandwidth of 70 nm. In addition, the topological power splitter only requires simple configurations of TPCs with different topological phases, which is more reliable in design and fabrication compared with the conventional designs.Comment: 8 pages,4 figure

OmniLytics: A Blockchain-based Secure Data Market for Decentralized Machine Learning

We propose OmniLytics, a blockchain-based secure data trading marketplace for machine learning applications. Utilizing OmniLytics, many distributed data owners can contribute their private data to collectively train an ML model requested by some model owners, and receive compensation for data contribution. OmniLytics enables such model training while simultaneously providing 1) model security against curious data owners; 2) data security against the curious model and data owners; 3) resilience to malicious data owners who provide faulty results to poison model training; and 4) resilience to malicious model owners who intend to evade payment. OmniLytics is implemented as a blockchain smart contract to guarantee the atomicity of payment. In OmniLytics, a model owner splits its model into the private and public parts and publishes the public part on the contract. Through the execution of the contract, the participating data owners securely aggregate their locally trained models to update the model owner\u27s public model and receive reimbursement through the contract. We implement a working prototype of OmniLytics on Ethereum blockchain and perform extensive experiments to measure its gas cost, execution time, and model quality under various parameter combinations. For training a CNN on the MNIST dataset, the MO is able to boost its model accuracy from 62% to 83% within 500ms in blockchain processing time.This demonstrates the effectiveness of OmniLytics for practical deployment

Freeze-thaw cycles drove chemical weathering and enriched sulfates in the Burns formation at Meridiani, Mars

Sulfate-rich sedimentary rocks explored by the Opportunity rover during its 14-year surface mission at Meridiani Planum provide an invaluable window into the thousands of sulfate deposits detected on Mars via remote sensing. Existing models explaining the formation of martian sulfates can be generally described as either bottom-up, groundwater-driven playa settings or top-down icy chemical weathering environments. Here, we propose a hybrid model involving both bottom-up and top-down processes driven by freeze-thaw cycles. Freezing leads to cryo-concentration of acidic fluids from precipitations at the surface, facilitating rapid chemical weathering despite low temperatures. Cryosuction causes the upwards migration of vadose water and even groundwater with dissolved ions, resulting in the accumulation of ions in near-surface environments. Evaporation precipitates salts but leaching separates chlorides from sulfates during the thawing period. Freeze-thaw cycles, therefore, can enrich sulfates at the surface. While freeze-thaw is more commonly understood as a mechanism of physical weathering, we suggest it is a fundamental aspect of chemical weathering on Mars

Rapid and non-destructive determination of tea polyphenols content in Chongzhou new loquat tea lines based on near infrared spectroscopy

Abstract Near infrared spectroscopy (NIRS) combined with multiple algorithms was used to determinate the tea polyphenols content in Chongzhou new loquat tea lines quickly and nondestructively. Samples of 26 Chongzhou new loquat tea lines were collected, then scanning NIRS, pretreating spectral noise information, screening characteristic spectral intervals by backward interval partial least squares, proceeding principal component analysis. Finally, the artificial neural network (BP-ANN) method with three kinds of transfer functions was applied to establish models. The best pretreated method was the combination of standard normal variation (SNV) and first derivative, and the characteristic spectral regions selected were 4381.5-4755.6 cm–1, 4759.5-5133.6 cm–1, 6266.6-6637.8 cm–1 and 7389.9-7760.2 cm–1, respectively. The cumulative contribution rate of the first three principal components of the selected characteristic spectra was 95.24%. When the BP-ANN calibration set model was established with the logistic function, NIRS model had the best results, whose root mean square error and determination coefficient of the cross validation were 0.975 and 0.372%, respectively. The root mean square error and the determination coefficient of the prediction set model were 0.962 and 0.400%, respectively. The results showed NIRS can predict the tea polyphenols content in Chongzhou new loquat tea lines quickly and accurately

Continuous Mott transition in semiconductor moir\'e superlattices

The evolution of a Landau Fermi liquid into a nonmagnetic Mott insulator with increasing electronic interactions is one of the most puzzling quantum phase transitions in physics. The vicinity of the transition is believed to host exotic states of matter such as quantum spin liquids, exciton condensates and unconventional superconductivity. Semiconductor moir\'e materials realize a highly controllable Hubbard model simulator on a triangular lattice, providing a unique opportunity to drive a metal-insulator transition (MIT) via continuous tuning of the electronic interactions. Here, by electrically tuning the effective interaction strength in MoTe2/WSe2 moir\'e superlattices, we observe a continuous MIT at a fixed filling of one electron per unit cell. The existence of quantum criticality is supported by the scaling behavior of the resistance, a continuously vanishing charge-gap as the critical point is approached from the insulating side, and a diverging quasiparticle effective mass from the metallic side. We also observe a smooth evolution of the low-temperature magnetic susceptibility across the MIT and find no evidence of long-range magnetic order down to ~ 5% of the Curie-Weiss temperature. The results signal an abundance of low-energy spinful excitations on the insulating side that is further corroborated by the presence of the Pomeranchuk effect on the metallic side. Our results are consistent with the universal critical theory of a continuous MIT from a Landau Fermi liquid to a nonmagnetic Mott insulator in two dimensions

Gigahertz-rate-switchable wavefront shaping through integration of metasurfaces with photonic integrated circuit

Achieving spatiotemporal control of light at high-speeds presents immense possibilities for various applications in communication, computation, metrology, and sensing. The integration of subwavelength metasurfaces and optical waveguides offers a promising approach to manipulate light across multiple degrees of freedom at high-speed in compact photonic integrated circuit (PICs) devices. Here, we demonstrate a gigahertz-rate-switchable wavefront shaping by integrating metasurface, lithium niobite on insulator (LNOI) photonic waveguide and electrodes within a PIC device. As proofs of concept, we showcase the generation of a focus beam with reconfigurable arbitrary polarizations, switchable focusing with lateral focal positions and focal length, orbital angular momentum light beams (OAMs) as well as Bessel beams. Our measurements indicate modulation speeds of up to gigahertz rate. This integrated platform offers a versatile and efficient means of controlling light field at high-speed within a compact system, paving the way for potential applications in optical communication, computation, sensing, and imaging