On the Depth of Deep Neural Networks: A Theoretical View

People believe that depth plays an important role in success of deep neural networks (DNN). However, this belief lacks solid theoretical justifications as far as we know. We investigate role of depth from perspective of margin bound. In margin bound, expected error is upper bounded by empirical margin error plus Rademacher Average (RA) based capacity term. First, we derive an upper bound for RA of DNN, and show that it increases with increasing depth. This indicates negative impact of depth on test performance. Second, we show that deeper networks tend to have larger representation power (measured by Betti numbers based complexity) than shallower networks in multi-class setting, and thus can lead to smaller empirical margin error. This implies positive impact of depth. The combination of these two results shows that for DNN with restricted number of hidden units, increasing depth is not always good since there is a tradeoff between positive and negative impacts. These results inspire us to seek alternative ways to achieve positive impact of depth, e.g., imposing margin-based penalty terms to cross entropy loss so as to reduce empirical margin error without increasing depth. Our experiments show that in this way, we achieve significantly better test performance.Comment: AAAI 201

LayoutDiffusion: Improving Graphic Layout Generation by Discrete Diffusion Probabilistic Models

Creating graphic layouts is a fundamental step in graphic designs. In this work, we present a novel generative model named LayoutDiffusion for automatic layout generation. As layout is typically represented as a sequence of discrete tokens, LayoutDiffusion models layout generation as a discrete denoising diffusion process. It learns to reverse a mild forward process, in which layouts become increasingly chaotic with the growth of forward steps and layouts in the neighboring steps do not differ too much. Designing such a mild forward process is however very challenging as layout has both categorical attributes and ordinal attributes. To tackle the challenge, we summarize three critical factors for achieving a mild forward process for the layout, i.e., legality, coordinate proximity and type disruption. Based on the factors, we propose a block-wise transition matrix coupled with a piece-wise linear noise schedule. Experiments on RICO and PubLayNet datasets show that LayoutDiffusion outperforms state-of-the-art approaches significantly. Moreover, it enables two conditional layout generation tasks in a plug-and-play manner without re-training and achieves better performance than existing methods.Comment: Accepted by ICCV2023, project page: https://layoutdiffusion.github.i

Electro-Chemo-Mechanical Failure of Solid Electrolytes Induced by Growth of Internal Lithium Filaments

Growth of lithium (Li) filaments within solid electrolytes, leading to mechanical degradation of the electrolyte and even short circuit of the cell under high current density, is a great barrier to commercialization of solid-state Li-metal batteries. Understanding of this electro-chemo-mechanical phenomenon is hindered by the challenge of tracking local fields inside the solid electrolyte. Here, a multiphysics simulation aiming to investigate evolution of the mechanical failure of the solid electrolyte induced by the internal growth of Li is reported. Visualization of local stress, damage, and crack propagation within the solid electrolyte enables examination of factors dominating the degradation process, including the geometry, number, and size of Li filaments and voids in the electrolyte. Relative damage induced by locally high stress is found to preferentially occur in the region of the electrolyte/Li interface having great fluctuations. A high number density of Li filaments or voids triggers integration of damage and crack networks by enhanced propagation. This model is built on coupling of mechanical and electrochemical processes for internal plating of Li, revealing evolution of multiphysical fields that can barely be captured by the state-of-the-art experimental techniques. Understanding mechanical degradation of solid electrolytes with the presence of Li filaments paves the way to design advanced solid electrolytes for future solid-state Li-metal batteries

LayoutPrompter: Awaken the Design Ability of Large Language Models

Conditional graphic layout generation, which automatically maps user constraints to high-quality layouts, has attracted widespread attention today. Although recent works have achieved promising performance, the lack of versatility and data efficiency hinders their practical applications. In this work, we propose LayoutPrompter, which leverages large language models (LLMs) to address the above problems through in-context learning. LayoutPrompter is made up of three key components, namely input-output serialization, dynamic exemplar selection and layout ranking. Specifically, the input-output serialization component meticulously designs the input and output formats for each layout generation task. Dynamic exemplar selection is responsible for selecting the most helpful prompting exemplars for a given input. And a layout ranker is used to pick the highest quality layout from multiple outputs of LLMs. We conduct experiments on all existing layout generation tasks using four public datasets. Despite the simplicity of our approach, experimental results show that LayoutPrompter can compete with or even outperform state-of-the-art approaches on these tasks without any model training or fine-tuning. This demonstrates the effectiveness of this versatile and training-free approach. In addition, the ablation studies show that LayoutPrompter is significantly superior to the training-based baseline in a low-data regime, further indicating the data efficiency of LayoutPrompter. Our project is available at https://github.com/microsoft/LayoutGeneration/tree/main/LayoutPrompter.Comment: NeurIPS 202

A Parse-Then-Place Approach for Generating Graphic Layouts from Textual Descriptions

Creating layouts is a fundamental step in graphic design. In this work, we propose to use text as the guidance to create graphic layouts, i.e., Text-to-Layout, aiming to lower the design barriers. Text-to-Layout is a challenging task, because it needs to consider the implicit, combined, and incomplete layout constraints from text, each of which has not been studied in previous work. To address this, we present a two-stage approach, named parse-then-place. The approach introduces an intermediate representation (IR) between text and layout to represent diverse layout constraints. With IR, Text-to-Layout is decomposed into a parse stage and a place stage. The parse stage takes a textual description as input and generates an IR, in which the implicit constraints from the text are transformed into explicit ones. The place stage generates layouts based on the IR. To model combined and incomplete constraints, we use a Transformer-based layout generation model and carefully design a way to represent constraints and layouts as sequences. Besides, we adopt the pretrain-then-finetune strategy to boost the performance of the layout generation model with large-scale unlabeled layouts. To evaluate our approach, we construct two Text-to-Layout datasets and conduct experiments on them. Quantitative results, qualitative analysis, and user studies demonstrate the effectiveness of our approach.Comment: Accepted by ICCV202

Enhanced phosphogypsum thermal reduction by carbon in presence of sodium chloride at high temperature

This paper reports an enhanced approach of thermal reduction of phosphogypsum (PG) in the presence of sodium chloride (NaCl) in its molten phase. The thermodynamic together with in-situ thermogravimetric-Fourier transform infrared spectroscopy (TG-FTIR) result indicates the yield of CaS (solid) at the investigated temperature range (800–850 °C). Addition of NaCl not only appreciably reduces the activation energy of this reaction (800–850 °C) on average from 315 to 175 (kJ·mol−1) materially, but also significantly improve the conversion from CaSO4 to CaS. Additionally, the NaCl based waste salt (NaCl-WS) containing organic compounds (0.21 wt%) was found to produce similar results when compared with the pure NaCl addition at investigated temperature range. The optimal conditions via parametric optimization: i.e., 800–850 °C, 15 wt% (NaCl-WS), and 6 (ratio of PG/C (carbon) wt/wt), were determined. The mechanistic study together with using density functional theories (DFT) shows that the synergistic effect by NaCl addition was caused by the coupled-vacancy diffusion mechanism when NaCl melts at high temperature and C atom shows proximity to oxygen (O) atom from the SO42- cluster in the CaSO4 crystallite structure. The evaluation of the process shows appealing advantages over the conventional PG thermal reduction. By carefully manipulating temperature range together with the NaCl-WS addition, the proposed new process simultaneously realizes PG thermal reduction and environmental utilization of NaCl-WS

Role of Li-Ion Depletion on Electrode Surface: Underlying Mechanism for Electrodeposition Behavior of Lithium Metal Anode

The application of lithium metal as an anode material for next generation high energy-density batteries has to overcome the major bottleneck that is the seemingly unavoidable growth of Li dendrites caused by non-uniform electrodeposition on the electrode surface. This problem must be addressed by clarifying the detailed mechanism. In this work the mass-transfer of Li-ions is investigated, a key process controlling the electrochemical reaction. By a phase field modeling approach, the Li-ion concentration and the electric fields are visualized to reveal the role of three key experimental parameters, operating temperature, Li-salt concentration in electrolyte, and applied current density, on the microstructure of deposited Li. It is shown that a rapid depletion of Li-ions on electrode surface, induced by, e.g., low operating temperature, diluted electrolyte and a high applied current density, is the underlying driving force for non-uniform electrodeposition of Li. Thus, a viable route to realize a dendrite-free Li plating process would be to mitigate the depletion of Li-ions on the electrode surface. The methodology and results in this work may boost the practical applicability of Li anodes in Li metal batteries and other battery systems using metal anodes