Convergence Theory of Learning Over-parameterized ResNet: A Full Characterization

ResNet structure has achieved great empirical success since its debut. Recent work established the convergence of learning over-parameterized ResNet with a scaling factor $\tau=1/L$ on the residual branch where $L$ is the network depth. However, it is not clear how learning ResNet behaves for other values of $\tau$. In this paper, we fully characterize the convergence theory of gradient descent for learning over-parameterized ResNet with different values of $\tau$. Specifically, with hiding logarithmic factor and constant coefficients, we show that for $\tau\le 1/\sqrt{L}$ gradient descent is guaranteed to converge to the global minma, and especially when $\tau\le 1/L$ the convergence is irrelevant of the network depth. Conversely, we show that for $\tau>L^{-\frac{1}{2}+c}$, the forward output grows at least with rate $L^c$ in expectation and then the learning fails because of gradient explosion for large $L$. This means the bound $\tau\le 1/\sqrt{L}$ is sharp for learning ResNet with arbitrary depth. To the best of our knowledge, this is the first work that studies learning ResNet with full range of $\tau$.Comment: 31 page

Real transmission and reflection zeros of periodic structures with a bound state in the continuum

For lossless periodic structures with a proper symmetry, the transmission and reflection spectra often have peaks and dips that are truly $100\%$ and $0\%$, respectively. The full peaks and zero dips typically appear near resonant frequencies, and they are robust with respect to structural perturbations that preserve the required symmetry. However, current theories on the existence of full peaks and zero dips are incomplete and difficult to use. For periodic structures with a bound state in the continuum (BIC), we present a new theory on the existence of real transmission and reflection zeros that correspond to the zero dips in the transmission and reflection spectra. Our theory is relatively simple, complete, and easy to use. Numerical examples are presented to validate the new theory.Comment: 8 pages, 5 figure

Multi-level Gated Bayesian Recurrent Neural Network for State Estimation

The optimality of Bayesian filtering relies on the completeness of prior models, while deep learning holds a distinct advantage in learning models from offline data. Nevertheless, the current fusion of these two methodologies remains largely ad hoc, lacking a theoretical foundation. This paper presents a novel solution, namely a multi-level gated Bayesian recurrent neural network specifically designed to state estimation under model mismatches. Firstly, we transform the non-Markov state-space model into an equivalent first-order Markov model with memory. It is a generalized transformation that overcomes the limitations of the first-order Markov property and enables recursive filtering. Secondly, by deriving a data-assisted joint state-memory-mismatch Bayesian filtering, we design a Bayesian multi-level gated framework that includes a memory update gate for capturing the temporal regularities in state evolution, a state prediction gate with the evolution mismatch compensation, and a state update gate with the observation mismatch compensation. The Gaussian approximation implementation of the filtering process within the gated framework is derived, taking into account the computational efficiency. Finally, the corresponding internal neural network structures and end-to-end training methods are designed. The Bayesian filtering theory enhances the interpretability of the proposed gated network, enabling the effective integration of offline data and prior models within functionally explicit gated units. In comprehensive experiments, including simulations and real-world datasets, the proposed gated network demonstrates superior estimation performance compared to benchmark filters and state-of-the-art deep learning filtering methods

Interaction-Driven Active 3D Reconstruction with Object Interiors

We introduce an active 3D reconstruction method which integrates visual perception, robot-object interaction, and 3D scanning to recover both the exterior and interior, i.e., unexposed, geometries of a target 3D object. Unlike other works in active vision which focus on optimizing camera viewpoints to better investigate the environment, the primary feature of our reconstruction is an analysis of the interactability of various parts of the target object and the ensuing part manipulation by a robot to enable scanning of occluded regions. As a result, an understanding of part articulations of the target object is obtained on top of complete geometry acquisition. Our method operates fully automatically by a Fetch robot with built-in RGBD sensors. It iterates between interaction analysis and interaction-driven reconstruction, scanning and reconstructing detected moveable parts one at a time, where both the articulated part detection and mesh reconstruction are carried out by neural networks. In the final step, all the remaining, non-articulated parts, including all the interior structures that had been exposed by prior part manipulations and subsequently scanned, are reconstructed to complete the acquisition. We demonstrate the performance of our method via qualitative and quantitative evaluation, ablation studies, comparisons to alternatives, as well as experiments in a real environment.Comment: Accepted to SIGGRAPH Asia 2023, project page at https://vcc.tech/research/2023/InterReco

Multi-Objective Considered Process Parameter Optimization of Welding Robots Based on Small Sample Size Dataset

The welding process is characterized by its high energy density, making it imperative to optimize the energy consumption of welding robots without compromising the quality and efficiency of the welding process for their sustainable development. The above evaluation objectives in a particular welding situation are mostly influenced by the welding process parameters. Although numerical analysis and simulation methods have demonstrated their viability in optimizing process parameters, there are still limitations in terms of modeling accuracy and efficiency. This paper presented a framework for optimizing process parameters of welding robots in industry settings, where data augmentation was applied to expand sample size, auto machine learning theory was incorporated to quantify reflections from process parameters to evaluation objectives, and the enhanced non-dominated sorting algorithm was employed to identify an optimal solution by balancing these objectives. Additionally, an experiment using Q235 as welding plates was designed and conducted on a welding platform, and the findings indicated that the prediction accuracy on different objectives obtained by the enlarged dataset through ensembled models all exceeded 95%. It is proven that the proposed methods enabled the efficient and optimal determination of parameter instructions for welding scenarios and exhibited superior performance compared with other optimization methods in terms of model correctness, modeling efficiency, and method applicability

Enhanced photo-piezo-catalytic properties of Co-doped Ba_0.85Ca_0.15Zr_0.1(Ti_1-xCo_x)_0.9 ferroelectric ceramics for dye degradation

This paper provides a detailed evaluation of the photo-piezo-catalytic properties of lead-free Ba0.85Ca0.15Zr0.1(Ti1-xCox)0.9(BCZT-xCo,x = 0–0.025) ferroelectric ceramics prepared by a solid-state process. By control of the Co doping level, the band gap was reduced to 2.40 eV at the composition x = 0.02, which improved the generation of photo-generated charges and enhanced the photocatalytic activity. When a solution containing BCZT-0.02Co particles was subjected to both ultrasound and illumination, the degree of degradation of Rhodamine B reached 99% within 60 min, which was grater than when subjected to illumination or ultrasound alone. Examination of the dielectric properties, photoelectrochemical measurements and band energy structure of the materials provided new insights into the catalytic mechanism, where a strong coupling between piezoelectricity and photoexcitation was clearly observed. This work therefore highlights the attractive photo-piezo-catalytic properties of BCZT-xCo doped ceramics and is the first demonstration that Co substitution in these lead-free ferroelectric ceramics provides significant potential for photo-piezo-catalysis applications.</p

Numerička studija izrađena pomoću ChemKin za rasplinjavanje vodene pare ugljene prašine i transformacije žive unutar rasplinjača s vodenom parom

Zero-emission coal (ZEC) technology has been actively studied recently. It aims to achieve zero emission of CO2 and other pollutants and the efficiency of this system can reach no less than 70%. Hydro-gasification of pulverized coal is a core process of ZEC. However, the mechanism of gasification and transformation of mercury speciation in the hydro-gasification is has not been understood precisely up until now. This restrains the ZEC’s commercialization. The purpose of this paper is to study the mechanism of hydro-gasification and mercury speciation transformation for coal in the gasifier with high temperature and pressure. Detailed chemical kinetics mechanism (CKM) has been proposed for hydro-gasification for pulverized coal in an entrained flow hydro-gasifier. The effects have been studied for different reaction conditions on hydro-gasification products and evolution of Hg in terms of the chemical reaction kinetics method. The CKM mechanism includes 130 elementary reactions and is solved with commercially available software, ChemKin. The calculation results are validated against the experimental data from literature and meaningful predictions are finally obtained. In addition, the chemical equilibrium calculation (CEC) is also used for predictions. Although the CEC method assumes all the reactions have reached chemical equilibrium, which is not the case in industrial reality, the calculation results are of value as reference.Tehnologija korištenja ugljena bez emisija (ZEC) se od nedavno aktivno proučava. Njezin cilj je postizanje nulte stope emisija CO2 te ostalih štetnih tvari dok efikasnost sustava mora biti minimalno 70%. Rasplinjavanje ugljene prašine vodenom parom je temeljni proces ZEC-a. Međutim, mehanizam rasplinjavanja i transformacije žive u rasplinjavanju vodenom parom još nije u potpunosti shvaćeno. To ograničava mogućnost komercijalne primjene ZEC-a. Cilj ovog rada je proučavanje mehanizama rasplinjavanja vodenom parom i transformacije žive za rasplinjavanje ugljena u rasplinjaču s visokim temperaturama i tlakom. Predloženi su detaljni kemijski kinetički mehanizmi (CKM) za rasplinjavanje ugljene prašine u fluidiziranom sloju sa zajedničkim tokom tvari. Proučeni su utjecaji raznih uvjeta pod kojim su se odvijale reakcije na produkte rasplinjavanja i evoluciju žive u uvjetima kemijskih reakcija kinetičke metode. CMK mehanizam sadrži 130 elementarnih reakcija i rješava se s komercijalno dostupnim programom, ChemKin. Rezultati simulacije se uspoređuju s eksperimentalnim iz literature te su konačno dobivena smislena predviđanja. Jednadžbe kemijske ravnoteže (CEC) su također korištene za predviđanja. Iako CEC metoda pretpostavlja da su sve reakcije postigle ravnotežu, što nije uvijek slučaj u industriji, rezultati tog proračuna mogu poslužiti kao referenca