Towards Spatio-Temporal Aware Traffic Time Series Forecasting--Full Version

Traffic time series forecasting is challenging due to complex spatio-temporal dynamics time series from different locations often have distinct patterns; and for the same time series, patterns may vary across time, where, for example, there exist certain periods across a day showing stronger temporal correlations. Although recent forecasting models, in particular deep learning based models, show promising results, they suffer from being spatio-temporal agnostic. Such spatio-temporal agnostic models employ a shared parameter space irrespective of the time series locations and the time periods and they assume that the temporal patterns are similar across locations and do not evolve across time, which may not always hold, thus leading to sub-optimal results. In this work, we propose a framework that aims at turning spatio-temporal agnostic models to spatio-temporal aware models. To do so, we encode time series from different locations into stochastic variables, from which we generate location-specific and time-varying model parameters to better capture the spatio-temporal dynamics. We show how to integrate the framework with canonical attentions to enable spatio-temporal aware attentions. Next, to compensate for the additional overhead introduced by the spatio-temporal aware model parameter generation process, we propose a novel window attention scheme, which helps reduce the complexity from quadratic to linear, making spatio-temporal aware attentions also have competitive efficiency. We show strong empirical evidence on four traffic time series datasets, where the proposed spatio-temporal aware attentions outperform state-of-the-art methods in term of accuracy and efficiency. This is an extended version of "Towards Spatio-Temporal Aware Traffic Time Series Forecasting", to appear in ICDE 2022 [1], including additional experimental results.Comment: Accepted at ICDE 202

Triformer:Triangular, Variable-Specific Attentions for Long Sequence Multivariate Time Series Forecasting--Full Version

A variety of real-world applications rely on far future information to make decisions, thus calling for efficient and accurate long sequence multivariate time series forecasting. While recent attention-based forecasting models show strong abilities in capturing long-term dependencies, they still suffer from two key limitations. First, canonical self attention has a quadratic complexity w.r.t. the input time series length, thus falling short in efficiency. Second, different variables' time series often have distinct temporal dynamics, which existing studies fail to capture, as they use the same model parameter space, e.g., projection matrices, for all variables' time series, thus falling short in accuracy. To ensure high efficiency and accuracy, we propose Triformer, a triangular, variable-specific attention. (i) Linear complexity: we introduce a novel patch attention with linear complexity. When stacking multiple layers of the patch attentions, a triangular structure is proposed such that the layer sizes shrink exponentially, thus maintaining linear complexity. (ii) Variable-specific parameters: we propose a light-weight method to enable distinct sets of model parameters for different variables' time series to enhance accuracy without compromising efficiency and memory usage. Strong empirical evidence on four datasets from multiple domains justifies our design choices, and it demonstrates that Triformer outperforms state-of-the-art methods w.r.t. both accuracy and efficiency. This is an extended version of "Triformer: Triangular, Variable-Specific Attentions for Long Sequence Multivariate Time Series Forecasting", to appear in IJCAI 2022 [Cirstea et al., 2022a], including additional experimental results

Graphene-based Material Systems for Nanoelectronics and Energy Storage Devices

Graphene is an allotrope of carbon in two-dimensional crystal form that has extraordinary electrical and optical properties. In this dissertation, we present the use of graphene in applications for chemical sensors, photovoltaics and supercapacitors. Firstly, carrier transport properties of single layer graphene films grown via chemical vapor deposition technique are tuned with functionalized molecules, polymers and inorganic nanoparticles. For example, cylindrical microdomains of polystyrene-4-polyvinylpyridine (PS-P4VP) block-copolymers (BCP) on graphene film provide spatial doping effects due to two distinct functional groups. Further, preferred interactions between CFx or fluorine radicals and BCP micro domains on graphene introduce localized doping of graphene film leading to controlling of Dirac point shift. Interaction between graphene and inorganic nanoparticles is studied by using CdSe quantum dots as a model system. Femtosecond time-resolved spectroscopy allowes us to demonstrate for the first time fast interfacial charge transfer for such systems in the picosecond and in the hundreds of femtosecond time domains, which also demonstrates high potential for photoelectrochemical cell. Secondly, graphene field effect transistors (GFET) as single strand DNA sensors are fabricated and detection limit as low as 3×10-9 M is demonstrated. Assembled BCP film on GFET sensor improved the sensor's stability and selectivity. The orientation and periodicity of the resulting cylindrical microdomains of BCP can facilitate the selective sensing property. With protective layer of BCP, sensor's stability under ambient atmosphere is improved up to 4 months. Thirdly, two different types of carbon nanotubes (CNT)/graphene hybrids are synthesized and used in fabrication of supercapacitors. The first type hybrid is graphene and vertically aligned carbon nanotubes which is successfully grown via one step chemical vapor deposition method. Our custom seamless growth method for such hybrids provides an attractive pathway for the fabrication of novel 3-Dimensional hybrid nanostructures. The second type hybrid is graphene oxide (GO) and SWCNT composite ink (GO-SWCNT ink). SWCNTs are dispersed using a GO aqueous solution (2mg/ml) with sonication support to achieve a SWCNT concentration of 12mg/ml, the highest reported value so far without surfactant assistance. Paper based electrodes for supercapacitors are fabricated using GO-SWCNT composite ink via dip casting method. By employing different concentrations of SWCNT inside the ink, supercapacitors demonstrated different capacitance values. The highest value of specific capacitance reaches up to 295 F/g at a current density of 0.5A/g with a GO/SWCNT weight ratio of 1:5. The cycling stability for the GO-SWCNT paper electrode supercapacitors indicates capacitance retention of 85% over 60,000 cycles. Finally, engineered interactions between nanomaterials, polymers, molecules and graphene/carbon nanotube can lead to the development of new types of devices for myriad applications

Direct Arsenic Removal from Water Using Non-Membrane, Low Temperature Directional Solvent Extraction

Arsenic (As) poisoning in water due to natural minerals or industrial pollution is a critical global problem that threatens the health and life of billions. Current arsenic removal techniques involving chemical reaction, ion exchange, or membrane processes can be expensive, inaccessible or infeasible for underdeveloped regions or remote areas. Here, we demonstrate that using a so-called directional solvent extraction (DSE) process, arsenic ions in water can be effectively removed without the need of a membrane or chemical reaction, and this process promises to utilize very low temperature heat (as low as 45 oC). We have tested feed water with different arsenic concentrations and arsenic ions in different forms (As-III and As-V) commonly found in nature. It is demonstrated that DSE using decanoic acid as the directional solvent can purify contaminated water to meet the drinking water standard (arsenic concentration < 10 parts per billion, ppb), and the arsenic removal efficiencies are higher than 91% for As-III and 97% for As-V. Moreover, DSE can remove As-III directly without the need of pre-oxidation, which is required in most of the state of art techniques. DSE can potentially lead to effective arsenic removal technologies with low resource settings that are suitable for remote and underdeveloped regions, which are impacted by arsenic poisoning the most

Theoretical Analysis of Continuous-Wave Mid-Infrared Optical Vortex Source Generated by Singly Resonant Optical Parametric Oscillator

Due to the important application in the study of vibrational circular dichroism and helical dichroism of chiral molecules, the tunable vortex beam at mid-infrared region has attracted increasing attention. Based on orbital angular momentum (OAM) conservation in nonlinear interactions, the vortex pumped singly resonant optical parametric oscillator (SRO) is recognized as a versatile source of coherent vortex radiation providing high power and broad wavelength coverage from a single device. However, the low parametric gain and high oscillation threshold under continuous wave (cw) pumping has so far been the most challenging factor in generating cw tunable vortex beams. To predict the output characteristic of vortex pumped SRO, a theoretical model describing the vortex pumped SRO is needed. In this study, the theoretical model describing the vortex pumped SRO is set up under collimated Gaussian beam approximation. Output characteristics of different SROs are simulated numerically. By proper selection of pump scheme (such as double-pass pumping scheme), the vortex pumped mid-infrared SRO can oscillate at a relatively low pump power. By controlling the gain (mode overlap ratio between the pump and resonant wave in the nonlinear crystal) and loss (employing a spot-defect mirror with different defect size as the output coupler) of the resonant signal mode in the SRO, the OAM of the pump beam can be directionally transferred to a specific down converted beam. The transfer mechanism of the OAM among the pump light and the down-converted beams and factors affecting the transfer are studied. Our study provides the guidelines for the design and optimization of vortex pumped SRO under cw operation

PriRepVGG: Privacy-Preserving 3-Party Inference Framework for Image-Based Defect Detection

Image classification is widely used in industrial defect detection, medical diagnosis, social welfare, and other fields, in which privacy and security of models and data must be involved. For example, in diamond synthesis, the diamond substrate image annotation data and the defect detection model are of value for conservation. Based on ensuring inference efficiency and the security of these private data intellectual property, the 3-party secure inference based on secure multi-party computation (MPC) can be adopted. MPC allows parties to use neural networks while preserving their input privacy for collaborative computing, but it will lead to huge communication and memory consumption. This paper propose PriRepVGG, a lightweight privacy-preserving image-based defect detection framework for 3-party. In this work, firstly, This work optimized the division and added an AdaptiveAvgpool layer in MPC framework FALCON; then, This work ported the inference architecture of the RegVGG network into FALCON creatively. Our work applied PriRepVGG to the secure inference of the diamond substrates defect detection under the data server, model server, and compute server settings, which can be carried out in batches with a low misjudgment rate and verify the feasibility of image-based secure inference with a lightweight network in an industrial case under MPC

PriRepVGG: Privacy-Preserving 3-Party Inference Framework for Image-Based Defect Detection

Image classification is widely used in industrial defect detection, medical diagnosis, social welfare, and other fields, in which privacy and security of models and data must be involved. For example, in diamond synthesis, the diamond substrate image annotation data and the defect detection model are of value for conservation. Based on ensuring inference efficiency and the security of these private data intellectual property, the 3-party secure inference based on secure multi-party computation (MPC) can be adopted. MPC allows parties to use neural networks while preserving their input privacy for collaborative computing, but it will lead to huge communication and memory consumption. This paper propose PriRepVGG, a lightweight privacy-preserving image-based defect detection framework for 3-party. In this work, firstly, This work optimized the division and added an AdaptiveAvgpool layer in MPC framework FALCON; then, This work ported the inference architecture of the RegVGG network into FALCON creatively. Our work applied PriRepVGG to the secure inference of the diamond substrates defect detection under the data server, model server, and compute server settings, which can be carried out in batches with a low misjudgment rate and verify the feasibility of image-based secure inference with a lightweight network in an industrial case under MPC

Theoretical Analysis of Continuous-Wave Mid-Infrared Optical Vortex Source Generated by Singly Resonant Optical Parametric Oscillator

Due to the important application in the study of vibrational circular dichroism and helical dichroism of chiral molecules, the tunable vortex beam at mid-infrared region has attracted increasing attention. Based on orbital angular momentum (OAM) conservation in nonlinear interactions, the vortex pumped singly resonant optical parametric oscillator (SRO) is recognized as a versatile source of coherent vortex radiation providing high power and broad wavelength coverage from a single device. However, the low parametric gain and high oscillation threshold under continuous wave (cw) pumping has so far been the most challenging factor in generating cw tunable vortex beams. To predict the output characteristic of vortex pumped SRO, a theoretical model describing the vortex pumped SRO is needed. In this study, the theoretical model describing the vortex pumped SRO is set up under collimated Gaussian beam approximation. Output characteristics of different SROs are simulated numerically. By proper selection of pump scheme (such as double-pass pumping scheme), the vortex pumped mid-infrared SRO can oscillate at a relatively low pump power. By controlling the gain (mode overlap ratio between the pump and resonant wave in the nonlinear crystal) and loss (employing a spot-defect mirror with different defect size as the output coupler) of the resonant signal mode in the SRO, the OAM of the pump beam can be directionally transferred to a specific down converted beam. The transfer mechanism of the OAM among the pump light and the down-converted beams and factors affecting the transfer are studied. Our study provides the guidelines for the design and optimization of vortex pumped SRO under cw operation