7 research outputs found
Pointer Networks Trained Better via Evolutionary Algorithms
Pointer Network (PtrNet) is a specific neural network for solving
Combinatorial Optimization Problems (COPs). While PtrNets offer real-time
feed-forward inference for complex COPs instances, its quality of the results
tends to be less satisfactory. One possible reason is that such issue suffers
from the lack of global search ability of the gradient descent, which is
frequently employed in traditional PtrNet training methods including both
supervised learning and reinforcement learning. To improve the performance of
PtrNet, this paper delves deeply into the advantages of training PtrNet with
Evolutionary Algorithms (EAs), which have been widely acknowledged for not
easily getting trapped by local optima. Extensive empirical studies based on
the Travelling Salesman Problem (TSP) have been conducted. Results demonstrate
that PtrNet trained with EA can consistently perform much better inference
results than eight state-of-the-art methods on various problem scales. Compared
with gradient descent based PtrNet training methods, EA achieves up to 30.21\%
improvement in quality of the solution with the same computational time. With
this advantage, this paper is able to at the first time report the results of
solving 1000-dimensional TSPs by training a PtrNet on the same dimensionality,
which strongly suggests that scaling up the training instances is in need to
improve the performance of PtrNet on solving higher-dimensional COPs.Comment: Non
Reliable Robustness Evaluation via Automatically Constructed Attack Ensembles
Attack Ensemble (AE), which combines multiple attacks together, provides a reliable way to evaluate adversarial robustness. In practice, AEs are often constructed and tuned by human experts, which however tends to be sub-optimal and time-consuming. In this work, we present AutoAE, a conceptually simple approach for automatically constructing AEs. In brief, AutoAE repeatedly adds the attack and its iteration steps to the ensemble that maximizes ensemble improvement per additional iteration consumed. We show theoretically that AutoAE yields AEs provably within a constant factor of the optimal for a given defense. We then use AutoAE to construct two AEs for l∞ and l2 attacks, and apply them without any tuning or adaptation to 45 top adversarial defenses on the RobustBench leaderboard. In all except one cases we achieve equal or better (often the latter) robustness evaluation than existing AEs, and notably, in 29 cases we achieve better robustness evaluation than the best known one. Such performance of AutoAE shows itself as a reliable evaluation protocol for adversarial robustness, which further indicates the huge potential of automatic AE construction. Code is available at https://github.com/LeegerPENG/AutoAE
Network Security Situation Prediction Model Based on VMD Decomposition and DWOA Optimized BiGRU-ATTN Neural Network
The widespread adoption of Internet-of-Things (IoT) devices has resulted in a comprehensive transformation of human life. However, the network security challenges posed by the IoT devices have become increasingly severe, necessitating the implementation of effective security mechanisms. Network security situational awareness enables an effective network state prediction for better formulation of network security defense strategies. Existing network security situational prediction methods are typically constrained by situational sequence data, especially those sequences with a high degree of non-stationarity, leading to unstable predictions and low performance. Moreover, in real-world application scenarios, the network security situational sequences are often highly non-stationary. To address these challenges, we introduce a novel hybrid prediction model named Variational Mode Decomposition (VMD) - Dynamic Whale Optimization Algorithm (DWOA) - Bidirectional Gated Recurrent Unit (BiGRU) - Attention Mechanism (ATTN). The proposed model integrates VMD, BiGRU, ATTN, and DWOA. Initially, network security situational awareness sequences are processed using VMD to decompose them into a series of subsequences, thus reducing the non-stationarity of the original sequences. Subsequently, an enhanced DWOA optimization algorithm is introduced for tuning the hyperparameters of the BiGRU-ATTN network. Ultimately, BiGRU-ATTN is employed to predict each of these subsequences, which are then aggregated to yield the final network security situational prediction value. When compared with several existing methods on public network security datasets, the proposed VMD-DWOA-BiGRU-ATTN method demonstrated an improvement in the R^2 values ranging from 6.34% to 52.61%. These results substantiate that the model significantly enhances predictive performance
Training Quantized Deep Neural Networks via Cooperative Coevolution
This work considers a challenging Deep Neural Network(DNN) quantization task
that seeks to train quantized DNNs without involving any full-precision
operations. Most previous quantization approaches are not applicable to this
task since they rely on full-precision gradients to update network weights. To
fill this gap, in this work we advocate using Evolutionary Algorithms (EAs) to
search for the optimal low-bits weights of DNNs. To efficiently solve the
induced large-scale discrete problem, we propose a novel EA based on
cooperative coevolution that repeatedly groups the network weights based on the
confidence in their values and focuses on optimizing the ones with the least
confidence. To the best of our knowledge, this is the first work that applies
EAs to train quantized DNNs. Experiments show that our approach surpasses
previous quantization approaches and can train a 4-bit ResNet-20 on the
Cifar-10 dataset with the same test accuracy as its full-precision counterpart.Comment: 13 pages, 4 figures, accepted for publication of ICS
<i>In Situ</i> Self-Template Synthesis of Fe–N-Doped Double-Shelled Hollow Carbon Microspheres for Oxygen Reduction Reaction
Herein, we reported
a special Fe–N-doped double-shelled
hollow carbon microsphere (Fe–N-DSC) which was prepared by
a facile, <i>in situ</i> polymerization followed by pyrolysis.
With porous ferroferric oxide (Fe<sub>3</sub>O<sub>4</sub>) hollow
microspheres as the templates, where pyrrole monomers were dispersed
around the outer surface and prefilled the interior space. By adding
hydrochloric acid, Fe<sup>3+</sup> ions were released to initiate
polymerization of pyrrole on both the outer and inner surfaces of
Fe<sub>3</sub>O<sub>4</sub> microspheres until they were completely
dissolved, resulting in the Fe-containing polypyrrole double-shelled
hollow carbon microspheres (Fe-PPY-DSC). The Fe-PPY-DSC was then pyrolyzed
to generate the Fe-N-DSC. The Fe<sub>3</sub>O<sub>4</sub> hollow microspheres
played trifunctional roles, <i>i.e.</i>, the template to
prepare a double-shelled hollow spherical structure, the initiator
(<i>i.e.</i>, Fe<sup>3+</sup> ions) for the polymerization
of pyrrole, and the Fe source for doping. The Fe–N-DSC exhibited
a superior catalytic activity for oxygen reduction as comparable to
commercial Pt/C catalysts in both alkaline and acidic media. The high
catalytic performance was ascribed to the special porous double-shelled
hollow spherical structure, which provided more active sites and was
beneficial to a high-flux mass transportation