FreeREA: Training-Free Evolution-based Architecture Search

In the last decade, most research in Machine Learning contributed to the improvement of existing models, with the aim of increasing the performance of neural networks for the solution of a variety of different tasks. However, such advancements often come at the cost of an increase of model memory and computational requirements. This represents a significant limitation for the deployability of research output in realistic settings, where the cost, the energy consumption, and the complexity of the framework play a crucial role. To solve this issue, the designer should search for models that maximise the performance while limiting its footprint. Typical approaches to reach this goal rely either on manual procedures, which cannot guarantee the optimality of the final design, or upon Neural Architecture Search algorithms to automatise the process, at the expenses of extremely high computational time. This paper provides a solution for the fast identification of a neural network that maximises the model accuracy while preserving size and computational constraints typical of tiny devices. Our approach, named FreeREA, is a custom cell-based evolution NAS algorithm that exploits an optimised combination of training-free metrics to rank architectures during the search, thus without need of model training. Our experiments, carried out on the common benchmarks NAS-Bench-101 and NATS-Bench, demonstrate that i) FreeREA is the first method able to provide very accurate models in minutes of search time; ii) it outperforms State of the Art training-based and training-free techniques in all the datasets and benchmarks considered, and iii) it can easily generalise to constrained scenarios, representing a competitive solution for fast Neural Architecture Search in generic constrained applications.Comment: 16 pages, 4 figurre

Entropic Score metric: Decoupling Topology and Size in Training-free NAS

Neural Networks design is a complex and often daunting task, particularly for resource-constrained scenarios typical of mobile-sized models. Neural Architecture Search is a promising approach to automate this process, but existing competitive methods require large training time and computational resources to generate accurate models. To overcome these limits, this paper contributes with: i) a novel training-free metric, named Entropic Score, to estimate model expressivity through the aggregated element-wise entropy of its activations; ii) a cyclic search algorithm to separately yet synergistically search model size and topology. Entropic Score shows remarkable ability in searching for the topology of the network, and a proper combination with LogSynflow, to search for model size, yields superior capability to completely design high-performance Hybrid Transformers for edge applications in less than 1 GPU hour, resulting in the fastest and most accurate NAS method for ImageNet classification.Comment: 10 pages, 3 figure

Adversarial Branch Architecture Search for Unsupervised Domain Adaptation

Unsupervised Domain Adaptation (UDA) is a key issue in visual recognition, as it allows to bridge different visual domains enabling robust performances in the real world. To date, all proposed approaches rely on human expertise to manually adapt a given UDA method (e.g. DANN) to a specific backbone architecture (e.g. ResNet). This dependency on handcrafted designs limits the applicability of a given approach in time, as old methods need to be constantly adapted to novel backbones. Existing Neural Architecture Search (NAS) approaches cannot be directly applied to mitigate this issue, as they rely on labels that are not available in the UDA setting. Furthermore, most NAS methods search for full architectures, which precludes the use of pre-trained models, essential in a vast range of UDA settings for reaching SOTA results. To the best of our knowledge, no prior work has addressed these aspects in the context of NAS for UDA. Here we tackle both aspects with an Adversarial Branch Architecture Search for UDA (ABAS): i. we address the lack of target labels by a novel data-driven ensemble approach for model selection; and ii. we search for an auxiliary adversarial branch, attached to a pre-trained backbone, which drives the domain alignment. We extensively validate ABAS to improve two modern UDA techniques, DANN and ALDA, on three standard visual recognition datasets (Office31, Office-Home and PACS). In all cases, ABAS robustly finds the adversarial branch architectures and parameters which yield best performances.Comment: Accepted at WACV 202

Entropic Score Metric: Decoupling Topology and Size in Training-Free NAS

Neural Networks design is a complex and often daunting task, particularly for resource-constrained scenarios typi- cal of mobile-sized models. Neural Architecture Search is a promising approach to automate this process, but existing competitive methods require large training time and com- putational resources to generate accurate models. To over- come these limits, this paper contributes with: i) a novel training-free metric, named Entropic Score, to estimate model expressivity through the aggregated element-wise en- tropy of its activations; ii) a cyclic search algorithm to sep- arately yet synergistically search model size and topology. Entropic Score shows remarkable ability in searching for the topology of the network, and a proper combination with LogSynflow, to search for model size, yields superior capa- bility to completely design high-performance Hybrid Trans- formers for edge applications in less than 1 GPU hour, re- sulting in the fastest and most accurate NAS method for Im- ageNet classification

Photoactive spherical colloids for opal photonic crystals

The synthesis and characterization of submicrometersized polymer particles, functionalized by different techniques with fluorescent dyes and featuring tunable surface charges, are described. Dyes with a polymerizable moiety were incorporated during emulsifier-free or seeded emulsion polymerization, whereas nonpolymerizable dyes were included into preformed nanoparticles by a swelling and deswelling process. The particle surface charge was controlled through the choice of suitable initiators. All the nanoparticles were successfully used to grow high optical quality opal photonic crystals by the vertical deposition technique. Optical characterization of such photonic crystals pointed out the presence of the optical stop band and the high-energy van Hove-like structures, both scaling with the particle diameter according to the scaling laws of photonic crystals and possessing the expected angular dispersion. These results are indicative of the high size uniformity and of the surface quality of the nanospheres independently on the synthetic method adopted. Preliminary data seem to suggest some effect of the opal photonic band structure on the dye fluorescence spectra

Fluorescent polystyrene photonic crystals self-assembled with water-soluble conjugated polyrotaxanes

We demonstrate control of the photoluminescence spectra and decay rates of water-soluble green-emitting conjugated polyrotaxanes by incorporating them in polystyrene opals with a stop-band spectrally tuned on the rotaxane emission (405–650 nm). We observe a suppression of the luminescence within the photonic stop-band and a corresponding enhancement of the high-energy edge (405–447 nm). Time-resolved measurements reveal a wavelength-dependent modification of the emission lifetime, which is shortened at the high-energy edge (by ∼11%, in the range 405–447 nm), but elongated within the stop-band (by ∼13%, in the range 448–482 nm). We assign both effects to the modification of the density of photonic states induced by the photonic crystal band structure. We propose the growth of fluorescent composite photonic crystals from blends of “solvent-compatible” non-covalently bonded nanosphere-polymer systems as a general method for achieving a uniform distribution of polymeric dopants in three-dimensional self-assembling photonic structures

Ionic Strength Responsive Sulfonated Polystyrene Opals

Stimuli-responsive photonic crystals (PCs) represent an intriguing class of smart materials very promising for sensing applications. Here, selective ionic strength responsive polymeric PCs are reported. They are easily fabricated by partial sulfonation of polystyrene opals, without using toxic or expensive monomers and etching steps. The color of the resulting hydrogel-like ordered structures can be continuously shifted over the entire visible range (405–760 nm) by changing the content of ions over an extremely wide range of concentration (from about 70 μM to 4 M). The optical response is completely independent from pH and temperature, and the initial color can be fully recovered by washing the sulfonated opals with pure water. These new smart photonic materials could find important applications as ionic strength sensors for environmental monitoring as well as for healthcare screening