Split Additive Manufacturing for Printed Neuromorphic Circuits

Printed and flexible electronics promises smart devices for application domains, such as smart fast moving consumer goods and medical wearables, which are generally untouchable by conventional rigid silicon technologies. This is due to their remarkable properties such as flexibility, non-toxic materials, and having low-cost per area. Combined with neuromorphic computing, printed neuromorphic circuits pose an attractive solution for these application domains. Particularly, the additive printing technologies can reduce large amount of fabrication complexities and costs. On the one hand, high-throughput additive printing processes, such as roll-to-roll printing, can reduce the per-device fabrication time and cost. On the other hand, jet-printing can provide point-of-use customization at the expense of lower fabrication throughput. In this work, we propose a machine learning based design framework, that respects the objective and physical constraints of split additive manufacturing for printed neuromorphic circuits. With the proposed framework, multiple printed neural networks are trained jointly with the aim to sensibly combine multiple fabrication techniques (e.g., roll-to-roll and jet-printing). This should lead to a cost-effective fabrication of multiple different printed neuromorphic circuits and achieve high fabrication throughput, lower cost, and point-of-use customization

Aging-Aware Training for Printed Neuromorphic Circuits

Printed electronics allow for ultra-low-cost circuit fabrication with unique properties such as flexibility, non-toxicity, and stretchability. Because of these advanced properties, there is a growing interest in adapting printed electronics for emerging areas such as fast-moving consumer goods and wearable technologies. In such domains, analog signal processing in or near the sensor is favorable. Printed neuromorphic circuits have been recently proposed as a solution to perform such analog processing natively. Additionally, their learning-based design process allows high efficiency of their optimization and enables them to mitigate the high process variations associated with low-cost printed processes. In this work, we propose a learning-based approach to address another major challenge of printed electronics, namely the aging of the printed components. This effect can significantly degrade the accuracy of printed neuromorphic circuits over time. For this, we develop a stochastic aging-model to describe the behavior of aged printed resistors and modify the training objective by considering the expected loss over the lifetime of the device. This approach ensures to provide acceptable accuracy over the device lifetime. Our experiments show that an overall 35.8\% improvement in terms of expected accuracy over the device lifetime can be achieved using the proposed learning approach

Improving Human Activity Recognition Models by Learnable Sparse Wavelet Layer

Modern machine learning algorithms for human activity recognition based on artificial neural networks often require a large amount of labelled training data to generalize between human subjects and training contexts. Large degrees of freedom make them susceptible to overfitting and often computationally intensive to implement on portable hardware. In this work, we introduce wavelet-based learnable filters as a feature extraction layer that greatly improves the generalization capability of the detector model. Our evaluations on six benchmark datasets show significant improvements in macro $F_1$ score when our wavelet-based learnable filter layer is prepended to three state-of-the-art human activity recognition models. As a side effect, in many cases we could drastically reduce the required model size to achieve competitive performance on the benchmark dataset, which is an important requirement for use in wearable computing

Power-Aware Training for Energy-Efficient Printed Neuromorphic Circuits

There is an increasing demand for next-generation flexible electronics in emerging low-cost applications such as smart packaging and smart bandages, where conventional silicon electronics cannot enter due to cost and form factor. In these domains, ultra-low-cost, high flexibility, and customizability are required. In this regard, printed electronics emerge as a complementary solution offering the aforementioned properties. To respect the constraints in those application scenarios and equip printed devices with the fundamental capability to process information, analog printed neuromorphic circuits offer multiple advantages, including strong expressiveness, streamlined circuit primitives, and a highly efficient machine learning-based design process. In this work, we focus on designing low-power printed neuromorphic circuits at the algorithmic level. By developing accurate power models for the circuit primitives, the power consumption can be considered into the design process. Subsequently, Pareto analysis is employed to examine the relationship between accuracy and power consumption. Experimental results reveal that, with the proposed approach, 2× reduction of the power consumption can be realized while maintaining 95% of classification accuracy. This approach has significant implications for the future development of energy-efficient printed neuromorphic circuits and their potential applications in IoT and AI intersections

Universal Distributional Decision-based Black-box Adversarial Attack with Reinforcement Learning

The vulnerability of the high-performance machine learning models implies a security risk in applications with real-world consequences. Research on adversarial attacks is beneficial in guiding the development of machine learning models on the one hand and finding targeted defenses on the other. However, most of the adversarial attacks today leverage the gradient or logit information from the models to generate adversarial perturbation. Works in the more realistic domain: decision-based attacks, which generate adversarial perturbation solely based on observing the output label of the targeted model, are still relatively rare and mostly use gradient-estimation strategies. In this work, we propose a pixel-wise decision-based attack algorithm that finds a distribution of adversarial perturbation through a reinforcement learning algorithm. We call this method Decision-based Black-box Attack with Reinforcement learning (DBAR). Experiments show that the proposed approach outperforms state-of-the-art decision-based attacks with a higher attack success rate and greater transferability

TinyHAR: A Lightweight Deep Learning Model Designed for Human Activity Recognition

Deep learning models have shown excellent performance in human activity recognition tasks. However, these models typically require large amounts of computational resources, which makes them inefficient to deploy on edge devices. Furthermore, the superior performance of deep learning models relies heavily on the availability of large datasets to avoid over-fitting. However, the expensive efforts for labeling limits the amount of datasets. We address both challenges by designing a more lightweight model, called TinyHAR. TinyHAR is designed specifically for human activity recognition employing different saliency of multi modalities, multimodal collaboration, and temporal information extraction. Initial experimental results show that TinyHAR is several times smaller and often meets or even surpasses the performance of DeepConvLSTM, a state-of-the-art human activity recognition model

Automatic Feature Engineering through Monte Carlo Tree Search

The performance of machine learning models depends heavily on the feature space and feature engineering. Although neural networks have made significant progress in learning latent feature spaces from data, compositional feature engineering through nested feature transformations can reduce model complexity and can be particularly desirable for interpretability. To find suitable transformations automatically, state-of-the-art methods model the feature transformation space by graph structures and use heuristics such as $\epsilon$-greedy to search for them. Such search strategies tend to become less efficient over time because they do not consider the sequential information of the candidate sequences and cannot dynamically adjust the heuristic strategy. To address these shortcomings, we propose a reinforcement learning-based automatic feature engineering method, which we call Monte Carlo tree search Automatic Feature Engineering (mCAFE). We employ a surrogate model that can capture the sequential information contained in the transformation sequence and thus can dynamically adjust the exploration strategy. It balances exploration and exploitation by Thompson sampling and uses a Long Short Term Memory (LSTM) based surrogate model to estimate sequences of promising transformations. In our experiments, mCAFE outperformed state-of-the-art automatic feature engineering methods on most common benchmark datasets