FastDeepIoT: Towards Understanding and Optimizing Neural Network Execution Time on Mobile and Embedded Devices

Deep neural networks show great potential as solutions to many sensing application problems, but their excessive resource demand slows down execution time, pausing a serious impediment to deployment on low-end devices. To address this challenge, recent literature focused on compressing neural network size to improve performance. We show that changing neural network size does not proportionally affect performance attributes of interest, such as execution time. Rather, extreme run-time nonlinearities exist over the network configuration space. Hence, we propose a novel framework, called FastDeepIoT, that uncovers the non-linear relation between neural network structure and execution time, then exploits that understanding to find network configurations that significantly improve the trade-off between execution time and accuracy on mobile and embedded devices. FastDeepIoT makes two key contributions. First, FastDeepIoT automatically learns an accurate and highly interpretable execution time model for deep neural networks on the target device. This is done without prior knowledge of either the hardware specifications or the detailed implementation of the used deep learning library. Second, FastDeepIoT informs a compression algorithm how to minimize execution time on the profiled device without impacting accuracy. We evaluate FastDeepIoT using three different sensing-related tasks on two mobile devices: Nexus 5 and Galaxy Nexus. FastDeepIoT further reduces the neural network execution time by $48\%$ to $78\%$ and energy consumption by $37\%$ to $69\%$ compared with the state-of-the-art compression algorithms.Comment: Accepted by SenSys '1

Attention-based machine perception for intelligent cyber-physical systems

Cyber-physical systems (CPS) fundamentally change the way of how information systems interact with the physical world. They integrate the sensing, computing, and communication capabilities on heterogeneous platforms and infrastructures. Efficient and effective perception of the environment lays the foundation of proper operations in other CPS components (e.g., planning and control). Recent advances in artificial intelligence (AI) have unprecedentedly changed the way of how cyber systems extract knowledge from the collected sensing data, and understand the physical surroundings. This novel data-to-knowledge transformation capability pushes a wide spectrum of recognition tasks (e.g., visual object detection, speech recognition, and sensor-based human activity recognition) to a higher level, and opens an new era of intelligent cyber-physical systems. However, the state-of-the-art neural perception models are typically computation-intensive and sensitive to data noises, which induce significant challenges when they are deployed on resources-limited embedded platforms. This dissertation works on optimizing both the efficiency and efficacy of deep-neural- network (DNN)-based machine perception in intelligent cyber-physical systems. We extensively exploit and apply the design philosophy of attention, originated from cognitive psychology field, from multiple perspectives of machine perception. It generally means al- locating different degrees of concentration to different perceived stimuli. Specifically, we address the following five research questions: First, can we run the computation-intensive neural perception models in real-time by only looking at (i.e., scheduling) the important parts of the perceived scenes, with the cueing from an external sensor? Second, can we eliminate the dependency on the external cueing and make the scheduling framework a self- cueing system? Third, how to distribute the workloads among cameras in a distributed (visual) perception system, where multiple cameras can observe the same parts of the environment? Fourth, how to optimize the achieved perception quality when sensing data from heterogeneous locations and sensor types are collected and utilized? Fifth, how to handle sensor failures in a distributed sensing system, when the deployed neural perception models are sensitive to missing data? We formulate the above problems, and introduce corresponding attention-based solutions for each, to construct the fundamental building blocks for envisioning an attention-based machine perception system in intelligent CPS with both efficiency and efficacy guarantees

One-step preparation of optically transparent Ni-Fe oxide film electrocatalyst for oxygen evolution reaction

Optically transparent cocatalyst film materials is very desirable for improved photoelectrochemical (PEC) oxygen evolution reaction (OER) over light harvesting photoelectrodes which require the exciting light to irradiate through the cocatalyst side, i.e., front-side illumination. In view of the reaction overpotential at electrode/electrolyte interface, the OER electrocatalysts have been extensively used as cocatalysts for PEC water oxidation on photoanode. In this work, the feasibility of a one-step fabrication of the transparent thin film catalyst for efficient electrochemical OER is investigated. The Ni-Fe bimetal oxide films, 200 nm in thickness, are used for study. Using a reactive magnetron co-sputtering technique, transparent (> 50% in wavelength range 500-2000 nm) Ni-Fe oxide films with high electrocatalytic activities were successfully prepared at room temperature. Upon optimization, the as-prepared bimetal oxide film with atomic ratio of Fe/Ni = 3:7 demonstrates the lowest overpotential for the OER in aqueous KOH solution, as low as 329 mV at current density of 2 mA cm 2, which is 135 and 108 mV lower than that of as-sputtered FeOx and NiOx thin films, respectively. It appears that this fabrication strategy is very promising to deposit optically transparent cocatalyst films on photoabsorbers for efficient PEC water splitting.This work was financially supported by the National Natural Science Foundation of China (No. 21090340), 973 National Basic Research Program of the Ministry of Science and Technology (No. 2014CB239400) and Solar Energy Action Plan of Chinese Academy of Sciences (KGCX2-YW-399+7-3)

FOCAL: Contrastive Learning for Multimodal Time-Series Sensing Signals in Factorized Orthogonal Latent Space

This paper proposes a novel contrastive learning framework, called FOCAL, for extracting comprehensive features from multimodal time-series sensing signals through self-supervised training. Existing multimodal contrastive frameworks mostly rely on the shared information between sensory modalities, but do not explicitly consider the exclusive modality information that could be critical to understanding the underlying sensing physics. Besides, contrastive frameworks for time series have not handled the temporal information locality appropriately. FOCAL solves these challenges by making the following contributions: First, given multimodal time series, it encodes each modality into a factorized latent space consisting of shared features and private features that are orthogonal to each other. The shared space emphasizes feature patterns consistent across sensory modalities through a modal-matching objective. In contrast, the private space extracts modality-exclusive information through a transformation-invariant objective. Second, we propose a temporal structural constraint for modality features, such that the average distance between temporally neighboring samples is no larger than that of temporally distant samples. Extensive evaluations are performed on four multimodal sensing datasets with two backbone encoders and two classifiers to demonstrate the superiority of FOCAL. It consistently outperforms the state-of-the-art baselines in downstream tasks with a clear margin, under different ratios of available labels. The code and self-collected dataset are available at https://github.com/tomoyoshki/focal.Comment: Code available at: [github](https://github.com/tomoyoshki/focal

The Evaluation of the Oxidative Stress Parameters in Patients with Primary Angle-Closure Glaucoma

Objective: To clarify the presence of oxidative stress in patients with primary angle-closure glaucoma (PACG) and to investigate the relationship between oxidative stress and PACG. Methods: Fifty patients with primary angle-closure glaucoma and fifty healthy controls of matched age and gender were included in the study prospectively. Serum samples were obtained to detect the oxidation degradation products malondialdehyde (MDA), conjugated diene (CD), 4-hydroxynonenal (4-HNE), advanced oxidation protein products (AOPP), protein carbonyl (PC), ischemia-modified albumin (IMA) and 8-hydroxydeoxyguanosin (8-OHdG). Results: The concentration of MDA and CD in PACG patients was significantly higher than those of the control subjects (P,0.05, P,0.01). The serum 4-HNE concentrations were increased in PACG patients, but the differences with those of the healthy controls were not statistically significant. Compared to normal subjects, there was significant higher in serum AOPP and PC in PACG patients (P,0.01). PACG patients had higher levels of 8-OHdG in serum with respect to the comparative group of normal subjects (P,0.01). When plasma IMA levels in the PACG group were compared with those in the control group, significant increases in IMA were observed in the former (P,0.05). Conclusions: Our study demonstrated that IMA is a new biomarker available for assessing oxidative stress in PCAG. Oxidative stress is an important risk factor in the development of primary angle-closure glaucoma. Increased levels o

On the Mechanism of Solvents Catalyzed Structural Transformation in Metal Halide Perovskites

Metal halide perovskites show the capability of performing structural transformation, allowing the formation of functional heterostructures. Unfortunately, the elusive mechanism governing these transformations limits their technological application. Herein, the mechanism of 2D–3D structural transformation is unraveled as catalyzed by solvents. By combining a spatial-temporal cation interdiffusivity simulation with experimental findings, it is validated that, protic solvents foster the dissociation degree of formadinium iodide (FAI) via dynamic hydrogen bond, then the stronger hydrogen bond of phenylethylamine (PEA) cation with selected solvents compared to dissociated FA cation facilitates 2D–3D transformation from (PEA)2PbI4 to FAPbI3. It is discovered that, the energy barrier of PEA out-diffusion and the lateral transition barrier of inorganic slab are diminished. For 2D films the protic solvents catalyze grain centers (GCs) and grain boundaries (GBs) transforme into 3D phases and quasi-2D phases, respectively. While in the solvent-free case, GCs transform into 3D–2D heterostructures along the direction perpendicular to the substrate, and most GBs evolve into 3D phases. Finally, memristor devices fabricated using the transformed films uncover that, GBs composed of 3D phases are more prone to ion migration. This work elucidates the fundamental mechanism of structural transformation in metal halide perovskites, allowing their use to fabricate complex heterostructures.</p