Resilient neural network training for accelerators with computing errors

—With the advancements of neural networks, customized accelerators are increasingly adopted in massive AI applications. To gain higher energy efficiency or performance, many hardware design optimizations such as near-threshold logic or overclocking can be utilized. In these cases, computing errors may happen and the computing errors are difficult to be captured by conventional training on general purposed processors (GPPs). Applying the offline trained neural network models to the accelerators with errors directly may lead to considerable prediction accuracy loss. To address this problem, we explore the resilience of neural network models and relax the accelerator design constraints to enable aggressive design options. First of all, we propose to train the neural network models using the accelerators’ forward computing results such that the models can learn both the data and the computing errors. In addition, we observe that some of the neural network layers are more sensitive to the computing errors. With this observation, we schedule the most sensitive layer to the attached GPP to reduce the negative influence of the computing errors. According to the experiments, the neural network models obtained from the proposed training outperform the original models significantly when the CNN accelerators are affected by computing errors

A MapReduce-based nearest neighbor approach for big-data-driven traffic flow prediction

In big-data-driven traffic flow prediction systems, the robustness of prediction performance depends on accuracy and timeliness. This paper presents a new MapReduce-based nearest neighbor (NN) approach for traffic flow prediction using correlation analysis (TFPC) on a Hadoop platform. In particular, we develop a real-time prediction system including two key modules, i.e., offline distributed training (ODT) and online parallel prediction (OPP). Moreover, we build a parallel k-nearest neighbor optimization classifier, which incorporates correlation information among traffic flows into the classification process. Finally, we propose a novel prediction calculation method, combining the current data observed in OPP and the classification results obtained from large-scale historical data in ODT, to generate traffic flow prediction in real time. The empirical study on real-world traffic flow big data using the leave-one-out cross validation method shows that TFPC significantly outperforms four state-of-the-art prediction approaches, i.e., autoregressive integrated moving average, Naïve Bayes, multilayer perceptron neural networks, and NN regression, in terms of accuracy, which can be improved 90.07% in the best case, with an average mean absolute percent error of 5.53%. In addition, it displays excellent speedup, scaleup, and sizeup

An efficient MapReduce-based parallel clustering algorithm for distributed traffic subarea division

Traffic subarea division is vital for traffic system management and traffic network analysis in intelligent transportation systems (ITSs). Since existing methods may not be suitable for big traffic data processing, this paper presents a MapReduce-based Parallel Three-Phase K -Means (Par3PKM) algorithm for solving traffic subarea division problem on a widely adopted Hadoop distributed computing platform. Specifically, we first modify the distance metric and initialization strategy of K -Means and then employ a MapReduce paradigm to redesign the optimized K -Means algorithm for parallel clustering of large-scale taxi trajectories. Moreover, we propose a boundary identifying method to connect the borders of clustering results for each cluster. Finally, we divide traffic subarea of Beijing based on real-world trajectory data sets generated by 12,000 taxis in a period of one month using the proposed approach. Experimental evaluation results indicate that when compared with K -Means, Par2PK-Means, and ParCLARA, Par3PKM achieves higher efficiency, more accuracy, and better scalability and can effectively divide traffic subarea with big taxi trajectory data

Attentive Neural Architecture Incorporating Song Features For Music Recommendation

Recommender Systems are an integral part of music sharing platforms. Often the aim of these systems is to increase the time, the user spends on the platform and hence having a high commercial value. The systems which aim at increasing the average time a user spends on the platform often need to recommend songs which the user might want to listen to next at each point in time. This is different from recommendation systems which try to predict the item which might be of interest to the user at some point in the user lifetime but not necessarily in the very near future. Prediction of the next song the user might like requires some kind of modeling of the user interests at the given point of time. Attentive neural networks have been exploiting the sequence in which the items were selected by the user to model the implicit short-term interests of the user for the task of next item prediction, however we feel that the features of the songs occurring in the sequence could also convey some important information about the short-term user interest which only the items cannot. In this direction, we propose a novel attentive neural architecture which in addition to the sequence of items selected by the user, uses the features of these items to better learn the user short-term preferences and recommend the next song to the user.Comment: Accepted as a paper at the 12th ACM Conference on Recommender Systems (RecSys 18

Time-Frequency Fault Feature Extraction for Rolling Bearing Based on the Tensor Manifold Method

Rolling-bearing faults can be effectively reflected using time-frequency characteristics. However, there are inevitable interference and redundancy components in the conventional time-frequency characteristics. Therefore, it is critical to extract the sensitive parameters that reflect the rolling-bearing state from the time-frequency characteristics to accurately classify rolling-bearing faults. Thus, a new tensor manifold method is proposed. First, we apply the Hilbert-Huang transform (HHT) to rolling-bearing vibration signals to obtain the HHT time-frequency spectrum, which can be transformed into the HHT time-frequency energy histogram. Then, the tensor manifold time-frequency energy histogram is extracted from the traditional HHT time-frequency spectrum using the tensor manifold method. Five time-frequency characteristic parameters are defined to quantitatively depict the failure characteristics. Finally, the tensor manifold time-frequency characteristic parameters and probabilistic neural network (PNN) are combined to effectively classify the rolling-bearing failure samples. Engineering data are used to validate the proposed method. Compared with traditional HHT time-frequency characteristic parameters, the information redundancy of the time-frequency characteristics is greatly reduced using the tensor manifold time-frequency characteristic parameters and different rolling-bearing fault states are more effectively distinguished when combined with the PNN

Trust in Software Supply Chains: Blockchain-Enabled SBOM and the AIBOM Future

Software Bill of Materials (SBOM) serves as a critical pillar in ensuring software supply chain security by providing a detailed inventory of the components and dependencies integral to software development. However, challenges abound in the sharing of SBOMs, including potential data tampering, hesitation among software vendors to disclose comprehensive information, and bespoke requirements from software procurers or users. These obstacles have stifled widespread adoption and utilization of SBOMs, underscoring the need for a more secure and flexible mechanism for SBOM sharing. This study proposes a novel solution to these challenges by introducing a blockchain-empowered approach for SBOM sharing, leveraging verifiable credentials to allow for selective disclosure. This strategy not only heightens security but also offers flexibility. Furthermore, this paper broadens the remit of SBOM to encompass AI systems, thereby coining the term AI Bill of Materials (AIBOM). This extension is motivated by the rapid progression in AI technology and the escalating necessity to track the lineage and composition of AI software and systems. Particularly in the era of foundational models like large language models (LLMs), understanding their composition and dependencies becomes crucial. These models often serve as a base for further development, creating complex dependencies and paving the way for innovative AI applications. The evaluation of our solution indicates the feasibility and flexibility of the proposed SBOM sharing mechanism, positing a new solution for securing (AI) software supply chains

Right to be Forgotten in the Era of Large Language Models: Implications, Challenges, and Solutions

The Right to be Forgotten (RTBF) was first established as the result of the ruling of Google Spain SL, Google Inc. v AEPD, Mario Costeja Gonz\'alez, and was later included as the Right to Erasure under the General Data Protection Regulation (GDPR) of European Union to allow individuals the right to request personal data be deleted by organizations. Specifically for search engines, individuals can send requests to organizations to exclude their information from the query results. With the recent development of Large Language Models (LLMs) and their use in chatbots, LLM-enabled software systems have become popular. But they are not excluded from the RTBF. Compared with the indexing approach used by search engines, LLMs store, and process information in a completely different way. This poses new challenges for compliance with the RTBF. In this paper, we explore these challenges and provide our insights on how to implement technical solutions for the RTBF, including the use of machine unlearning, model editing, and prompting engineering

SeePrivacy: Automated Contextual Privacy Policy Generation for Mobile Applications

Privacy policies have become the most critical approach to safeguarding individuals' privacy and digital security. To enhance their presentation and readability, researchers propose the concept of contextual privacy policies (CPPs), aiming to fragment policies into shorter snippets and display them only in corresponding contexts. In this paper, we propose a novel multi-modal framework, namely SeePrivacy, designed to automatically generate contextual privacy policies for mobile apps. Our method synergistically combines mobile GUI understanding and privacy policy document analysis, yielding an impressive overall 83.6% coverage rate for privacy-related context detection and an accuracy of 0.92 in extracting corresponding policy segments. Remarkably, 96% of the retrieved policy segments can be correctly matched with their contexts. The user study shows SeePrivacy demonstrates excellent functionality and usability (4.5/5). Specifically, participants exhibit a greater willingness to read CPPs (4.1/5) compared to original privacy policies (2/5). Our solution effectively assists users in comprehending privacy notices, and this research establishes a solid foundation for further advancements and exploration

Comparing Palmer Drought Severity Index drought assessments using the traditional offline approach with direct climate model outputs

Anthropogenic warming has been projected to increase global drought for the 21st century when calculated using traditional offline drought indices. However, this contradicts observations of the overall global greening and little systematic change in runoff over the past few decades and climate projections of future greening with slight increases in global runoff for the coming century. This calls into question the drought projections based on traditional offline drought indices. Here we calculate a widely used traditional drought index (i.e., the Palmer Drought Severity Index, PDSI) using direct outputs from 16 Coupled Model Intercomparison Project Phase 5 (CMIP5) models (PDSI_CMIP5) such that the hydrologic consistency between PDSI_CMIP5 and CMIP5 models is maintained. We find that the PDSI_CMIP5-depicted drought increases (in terms of drought severity, frequency, and extent) are much smaller than that reported when PDSI is calculated using the traditional offline approach that has been widely used in previous drought assessments under climate change. Further analyses indicate that the overestimation of PDSI drought increases reported previously using the PDSI is primarily due to ignoring the vegetation response to elevated atmospheric CO2 concentration ([CO2]) in the traditional offline calculations. Finally, we show that the overestimation of drought using the traditional PDSI approach can be minimized by accounting for the effect of CO2 on evapotranspiration.This research has been supported by the National Natural Science Foundation of China (grant no. 41890821), the Qinghai Department of Science and Technology (grant no. 2019-SF-A4), the Ministry of Science and Technology of China (grant no. 2019YFC1510604), the Australian Research Council (grant no. CE170100023), and CSIRO Land and Water

Assessing the Steady-State Assumption in Water Balance Calculation Across Global Catchments

It has long been assumed that over a sufficiently long period of time, changes in catchment water storage (ΔS) are a relatively minor term compared to other fluxes and can be neglected in the catchment water balance equation. However, the validity of this fundamental assumption has rarely been tested, and the associated uncertainties in water balance calculations remain unknown. Here, we use long‐term (1982-2011) observations of monthly streamflow (Q) and precipitation (P) for 1,057 global unimpaired catchments, combined with four independent evapotranspiration (E) estimates to infer ΔS and to provide a global assessment of the steady‐state assumption in catchment water balance calculations. Results show that when the threshold for steady state is set to 5% of the mean monthly P, ~70% of the catchments attain steady state within 10 years while ~6% of the catchments fail to reach a steady state even after 30 years. The time needed for a catchment to reach steady state (τs) shows a close relationship with climatic aridity and vegetation coverage, with arid/semiarid and sparsely vegetated catchments generally having a longer τs. Additionally, increasing snowfall fraction also increases τs. The imbalance (ewb) caused by ignoring ΔS decreases as averaging period for water balance calculations increases as expected. For a typical 10‐year averaging period, ewb accounts for ~7% of P in arid, but that decreases to ~3% of P in humid catchments. These results suggest that catchment properties should be considered when applying the steady‐state assumption and call for caution when ignoring ΔS in arid/semiarid regions.This study is financially supported by the National Natural Science Foundation of China (Grant 41890821), the Ministry of Science and Technology of China (Grant 2019YFC1510604), the Qinghai Department of Science and Technology (Grant 2019‐SF‐A4), and the Guoqiang Institute of Tsinghua University (Grant 2019GQG1020). M. L. Roderick and T. R. McVicar acknowledge support from the Australian Research Council (CE170100023)