VLSI Implementation of Deep Neural Network Using Integral Stochastic Computing

The hardware implementation of deep neural networks (DNNs) has recently received tremendous attention: many applications in fact require high-speed operations that suit a hardware implementation. However, numerous elements and complex interconnections are usually required, leading to a large area occupation and copious power consumption. Stochastic computing has shown promising results for low-power area-efficient hardware implementations, even though existing stochastic algorithms require long streams that cause long latencies. In this paper, we propose an integer form of stochastic computation and introduce some elementary circuits. We then propose an efficient implementation of a DNN based on integral stochastic computing. The proposed architecture has been implemented on a Virtex7 FPGA, resulting in 45% and 62% average reductions in area and latency compared to the best reported architecture in literature. We also synthesize the circuits in a 65 nm CMOS technology and we show that the proposed integral stochastic architecture results in up to 21% reduction in energy consumption compared to the binary radix implementation at the same misclassification rate. Due to fault-tolerant nature of stochastic architectures, we also consider a quasi-synchronous implementation which yields 33% reduction in energy consumption w.r.t. the binary radix implementation without any compromise on performance.Comment: 11 pages, 12 figure

Dynamic stochastic joint expansion planning of power systems, natural gas networks, and electrical and natural gas storage

Over the last decades, electricity generation from natural gas has substantially increased, mostly driven by low natural gas prices due to fracturing and lower extraction costs. The geographic distance between natural gas resources and load centers calls for a holistic tool for joint expansion of power systems and natural gas networks. In this paper, a Dynamic Stochastic Joint Expansion Planning (DSJEP) of power systems and natural gas networks is proposed to minimize the investment and operational costs of power and natural gas systems. Electrical and natural gas storage (ENGS) are considered as an option for decision-makers in the DSJEP problem. The proposed approach takes into account long-term uncertainties in natural gas prices and electric and natural gas demands through scenario realizations. In dynamic planning, more scenario needs more time for computation; therefore, scenario reduction is implemented to eschew unnecessary scenarios. The proposed formulation is implemented on a four-bus electricity system with a five-node natural gas network. To demonstrate the efficiency and scalability of the proposed approach, it is also tested on the IEEE 118-bus system with a 14-node natural gas network. The numerical results demonstrate that ENGS can reduce the total investment cost, up to 52% in the test cases, and operational cost, up to 3%. In this paper, co-planning of power and natural gas systems considering natural gas and electrical storage is represented. Also, electrical and natural gas load growth uncertainties are taken into account to model the real situations. The purpose of the model is to minimize investing and operational costs

SlimFit: Memory-Efficient Fine-Tuning of Transformer-based Models Using Training Dynamics

Transformer-based models, such as BERT and ViT, have achieved state-of-the-art results across different natural language processing (NLP) and computer vision (CV) tasks. However, these models are extremely memory intensive during their fine-tuning process, making them difficult to deploy on GPUs with limited memory resources. To address this issue, we introduce a new tool called SlimFit that reduces the memory requirements of these models by dynamically analyzing their training dynamics and freezing less-contributory layers during fine-tuning. The layers to freeze are chosen using a runtime inter-layer scheduling algorithm. SlimFit adopts quantization and pruning for particular layers to balance the load of dynamic activations and to minimize the memory footprint of static activations, where static activations refer to those that cannot be discarded regardless of freezing. This allows SlimFit to freeze up to 95% of layers and reduce the overall on-device GPU memory usage of transformer-based models such as ViT and BERT by an average of 2.2x, across different NLP and CV benchmarks/datasets such as GLUE, SQuAD 2.0, CIFAR-10, CIFAR-100 and ImageNet with an average degradation of 0.2% in accuracy. For such NLP and CV tasks, SlimFit can reduce up to 3.1x the total on-device memory usage with an accuracy degradation of only up to 0.4%. As a result, while fine-tuning of ViT on ImageNet and BERT on SQuAD 2.0 with a batch size of 128 requires 3 and 2 32GB GPUs respectively, SlimFit enables their fine-tuning on a single 32GB GPU without any significant accuracy degradation

Isolation and identification α-Naphthol-degrading bacteria from oil-contaminated soils of Masjed-e-Soleyman

Introduction: α-Naphthol is a two-ring aromatic hydrocarbon that is a toxic compound for all  organisms in different ecosystems. Bioremediation technology for remediating PAH-contaminated sites has been proposed to be an efficient, economical and versatile alternative compared with physicochemical methods. Materials and Methods: In this study, the basal salt medium was prepared and then α-Naphthol was added with the concentration of 100ppm. α-Naphthol was a sole source of carbon for the growth of bacteria. Eventually, the medium was inoculated with the soil samples collected from polluted region and incubated for six weeks. Bacteria were isolated using double-layer BSM-agar containing the α-naphthol concentration of 200ppm. The degradation rate of α-naphthol and the other PAHs were determined using HPLC and the effective isolate was finally identified using biochemical and ‏molecular methods. Results: In this study, two isolates (N1 and N2) were isolated that were able to utilize α-Naphthol as a sole source of carbon. The isolate N1 could degrade α-naph‏thol by 80.5%. Moreover, it was effectively able to degrade the other PAHs than the isolate N2, therefore, it was selected as an efficient isolate. The isolate N1 was identified as Psuedomonas putida UW4 with respect to its 16S rDNA sequence and using biochemical tests. Discussion and conclusion: The isolate N1 could degrade α-naph‏thol by 80.5% from BSM medium at 30° C, pH 7.0 and the α-naph‏thol concentration of 100ppm in fifteen days of incub‏a‏tion.  According to the results, the isolate N1 can remove a large amount of the α-naph‏thol from BSM medium under the mentioned conditions and it is possible that under a similar situation the isolate N1 can remove a large amount of α-naph‏thol