Deciding Fast and Slow in Risk Decision Making: An Experimental Study

The current study presents findings of an experiment. Response time was used to investigate fast deciders’ (FD) and slow deciders’ (SD) behavioral differences. SDs were found to be more cognitive than FDs and this could induce an increase in average response time. Both FDs and SDs showed aversion to extreme options, but they behaved differently with option ‘S’ being “safer” among groups. Moreover, FDs responded more instinctively to negative feedbacks

ONCHIP TRAINING OF SPIKING NEURAL ACCELERATORS USING SPIKE-TRAIN LEVEL DIRECT FEEDBACK ALIGNMENT

Spiking Neural Networks (SNNs) are widely researched in recent years and present a promising computing model. Several key properties including biologically plausible information processing and event driven sample learning make SNNs be able for ultra-low power neuromorphic hardware implementation. However, to achieve the same level of performance in training conventional deep artificial neural networks (ANNs), especially for networks with error backpropagation (BP) algorithm, is a significant challenge existing in SNNs training, which is due to inherent complex dynamics and non-differentiable spike activities of spiking neurons. To solve this problem, this thesis proposes the first study on realizing competitive spike-train level backpropagation (BP) like algorithms to enable on-chip BP training of SNNs. This novel alrogithm, called spike-train level direct feedback alignment (ST-DFA), performs better in computation complexity and training latency compared to traditional BP methods. Furthermore, algorithm and hardware cooptimization as well as efficient online neural signal computation are explored for on-chip implementation of ST-DFA. To figure out the performance of this proposed algorithm, the final online version of ST-DFA is tested on the Xilinx ZC706 FPGA board. During testing on real-world speech and image classification applications, it shows excellent performance vs. overhead tradeoffs. SNN neural processors with on-chip ST-DFA training show competitive classification accuracy of 97.23% for the MNIST dataset with 4X input resolution reduction and 87.40% for the challenging 16-speaker TI46 speech corpus, respectively. This experimental result is then compared to the hardware implementation of the state-of-the-art BP algorithm HM2-BP. While trading off classification performance very gracefully, the design of the proposed online ST-DFA training reduces functional resources by 76.7% and backward training latency by 31.6%, which dramatically cut resource and power demand for hardware implementatio

Resource levelling in repetitive construction projects with interruptions: an integrated approach

Despite the significance of resource levelling, project managers lack various ways to smooth resource usage fluctuation of a repetitive construction project besides changing resource usage. Tolerating interruptions is an effective way to provide flexibility for a schedule but is ignored when solving resource levelling problems. Therefore, this paper investigates the impacts of interruptions on resource usage fluctuation and develops an integrated approach that simultaneously integrates two scheduling adjusting processes: changing resource usage and tolerating interruptions. In this paper, two interruption conditions are proposed to identify which activities are suitable to be interrupted for smoothing resource usage fluctuation. The traditional resource levelling model is modified to a new scheduling model by incorporating interruptions. A two-stage GA-based scheduling algorithm is developed by integrating changing resource usage and tolerating interruptions. A commonly used pipeline project is adopted to illustrate the steps of the proposed approach and demonstrate its effectiveness and superiority through comparison with previous studies. A large-scale project further verifies the usability of the proposed approach. The results confirmed the feasibility to smooth resource usage fluctuation by interruptions, and the integrated approach can achieve a more competitive resource levelling result

BioGPT: Generative Pre-trained Transformer for Biomedical Text Generation and Mining

Pre-trained language models have attracted increasing attention in the biomedical domain, inspired by their great success in the general natural language domain. Among the two main branches of pre-trained language models in the general language domain, i.e., BERT (and its variants) and GPT (and its variants), the first one has been extensively studied in the biomedical domain, such as BioBERT and PubMedBERT. While they have achieved great success on a variety of discriminative downstream biomedical tasks, the lack of generation ability constrains their application scope. In this paper, we propose BioGPT, a domain-specific generative Transformer language model pre-trained on large scale biomedical literature. We evaluate BioGPT on six biomedical NLP tasks and demonstrate that our model outperforms previous models on most tasks. Especially, we get 44.98%, 38.42% and 40.76% F1 score on BC5CDR, KD-DTI and DDI end-to-end relation extraction tasks respectively, and 78.2% accuracy on PubMedQA, creating a new record. Our case study on text generation further demonstrates the advantage of BioGPT on biomedical literature to generate fluent descriptions for biomedical terms. Code is available at https://github.com/microsoft/BioGPT.Comment: Published at Briefings in Bioinformatics. Code is available at https://github.com/microsoft/BioGP

Can Generalist Foundation Models Outcompete Special-Purpose Tuning? Case Study in Medicine

Generalist foundation models such as GPT-4 have displayed surprising capabilities in a wide variety of domains and tasks. Yet, there is a prevalent assumption that they cannot match specialist capabilities of fine-tuned models. For example, most explorations to date on medical competency benchmarks have leveraged domain-specific training, as exemplified by efforts on BioGPT and Med-PaLM. We build on a prior study of GPT-4's capabilities on medical challenge benchmarks in the absence of special training. Rather than using simple prompting to highlight the model's out-of-the-box capabilities, we perform a systematic exploration of prompt engineering. We find that prompting innovation can unlock deeper specialist capabilities and show that GPT-4 easily tops prior leading results for medical benchmarks. The prompting methods we explore are general purpose, and make no specific use of domain expertise, removing the need for expert-curated content. Our experimental design carefully controls for overfitting during the prompt engineering process. We introduce Medprompt, based on a composition of several prompting strategies. With Medprompt, GPT-4 achieves state-of-the-art results on all nine of the benchmark datasets in the MultiMedQA suite. The method outperforms leading specialist models such as Med-PaLM 2 by a significant margin with an order of magnitude fewer calls to the model. Steering GPT-4 with Medprompt achieves a 27% reduction in error rate on the MedQA dataset over the best methods to date achieved with specialist models and surpasses a score of 90% for the first time. Beyond medical problems, we show the power of Medprompt to generalize to other domains and provide evidence for the broad applicability of the approach via studies of the strategy on exams in electrical engineering, machine learning, philosophy, accounting, law, nursing, and clinical psychology.Comment: 21 pages, 7 figure

A variable neighborhood search with an effective local search for uncapacitated multilevel lot-sizing problems

In this study, we improved the variable neighborhood search (VNS) algorithm for solving uncapacitated multilevel lot-sizing (MLLS) problems. The improvement is two-fold. First, we developed an effective local search method known as the Ancestors Depth-first Traversal Search (ADTS), which can be embedded in the VNS to significantly improve the solution quality. Second, we proposed a common and efficient approach for the rapid calculation of the cost change for the VNS and other generate-and-test algorithms. The new VNS algorithm was tested against 176 benchmark problems of different scales (small, medium, and large). The experimental results show that the new VNS algorithm outperforms all of the existing algorithms in the literature for solving uncapacitated MLLS problems because it was able to find all optimal solutions (100%) for 96 small-sized problems and new best-known solutions for 5 of 40 medium-sized problems and for 30 of 40 large-sized problems

ONCHIP TRAINING OF SPIKING NEURAL ACCELERATORS USING SPIKE-TRAIN LEVEL DIRECT FEEDBACK ALIGNMENT

Spiking Neural Networks (SNNs) are widely researched in recent years and present a promising computing model. Several key properties including biologically plausible information processing and event driven sample learning make SNNs be able for ultra-low power neuromorphic hardware implementation. However, to achieve the same level of performance in training conventional deep artificial neural networks (ANNs), especially for networks with error backpropagation (BP) algorithm, is a significant challenge existing in SNNs training, which is due to inherent complex dynamics and non-differentiable spike activities of spiking neurons. To solve this problem, this thesis proposes the first study on realizing competitive spike-train level backpropagation (BP) like algorithms to enable on-chip BP training of SNNs. This novel alrogithm, called spike-train level direct feedback alignment (ST-DFA), performs better in computation complexity and training latency compared to traditional BP methods. Furthermore, algorithm and hardware cooptimization as well as efficient online neural signal computation are explored for on-chip implementation of ST-DFA. To figure out the performance of this proposed algorithm, the final online version of ST-DFA is tested on the Xilinx ZC706 FPGA board. During testing on real-world speech and image classification applications, it shows excellent performance vs. overhead tradeoffs. SNN neural processors with on-chip ST-DFA training show competitive classification accuracy of 97.23% for the MNIST dataset with 4X input resolution reduction and 87.40% for the challenging 16-speaker TI46 speech corpus, respectively. This experimental result is then compared to the hardware implementation of the state-of-the-art BP algorithm HM2-BP. While trading off classification performance very gracefully, the design of the proposed online ST-DFA training reduces functional resources by 76.7% and backward training latency by 31.6%, which dramatically cut resource and power demand for hardware implementatio

Integrated Production-Delivery Lot Sizing Model with Limited Production Capacity and Transportation Cost considering Overtime Work and Maintenance Time

An extension of the integrated production-delivery lot sizing model with limited production capacity and transportation cost is investigated. We introduce the factor of overtime work into the model to improve the manufacturer’s production. In addition, when finishing a lot, the manufacturer has maintenance time to maintain and repair equipment for ensuring that the supply chain is operating continuously. By analyzing the integrated model, the solution procedure is provided to determine the optimal delivery and order policy. We conduct a numerical experiment and give sensitive analysis by varying some parameters to illustrate the problem and its solution procedure