Real-Time Bidding with Multi-Agent Reinforcement Learning in Display Advertising

Real-time advertising allows advertisers to bid for each impression for a visiting user. To optimize specific goals such as maximizing revenue and return on investment (ROI) led by ad placements, advertisers not only need to estimate the relevance between the ads and user's interests, but most importantly require a strategic response with respect to other advertisers bidding in the market. In this paper, we formulate bidding optimization with multi-agent reinforcement learning. To deal with a large number of advertisers, we propose a clustering method and assign each cluster with a strategic bidding agent. A practical Distributed Coordinated Multi-Agent Bidding (DCMAB) has been proposed and implemented to balance the tradeoff between the competition and cooperation among advertisers. The empirical study on our industry-scaled real-world data has demonstrated the effectiveness of our methods. Our results show cluster-based bidding would largely outperform single-agent and bandit approaches, and the coordinated bidding achieves better overall objectives than purely self-interested bidding agents

ASP: Automatic Selection of Proxy dataset for efficient AutoML

Deep neural networks have gained great success due to the increasing amounts of data, and diverse effective neural network designs. However, it also brings a heavy computing burden as the amount of training data is proportional to the training time. In addition, a well-behaved model requires repeated trials of different structure designs and hyper-parameters, which may take a large amount of time even with state-of-the-art (SOTA) hyper-parameter optimization (HPO) algorithms and neural architecture search (NAS) algorithms. In this paper, we propose an Automatic Selection of Proxy dataset framework (ASP) aimed to dynamically find the informative proxy subsets of training data at each epoch, reducing the training data size as well as saving the AutoML processing time. We verify the effectiveness and generalization of ASP on CIFAR10, CIFAR100, ImageNet16-120, and ImageNet-1k, across various public model benchmarks. The experiment results show that ASP can obtain better results than other data selection methods at all selection ratios. ASP can also enable much more efficient AutoML processing with a speedup of 2x-20x while obtaining better architectures and better hyper-parameters compared to utilizing the entire dataset.Comment: This paper was actually finished in 202

More Than Just Statics: Static and Temporal Dynamic Changes in Intrinsic Brain Activity in Unilateral Temporal Lobe Epilepsy

BACKGROUND: Temporal lobe epilepsy (TLE) is the most prevalent refractory focal epilepsy and is more likely accompanied by cognitive impairment. The fully understanding of the neuronal activity underlying TLE is of great significance. OBJECTIVE: This study aimed to comprehensively explore the potential brain activity abnormalities affected by TLE and detect whether the changes were associated with cognition. METHODS: Six static intrinsic brain activity (IBA) indicators [amplitude of low-frequency fluctuation (ALFF), fractional ALFF (fALFF), regional homogeneity (ReHo), degree centrality (DC), global signal correlation (GSCorr), and voxel-mirrored homotopic connectivity (VMHC)] and their corresponding dynamic indicators, such as dynamic ALFF (dALFF), dynamic fALFF (dfALFF), dynamic ReHo (dReHo), dynamic DC (dDC), dynamic VMHC (dVMHC), and dynamic GSCorr (dGSCorr), in 57 patients with unilateral TLE and 42 healthy volunteers were compared. Correlation analyses were also performed between these indicators in areas displaying group differences and cognitive function, epilepsy duration, and severity. RESULTS: Marked overlap was present among the abnormal brain regions detected using various static and dynamic indicators, primarily including increased ALFF/dALFF/fALFF in the bilateral medial temporal lobe and thalamus, decreased ALFF/dALFF/fALFF in the frontal lobe contralateral to the epileptogenic side, decreased fALFF, ReHo, dReHo, DC, dDC, GSCorr, dGSCorr, and VMHC in the temporal neocortex ipsilateral to the epileptogenic foci, decreased dReHo, dDC, dGSCorr, and dVMHC in the occipital lobe, and increased ALFF, fALFF, dfALFF, ReHo, and DC in the supplementary motor area ipsilateral to the epileptogenic foci. Furthermore, most IBA indicators in the abnormal brain region significantly correlated with the duration of epilepsy and several cognitive scale scores ( CONCLUSION: The combined application of static and dynamic IBA indicators could comprehensively reveal more real abnormal neuronal activity and the impairment and compensatory mechanisms of cognitive function in TLE. Moreover, it might help in the lateralization of epileptogenic foci and exploration of the transmission and inhibition pathways of epileptic activity

Alterations in Static and Dynamic Regional Homogeneity in Mesial Temporal Lobe Epilepsy With and Without Initial Precipitating Injury

Objectives Initial precipitating injury (IPI) such as febrile convulsion and intracranial infection will increase the susceptibility to epilepsy. It is still unknown if the functional deficits differ between mesial temporal lobe epilepsy with IPI (mTLE-IPI) and without IPI (mTLE-NO). Methods We recruited 25 mTLE-IPI patients, 35 mTLE-NO patients and 33 healthy controls (HC). Static regional homogeneity (sReHo) and dynamic regional homogeneity (dReHo) were then adopted to estimate the alterations of local neuronal activity. One-way analysis of variance was used to analyze the differences between the three groups in sReHo and dReHo. Then the results were utilized as masks for further between-group comparisons. Besides, correlation analyses were carried out to detect the potential relationships between abnormal regional homogeneity indicators and clinical characteristics. Results When compared with HC, the bilateral thalamus and the visual cortex in mTLE-IPI patients showed an increase in both sReHo and variability of dReHo. Besides, mTLE-IPI patients exhibited decreased sReHo in the right cerebellum crus1/crus2, inferior parietal lobule and temporal neocortex. mTLE-NO patients showed decreased sReHo and variability of dReHo in the bilateral temporal neocortex compared with HC. Increased sReHo and variability of dReHo were found in the bilateral visual cortex when mTLE-IPI patients was compared with mTLE-NO patients, as well as increased variability of dReHo in the left thalamus and decreased sReHo in the right dorsolateral prefrontal cortex. Additionally, we discovered a negative correlation between the national hospital seizure severity scale testing score and sReHo in the right cerebellum crus1 in mTLE-IPI patients. Conclusion According to the aforementioned findings, both mTLE-IPI and mTLE-NO patients had significant anomalies in local neuronal activity, although the functional deficits were much severer in mTLE-IPI patients. The use of sReHo and dReHo may provide a novel insight into the impact of the presence of IPI on the development of mTLE

Unified Language-Vision Pretraining in LLM with Dynamic Discrete Visual Tokenization

Recently, the remarkable advance of the Large Language Model (LLM) has inspired researchers to transfer its extraordinary reasoning capability to both vision and language data. However, the prevailing approaches primarily regard the visual input as a prompt and focus exclusively on optimizing the text generation process conditioned upon vision content by a frozen LLM. Such an inequitable treatment of vision and language heavily constrains the model's potential. In this paper, we break through this limitation by representing both vision and language in a unified form. Specifically, we introduce a well-designed visual tokenizer to translate the non-linguistic image into a sequence of discrete tokens like a foreign language that LLM can read. The resulting visual tokens encompass high-level semantics worthy of a word and also support dynamic sequence length varying from the image. Coped with this tokenizer, the presented foundation model called LaVIT can handle both image and text indiscriminately under the same generative learning paradigm. This unification empowers LaVIT to serve as an impressive generalist interface to understand and generate multi-modal content simultaneously. Extensive experiments further showcase that it outperforms the existing models by a large margin on massive vision-language tasks. Our code and models will be available at https://github.com/jy0205/LaVIT

KwaiYiiMath: Technical Report

Recent advancements in large language models (LLMs) have demonstrated remarkable abilities in handling a variety of natural language processing (NLP) downstream tasks, even on mathematical tasks requiring multi-step reasoning. In this report, we introduce the KwaiYiiMath which enhances the mathematical reasoning abilities of KwaiYiiBase1, by applying Supervised Fine-Tuning (SFT) and Reinforced Learning from Human Feedback (RLHF), including on both English and Chinese mathematical tasks. Meanwhile, we also constructed a small-scale Chinese primary school mathematics test set (named KMath), consisting of 188 examples to evaluate the correctness of the problem-solving process generated by the models. Empirical studies demonstrate that KwaiYiiMath can achieve state-of-the-art (SOTA) performance on GSM8k, CMath, and KMath compared with the similar size models, respectively.Comment: technical report. arXiv admin note: text overlap with arXiv:2306.16636 by other author

FogChain: a blockchain-based Peer-to-Peer solar power trading system powered by fog AI

Microgrids, gaining traction from rising distributed generation for carbon reduction, demand novel solutions to regulate on-and off-grid operations, as well as both energy and monetary transfers between the microgrid and the central grid and among different microgrid participants. This research aims to develop and validate an intelligent microgrid management system to secure the competitiveness of Singapore's energy market, by leveraging the inherent synergy between two emerging technologies, i.e., blockchain for Peer-to-Peer (P2P) solar power trading and fog computing for grid infrastructure management. For this vision, we have developed FogChain, an integrative, cost-effective, and scalable microgrid operating system (MGOS), consisting of three technical service layers: 1) a novel microgrid information infrastructure based on the fog-computing paradigm (i.e., intelligence on edge); 2) a blockchain-based microgrid service layer, providing smart contract and decentralized control capabilities for grid application development; and 3) a microgrid application layer (i.e., P2P energy trading) over the blockchain-based grid service. This MGOS would fundamentally transform how solar power is traded among participating electricity prosumers, leading to potentially new operational and business models. We have implemented the FogChain system and conducted extensive experiments to verify its performance advantages. Our results demonstrate that FogChain can efficiently process energy auction among 1000 participants with 1.1 s delay on average, reduce transmission cost up to 20% under the loss-aware trading mechanism, and reduce the solar yield prediction error to 0.11. Our system prototype suggests that FogChain provides a promising solution for efficient decentralized energy trading and intelligent distributed control for microgrids.Energy Market Authority (EMA)Ministry of Education (MOE)National Research Foundation (NRF)This work was supported in part by the National Research Foundation, Singapore, and the Energy Market Authority through Energy Programme under EP Award NRF2017EWT-EP003-023, and in part by the MOE through Tier-1 Grant Call under Award RG96/20