DeepRebirth: Accelerating Deep Neural Network Execution on Mobile Devices

Deploying deep neural networks on mobile devices is a challenging task. Current model compression methods such as matrix decomposition effectively reduce the deployed model size, but still cannot satisfy real-time processing requirement. This paper first discovers that the major obstacle is the excessive execution time of non-tensor layers such as pooling and normalization without tensor-like trainable parameters. This motivates us to design a novel acceleration framework: DeepRebirth through "slimming" existing consecutive and parallel non-tensor and tensor layers. The layer slimming is executed at different substructures: (a) streamline slimming by merging the consecutive non-tensor and tensor layer vertically; (b) branch slimming by merging non-tensor and tensor branches horizontally. The proposed optimization operations significantly accelerate the model execution and also greatly reduce the run-time memory cost since the slimmed model architecture contains less hidden layers. To maximally avoid accuracy loss, the parameters in new generated layers are learned with layer-wise fine-tuning based on both theoretical analysis and empirical verification. As observed in the experiment, DeepRebirth achieves more than 3x speed-up and 2.5x run-time memory saving on GoogLeNet with only 0.4% drop of top-5 accuracy on ImageNet. Furthermore, by combining with other model compression techniques, DeepRebirth offers an average of 65ms inference time on the CPU of Samsung Galaxy S6 with 86.5% top-5 accuracy, 14% faster than SqueezeNet which only has a top-5 accuracy of 80.5%.Comment: AAAI 201

Learning Personalized User Preference from Cold Start in Multi-turn Conversations

This paper presents a novel teachable conversation interaction system that is capable of learning users preferences from cold start by gradually adapting to personal preferences. In particular, the TAI system is able to automatically identify and label user preference in live interactions, manage dialogue flows for interactive teaching sessions, and reuse learned preference for preference elicitation. We develop the TAI system by leveraging BERT encoder models to encode both dialogue and relevant context information, and build action prediction (AP), argument filling (AF) and named entity recognition (NER) models to understand the teaching session. We adopt a seeker-provider interaction loop mechanism to generate diverse dialogues from cold-start. TAI is capable of learning user preference, which achieves 0.9122 turn level accuracy on out-of-sample dataset, and has been successfully adopted in production.Comment: preference, personalization, cold-start, dialogue, LLM. embeddin

Do not Waste Money on Advertising Spend: Bid Recommendation via Concavity Changes

In computational advertising, a challenging problem is how to recommend the bid for advertisers to achieve the best return on investment (ROI) given budget constraint. This paper presents a bid recommendation scenario that discovers the concavity changes in click prediction curves. The recommended bid is derived based on the turning point from significant increase (i.e. concave downward) to slow increase (convex upward). Parametric learning based method is applied by solving the corresponding constraint optimization problem. Empirical studies on real-world advertising scenarios clearly demonstrate the performance gains for business metrics (including revenue increase, click increase and advertiser ROI increase).Comment: 10 page

Demystifying Advertising Campaign Bid Recommendation: A Constraint target CPA Goal Optimization

In cost-per-click (CPC) or cost-per-impression (CPM) advertising campaigns, advertisers always run the risk of spending the budget without getting enough conversions. Moreover, the bidding on advertising inventory has few connections with propensity one that can reach to target cost-per-acquisition (tCPA) goals. To address this problem, this paper presents a bid optimization scenario to achieve the desired tCPA goals for advertisers. In particular, we build the optimization engine to make a decision by solving the rigorously formalized constrained optimization problem, which leverages the bid landscape model learned from rich historical auction data using non-parametric learning. The proposed model can naturally recommend the bid that meets the advertisers' expectations by making inference over advertisers' historical auction behaviors, which essentially deals with the data challenges commonly faced by bid landscape modeling: incomplete logs in auctions, and uncertainty due to the variation and fluctuations in advertising bidding behaviors. The bid optimization model outperforms the baseline methods on real-world campaigns, and has been applied into a wide range of scenarios for performance improvement and revenue liftup

Personalized Search Via Neural Contextual Semantic Relevance Ranking

Existing neural relevance models do not give enough consideration for query and item context information which diversifies the search results to adapt for personal preference. To bridge this gap, this paper presents a neural learning framework to personalize document ranking results by leveraging the signals to capture how the document fits into users' context. In particular, it models the relationships between document content and user query context using both lexical representations and semantic embeddings such that the user's intent can be better understood by data enrichment of personalized query context information. Extensive experiments performed on the search dataset, demonstrate the effectiveness of the proposed method.Comment: Contextual, Personalization, Search, Semantics, LLM, embeddin

STREET: A Multi-Task Structured Reasoning and Explanation Benchmark

We introduce STREET, a unified multi-task and multi-domain natural language reasoning and explanation benchmark. Unlike most existing question-answering (QA) datasets, we expect models to not only answer questions, but also produce step-by-step structured explanations describing how premises in the question are used to produce intermediate conclusions that can prove the correctness of a certain answer. We perform extensive evaluation with popular language models such as few-shot prompting GPT-3 and fine-tuned T5. We find that these models still lag behind human performance when producing such structured reasoning steps. We believe this work will provide a way for the community to better train and test systems on multi-step reasoning and explanations in natural language.Comment: Published in ICLR 202

The gap in injury mortality rates between urban and rural residents of Hubei province, China

<p>Abstract</p> <p>Background</p> <p>Injury is a growing public health concern in China. Injury death rates are often higher in rural areas than in urban areas in general. The objective of this study is to compare the injury mortality rates in urban and rural residents in Hubei Province in central China by age, sex and mechanism of injury.</p> <p>Methods</p> <p>Using data from the Disease Surveillance Points (DSP) system maintained by the Hubei Province Centers for Disease Control and Prevention (CDC) from 2006 to 2008, injury deaths were classified according to the International Classification of Disease-10^thRevision (ICD-10). Crude and age-adjusted annual mortality rates were calculated for rural and urban residents of Hubei Province.</p> <p>Results</p> <p>The crude and age-adjusted injury death rates were significantly higher for rural residents than for urban residents (crude rate ratio 1.9, 95% confidence interval 1.8-2.0; adjusted rate ratio 2.4, 95% confidence interval 2.3-2.4). The age-adjusted injury death rate for males was 81.6/100,000 in rural areas compared with 37.0/100 000 in urban areas; for females, the respective rates were 57.9/100,000 and 22.4/100 000. Death rates for suicide (32.4 per 100 000 vs 3.9 per 100 000), traffic-related injuries (15.8 per 100 000 vs 9.5 per 100 000), drowning (6.9 per 100 000 vs 2.3 per 100 000) and crushing injuries (2.0 per 100 000 vs 0.7 per 100 000) were significantly higher in rural areas. Overall injury death rates were much higher in persons over 65 years, with significantly higher rates in rural residents compared with urban residents for suicide (279.8 per 100 000 vs 10.7 per 100 000), traffic-related injuries, and drownings in this age group. Death rates for falls, poisoning, and suffocation were similar in the two geographic groups.</p> <p>Conclusions</p> <p>Rates of suicide, traffic-related injury deaths and drownings are demonstrably higher in rural compared with urban locations and should be targeted for injury prevention activity. There is a need for injury prevention policies targeted at elderly residents, especially with regard to suicide prevention in rural areas in Central China.</p