Lipschitz Adaptivity with Multiple Learning Rates in Online Learning

We aim to design adaptive online learning algorithms that take advantage of any special structure that might be present in the learning task at hand, with as little manual tuning by the user as possible. A fundamental obstacle that comes up in the design of such adaptive algorithms is to calibrate a so-called step-size or learning rate hyperparameter depending on variance, gradient norms, etc. A recent technique promises to overcome this difficulty by maintaining multiple learning rates in parallel. This technique has been applied in the MetaGrad algorithm for online convex optimization and the Squint algorithm for prediction with expert advice. However, in both cases the user still has to provide in advance a Lipschitz hyperparameter that bounds the norm of the gradients. Although this hyperparameter is typically not available in advance, tuning it correctly is crucial: if it is set too small, the methods may fail completely; but if it is taken too large, performance deteriorates significantly. In the present work we remove this Lipschitz hyperparameter by designing new versions of MetaGrad and Squint that adapt to its optimal value automatically. We achieve this by dynamically updating the set of active learning rates. For MetaGrad, we further improve the computational efficiency of handling constraints on the domain of prediction, and we remove the need to specify the number of rounds in advance.Comment: 22 pages. To appear in COLT 201

The roles of political skill and intrinsic motivation in performance prediction of adaptive selling

Previous studies have long recognized and examined adaptive selling behavior as an effective selling behavior in current selling situations. Although some studies assumed and revealed moderating factors that affect the effectiveness of adaptive selling behavior, few studies examined an individual’s skill as a moderator on this effect. This study focuses on political skill as a type of skill that has been recently found to have positive effects on sales performance. In addition, this study includes intrinsic motivation as an additional moderator that enables political skill to be invested for effective selling behavior. Our analysis of 249 salespeople and 145 supervisors in a matching sample largely supports our hypotheses that the positive effects of adaptive selling behavior on sales performance are the highest when both political skill and intrinsic motivation are high

Cultural differences in complex addition: efficient Chinese versus adaptive Belgians and Canadians

In the present study, the authors tested the effects of working-memory load on math problem solving in 3 different cultures: Flemish-speaking Belgians, English-speaking Canadians, and Chinese-speaking Chinese currently living in Canada. Participants solved complex addition problems (e.g., 58 + 76) in no-load and working-memory load conditions, in which either the central executive or the phonological loop was loaded. The authors used the choice/no-choice method to obtain unbiased measures of strategy selection and strategy efficiency. The Chinese participants were faster than the Belgians, who were faster and more accurate than the Canadians. The Chinese also required fewer working-memory resources than did the Belgians and Canadians. However, the Chinese chose less adaptively from the available strategies than did the Belgians and Canadians. These cultural differences in math problem solving are likely the result of different instructional approaches during elementary school (practice and training in Asian countries vs. exploration and flexibility in non-Asian countries), differences in the number language, and informal cultural norms and standards. The relevance of being adaptive is discussed as well as the implications of the results in regards to the strategy choice and discovery simulation model of strategy selection (J. Shrager & R. S. Siegler, 1998)

Adaptive Transactional Memories: Performance and Energy Consumption Tradeoffs

Energy efficiency is becoming a pressing issue, especially in large data centers where it entails, at the same time, a non-negligible management cost, an enhancement of hardware fault probability, and a significant environmental footprint. In this paper, we study how Software Transactional Memories (STM) can provide benefits on both power saving and the overall applications’ execution performance. This is related to the fact that encapsulating shared-data accesses within transactions gives the freedom to the STM middleware to both ensure consistency and reduce the actual data contention, the latter having been shown to affect the overall power needed to complete the application’s execution. We have selected a set of self-adaptive extensions to existing STM middlewares (namely, TinySTM and R-STM) to prove how self-adapting computation can capture the actual degree of parallelism and/or logical contention on shared data in a better way, enhancing even more the intrinsic benefits provided by STM. Of course, this benefit comes at a cost, which is the actual execution time required by the proposed approaches to precisely tune the execution parameters for reducing power consumption and enhancing execution performance. Nevertheless, the results hereby provided show that adaptivity is a strictly necessary requirement to reduce energy consumption in STM systems: Without it, it is not possible to reach any acceptable level of energy efficiency at all

Basic Filters for Convolutional Neural Networks Applied to Music: Training or Design?

When convolutional neural networks are used to tackle learning problems based on music or, more generally, time series data, raw one-dimensional data are commonly pre-processed to obtain spectrogram or mel-spectrogram coefficients, which are then used as input to the actual neural network. In this contribution, we investigate, both theoretically and experimentally, the influence of this pre-processing step on the network's performance and pose the question, whether replacing it by applying adaptive or learned filters directly to the raw data, can improve learning success. The theoretical results show that approximately reproducing mel-spectrogram coefficients by applying adaptive filters and subsequent time-averaging is in principle possible. We also conducted extensive experimental work on the task of singing voice detection in music. The results of these experiments show that for classification based on Convolutional Neural Networks the features obtained from adaptive filter banks followed by time-averaging perform better than the canonical Fourier-transform-based mel-spectrogram coefficients. Alternative adaptive approaches with center frequencies or time-averaging lengths learned from training data perform equally well.Comment: Completely revised version; 21 pages, 4 figure