290 research outputs found
Techniques d'ingénierie de trafic dynamique pour l'internet
Network convergence and new applications running on end-hosts result in increasingly variable and unpredictable traffic patterns. By providing origin-destination pairs with several possible paths, Dynamic Load-Balancing (DLB) has proved itself an excellent tool to face this uncertainty. The objective in DLB is to distribute traffic among these paths in real-time so that a certain objective function is optimized. In these dynamic schemes, paths are established a priori and the amount of traffic sent through each of them depends on the current traffic demand and network condition. In this thesis we study and propose various DLB mechanisms, differing in two important aspects. The first difference resides in the assumption, or not, that resources are reserved for each path. The second lies on the objective function, which clearly dictates the performance obtained from the network. However, a performance benchmarking of the possible choices has not been carried out so far. In this sense, for the case in which no reservations are performed, we study and compare several objective functions, including a proposal of ours. We will also propose and study a new distributed algorithm to attain the optimum of these objective functions. Its advantage with respect to previous proposals is its complete self-configuration (i. E. Convergence is guaranteed without any parametrization). Finally, we present the first complete comparative study between DLB and Robust Routing (a fixed routing configuration for all possible traffic demands). In particular, we analyze which scheme is more convenient in each given situation, and highlight some of their respective shortcomings and virtues.Avec la multiplication des services dans un même réseau et les diversités des applications utilisées par les usagers finaux, le trafic transporté est devenu très complexe et dynamique. Le Partage de la Charge Dynamique (PCD) constitue une alternative intéressante pour résoudre cette problématique. Si une paire Source-Destination est connectée par plusieurs chemins, le problème est le suivant : comment distribuer le trafic parmi ces chemins de telle façon qu’une fonction objective soit optimisé. Dans ce cas les chemins sont fixés a priori et la quantité de trafic acheminée sur chaque route est déterminée dynamiquement en fonction de la demande de trafic et de la situation actuelle du réseau. Dans cette thèse nous étudions puis nous proposons plusieurs mécanismes de PCD. Tout d'abord, nous distinguons deux types d’architecture : celles dans lesquelles les ressources sont réservées pour chaque chemin, et celles pour lesquelles aucune réservation n'est effectuée. La simplification faite dans le premier type d’architecture nous permet de proposer l'utilisation d'un nouveau mécanisme pour gérer les chemins. Partant de ce mécanisme, nous définissons un nouvel algorithme de PCD. Concernant la deuxième architecture, nous étudions et comparons plusieurs fonctions objectives. À partir de notre étude, nous proposons un nouvel algorithme distribué permettant d’atteindre l'optimum de ces fonctions objectives. La principale caractéristique de notre algorithme, et son avantage par rapport aux propositions antérieures, est sa capacité d'auto-configuration, dans la mesure où la convergence de l'algorithme est garantie sans aucun besoin de réglage préalable de ses paramètres
Black-Box Parallelization for Machine Learning
The landscape of machine learning applications is changing rapidly: large centralized datasets are replaced by high volume, high velocity data streams generated by a vast number of geographically distributed, loosely connected devices, such as mobile phones, smart sensors, autonomous vehicles or industrial machines. Current learning approaches centralize the data and process it in parallel in a cluster or computing center. This has three major disadvantages: (i) it does not scale well with the number of data-generating devices since their growth exceeds that of computing centers, (ii) the communication costs for centralizing the data are prohibitive in many applications, and (iii) it requires sharing potentially privacy-sensitive data. Pushing computation towards the data-generating devices alleviates these problems and allows to employ their otherwise unused computing power. However, current parallel learning approaches are designed for tightly integrated systems with low latency and high bandwidth, not for loosely connected distributed devices. Therefore, I propose a new paradigm for parallelization that treats the learning algorithm as a black box, training local models on distributed devices and aggregating them into a single strong one. Since this requires only exchanging models instead of actual data, the approach is highly scalable, communication-efficient, and privacy-preserving. Following this paradigm, this thesis develops black-box parallelizations for two broad classes of learning algorithms. One approach can be applied to incremental learning algorithms, i.e., those that improve a model in iterations. Based on the utility of aggregations it schedules communication dynamically, adapting it to the hardness of the learning problem. In practice, this leads to a reduction in communication by orders of magnitude. It is analyzed for (i) online learning, in particular in the context of in-stream learning, which allows to guarantee optimal regret and for (ii) batch learning based on empirical risk minimization where optimal convergence can be guaranteed. The other approach is applicable to non-incremental algorithms as well. It uses a novel aggregation method based on the Radon point that allows to achieve provably high model quality with only a single aggregation. This is achieved in polylogarithmic runtime on quasi-polynomially many processors. This relates parallel machine learning to Nick's class of parallel decision problems and is a step towards answering a fundamental open problem about the abilities and limitations of efficient parallel learning algorithms. An empirical study on real distributed systems confirms the potential of the approaches in realistic application scenarios
On the Intersection of Communication and Machine Learning
The intersection of communication and machine learning is attracting increasing interest from both communities. On the one hand, the development of modern communication system brings large amount of data and high performance requirement, which challenges the classic analytical-derivation based study philosophy and encourages the researchers to explore the data driven method, such as machine learning, to solve the problems with high complexity and large scale. On the other hand, the usage of distributed machine learning introduces the communication cost as one of the basic considerations for the design of machine learning algorithm and system.In this thesis, we first explore the application of machine learning on one of the classic problems in wireless network, resource allocation, for heterogeneous millimeter wave networks when the environment is with high dynamics. We address the practical concerns by providing the efficient online and distributed framework. In the second part, some sampling based communication-efficient distributed learning algorithm is proposed. We utilize the trade-off between the local computation and the total communication cost and propose the algorithm with good theoretical bound. In more detail, this thesis makes the following contributionsWe introduced an reinforcement learning framework to solve the resource allocation problems in heterogeneous millimeter wave network. The large state/action space is decomposed according to the topology of the network and solved by an efficient distribtued message passing algorithm. We further speed up the inference process by an online updating process.We proposed the distributed coreset based boosting framework. An efficient coreset construction algorithm is proposed based on the prior knowledge provided by clustering. Then the coreset is integrated with boosting with improved convergence rate. We extend the proposed boosting framework to the distributed setting, where the communication cost is reduced by the good approximation of coreset.We propose an selective sampling framework to construct a subset of sample that could effectively represent the model space. Based on the prior distribution of the model space or the large amount of samples from model space, we derive a computational efficient method to construct such subset by minimizing the error of classifying a classifier
Objective measures of complexity
Mesures Objectives de la Complexité pour la Prise de Décision Dynamique. La gestion efficace de systèmes sociotechniques complexes dépend d’une compréhension des interrelations dynamiques entre les composantes de ces systèmes, de leur évolution à travers le temps, ainsi que du degré d’incertitude auquel les décideurs sont exposés. Quelles sont les caractéristiques de la prise de décision complexe qui ont un impact sur la performance humaine dans l’environnement moderne du travail, constamment en fluctuation et sous la pression du temps, exerçant de lourdes demandes sur la cognition ? La prise de décision complexe est un concept issu de la macrocognition, impliquant des processus et des fonctions de bas et haut niveaux de description tels que la métacognition, soit pour un individu de penser à propos de son propre processus de pensées. Dans le cas particulier de la prise de décision complexe, ce phénomène est nommé la pensée systémique. L’étude de la prise de décision complexe en dehors de l’environnement traditionnel du laboratoire, permettant un haut niveau de contrôle mais un faible degré de réalisme, est malheureusement difficile et presque impossible. Une méthode de recherche plus appropriée pour la macrocognition est l’expérimentation basée sur la simulation, à l’aide de micromondes numérisés sous la forme de jeux sérieux. Ce paradigme de recherche est nommé la prise de décision dynamique (PDD), en ce qu’il tient compte des caractéristiques de problèmes de prise de décision complexe telles que des séquences complexes de décisions et de changements d’états d’un problème interdépendants, qui peuvent changer de façon spontanée ou comme conséquence de décisions préalables, et pour lesquels la connaissance et la compréhension du décideur peut n’être que partielle ou incertaine. Malgré la quantité de recherche concernant la PDD à propos des difficultés encourues pour la performance humaine face à des problèmes de prise de décision complexe, l’acquisition de connaissances à propos de systèmes complexes, et à savoir si le transfert de l’apprentissage est possible, il n’existe pas de mesure quantitative de ce en quoi un problème de décision est considéré comme étant complexe. La littérature scientifique mentionne des éléments qualitatifs concernant les systèmes complexes (tels que des interrelations dynamiques, une évolution non-linéaire d’un système à travers le temps, et l’incertitude à propos des états d’un système et des issues des décisions), mais des mesures quantitatives et objectives exprimant la complexité de problèmes de décision n’ont pas été développées. Cette dissertation doctorale présente les concepts, la méthodologie et les résultats impliqués dans un projet de recherche visant à développer des mesures objectives de la complexité basées sur les caractéristiques de problèmes de prise de décision dynamique pouvant expliquer et prédire la performance humaine. En s’inspirant de divers domaines d’application de la théorie de la complexité tels que la complexité computationnelle, la complexité systémique, et l’informatique cognitive, un modèle formel des paramètre de la complexité pour des tâches de prise de décision dynamique a été élaboré. Un ensemble de dix mesures objectives de la complexité a été développé, consistant en des mesures de la complexité structurelle, des mesures de la complexité informationnelle, la complexité de la charge cognitive, et des mesures de la difficulté d’un problème, de la non-linéarité des relations, de l’incertitude concernant l’information et les décisions, ainsi qu’une mesure de l’instabilité d’un système dynamique sous des conditions d’inertie. Une analyse des résultats expérimentaux colligés à partir de cinq scénarios de PDD révèle qu’un nombre restreint de candidats parmi des modèles de régression linéaires multiple permet d’expliquer et de prédire les résultats de performance humaine, mais au prix de certaines violations des postulats de l’approche classique de la régression linéaire. De plus, ces mesures objectives de la complexité présentent un degré élevé de multicolinéarité, causée d’une part par l’inclusion de caractéristiques redondantes dans les calculs, et d’autre part par une colinéarité accidentelle imputable à la conception des scénarios de PDD. En tenant compte de ces deux considérations ainsi que de la variance élevée observée dans les processus macrocognitifs impliqués dans la prise de décision complexe, ces modèles présentent des valeurs élevées pour le terme d’erreur exprimant l’écart entre les observations et les prédictions des modèles. Une analyse additionnelle explore l’utilisation de méthodes alternatives de modélisation par régression afin de mieux comprendre la relation entre les paramètres de la complexité et les données portant sur performance humaine. Nous avons d’abord opté pour une approche de régression robuste afin d’augmenter l’efficience de l’analyse de régression en utilisant une méthode réduisant la sensibilité des modèles de régression aux observations influentes. Une seconde analyse élimine la source de variance imputable aux différences individuelles en focalisant exclusivement sur les effets imputables aux conditions expérimentales. Une dernière analyse utilise des modèles non-linéaires et non-paramétriques afin de pallier les postulats de la modélisation par régression, à l’aide de méthodes d’apprentissage automatique (machine learning). Les résultats suggèrent que l’approche de régression robuste produit des termes d’erreur substantiellement plus faibles, en combinaison avec des valeurs élevées pour les mesures de variance expliquée dans les données de la performance humaine. Bien que les méthodes non-linéaires et non-paramétriques produisent des modèles marginalement plus efficients en comparaison aux modèles de régression linéaire, la combinaison de ces modèles issus du domaine de l’apprentissage automatique avec les données restreintes aux effets imputables aux conditions expérimentales produit les meilleurs résultats relativement à l’ensemble de l’effort de modélisation et d’analyse de régression. Une dernière section présente un programme de recherche conçu pour explorer l’espace des paramètres pour les mesures objectives de la complexité avec plus d’ampleur et de profondeur, afin d’appréhender les combinaisons des caractéristiques des problèmes de prise de décision complexe qui sont des facteurs déterminants de la performance humaine. Les discussions concernant l’approche expérimentale pour la PDD, les résultats de l’expérimentation relativement aux modèles de régression, ainsi qu’à propos de l’investigation de méthodes alternatives visant à réduire la composante de variance menant à la disparité entre les observations et les prédictions des modèles suggèrent toutes que le développement de mesures objectives de la complexité pour la performance humaine dans des scénarios de prise de décision dynamique est une approche viable à l’approfondissement de nos connaissances concernant la compréhension et le contrôle exercés par un être humain face à des problèmes de décision complexe.Objective Measures of Complexity for Dynamic Decision-Making. Managing complex sociotechnical systems depends on an understanding of the dynamic interrelations of such systems’ components, their evolution over time, and the degree of uncertainty to which decision makers are exposed. What features of complex decision-making impact human performance in the cognitively demanding, ever-changing and time pressured modern workplaces? Complex decision-making is a macrocognitive construct, involving low to high cognitive processes and functions, such as metacognition, or thinking about one’s own thought processes. In the particular case of complex decision-making, this is called systems thinking. The study of complex decision-making outside of the controlled, albeit lacking in realism, traditional laboratory environment is difficult if not impossible. Macrocognition is best studied through simulation-based experimentation, using computerized microworlds in the form of serious games. That research paradigm is called dynamic decision-making (DDM), as it takes into account the features of complex decision problems, such as complex sequences of interdependent decisions and changes in problem states, which may change spontaneously or as a consequence of earlier decisions, and for which the knowledge and understanding may be only partial or uncertain. For all the research in DDM concerning the pitfalls of human performance in complex decision problems, the acquisition of knowledge about complex systems, and whether a learning transfer is possible, there is no quantitative measure of what constitutes a complex decision problem. The research literature mentions the qualities of complex systems (a system’s dynamical relationships, the nonlinear evolution of the system over time, and the uncertainty about the system states and decision outcomes), but objective quantitative measures to express the complexity of decision problems have not been developed. This dissertation presents the concepts, methodology, and results involved in a research endeavor to develop objective measures of complexity based on characteristics of dynamic decision-making problems which can explain and predict human performance. Drawing on the diverse fields of application of complexity theory such as computational complexity, systemic complexity, and cognitive informatics, a formal model of the parameters of complexity for dynamic decision-making tasks has been elaborated. A set of ten objective measures of complexity were developed, ranging from structural complexity measures, measures of information complexity, the cognitive weight complexity, and measures of problem difficulty, nonlinearity among relationships, information and decision uncertainty, as well as a measure of the dynamical system’s instability under inertial conditions. An analysis of the experimental results gathered using five DDM scenarios revealed that a small set of candidate models of multiple linear regression could explain and predict human performance scores, but at the cost of some violations of the assumptions of classical linear regression. Additionally, the objective measures of complexity exhibited a high level of multicollinearity, some of which were caused by redundant feature computation while others were accidentally collinear due to the design of the DDM scenarios. Based on the aforementioned constraints, and due to the high variance observed in the macrocognitive processes of complex decision-making, the models exhibited high values of error in the discrepancy between the observations and the model predictions. Another exploratory analysis focused on the use of alternative means of regression modeling to better understand the relationship between the parameters of complexity and the human performance data. We first opted for a robust regression analysis to increase the efficiency of the regression models, using a method to reduce the sensitivity of candidate regression models to influential observations. A second analysis eliminated the within-treatment source of variance in order to focus exclusively on between-treatment effects. A final analysis used nonlinear and non-parametric models to relax the regression modeling assumptions, using machine learning methods. It was found that the robust regression approach produced substantially lower error values, combined with high measures of the variance explained for the human performance data. While the machine learning methods produced marginally more efficient models of regression for the same candidate models of objective measures of complexity, the combination of the nonlinear and non-parametric methods with the restricted between-treatment dataset yielded the best results of all of the modeling and analyses endeavors. A final section presents a research program designed to explore the parameter space of objective measures of complexity in more breadth and depth, so as to weight which combinations of the characteristics of complex decision problems are determinant factors on human performance. The discussions about the experimental approach to DDM, the experimental results relative to the regression models, and the investigation of further means to reduce the variance component underlying the discrepancy between the observations and the model predictions all suggest that establishing objective measures of complexity for human performance in dynamic decision-making scenarios is a viable approach to furthering our understanding of a decision maker’s comprehension and control of complex decision problems
On Price Responsive Consumer Behavior in Electricity Markets: to Machina Economicus from Homo Agens
The electricity power market is well known for its highly volatile nature due to its innate variability characteristic of demand and the absence of practical bulk storage at reasonable cost. Any discordance between rapid fluctuation in wholesale prices and near flat retail prices not only incurs economic inefficiency in terms of social welfare, but also creates price-inelastic wholesale demand which severely exacerbates the volatility of wholesale electricity prices. While the market has a fundamental dynamic nature, the behavioral aspect of power consumption in response to price changes is not well understood. This necessitate to develop a empirical modeling methodology of demand which can potentially provide practical insights into demand response. In the former part of this work, we focus on dynamic aspect of demand response in Chapter 2. We first show that (i) demand is well responsive to outlier high price surges, and (ii) demand response can incur a certain amount of delay. Examining further data, it appears that demand is responsive to anticipated prices. This is in conformity with our previous observations on the inertia of demand, and testing the hypothesis that demand actually responds to anticipated prices rather than actual real time prices is an important next step.
While it is impractical to obtain a particular individual’s own price prediction, We propose to test the hypothesis with day-ahead electricity prices (DAP). In addition, as an initial step toward the derivation of a quantitative model of electricity load and price, we propose a model of “appliance” usage as a representative basic component of electricity load. In the latter part of this work, we investigate more fundamental aspect of data-centric modeling in Chapter 3. First, we show the limitation of pure data-centric modeling strategy by proving that having a perfect knowledge on the joint distribution on price and load does not identify the load behavior in response to price. As it turns out that the causal structure of the variables of interest is the central matter that determines load behavior identifiability, we derive a minimal identifiable causal structure of demand response from the preexisting economic theories. Based on the discovered causal structure, we propose a minimal Bayesian model representation called “stochastic neuron” which connects machine learning technique to demand response modeling. We show that a stochastic neuron is an explainable tool as expressive as an ordinary neural network, and well extends the arguments from “appliance” usage model
Computational Intelligence in Healthcare
This book is a printed edition of the Special Issue Computational Intelligence in Healthcare that was published in Electronic
- …