A Reinforcement Learning Based Model for Adaptive ServiceQuality Management in E-Commerce Websites

Providing high-quality service to all users is adifficult and inefficient strategy for e-commerce providers,especially when Web servers experience overload condi-tions that cause increased response time and requestrejections, leading to user frustration and reduced revenue.In an e-commerce system, customer Web sessions havediffering values for service providers. These tend to: givepreference to customer Web sessions that are likely tobring more profit by providing better service quality. Thispaper proposes a reinforcement-learning based adaptivee-commerce system model that adapts the service qualitylevel for different Web sessions within the customer’snavigation in order to maximize total profit. The e-com-merce system is considered as an electronic supply chainwhich includes a network of basic e- providers used tosupply e-commerce services for end customers. The learneragent noted as e-commerce supply chain manager(ECSCM) agent allocates a service quality level to thecustomer’s request based on his/her navigation pattern inthe e-commerce Website and selects an optimized combi-nation of service providers to respond to the customer’srequest. To evaluate the proposed model, a multi agentframework composed of three agent types, the ECSCMagent, customer agent (buyer/browser) and service provideragent, is employed. Experimental results show that theproposed model improves total profits through costreduction and revenue enhancement simultaneously andencourages customers to purchase from the Websitethrough service quality adaptation

Detecting Structure In Chaos: A Customer Process Analysis Method

Detecting typical patterns in customer processes is the precondition for gaining an understanding about customer issues and needs in the course of performing their processes. Such insights can be translated into customer-centric service offerings that provide added value by enabling customers to reach their process objectives more effectively and rapidly, and with less effort. However, customer processes performed in less restrictive environments are extremely heterogeneous, which makes them difficult to analyse. Current approaches deal with this issue by considering customer processes in large scope and low detail, or vice versa. However, both views are required to understand customer processes comprehensively. Therefore, we present a novel customer process analysis method capable of detecting the hidden activity-cluster structure of customer processes. Consequently, both the detailed level of process activities and the aggregated cluster level are available for customer process analysis, which increases the chances of detecting patterns in these heterogeneous processes. We apply the method to two datasets and evaluate the results’ validity and utility. Moreover, we demonstrate that the method outperforms alternative solution technologies. Finally, we provide new insights into customer process theory

An Artificial Immune System-Inspired Multiobjective Evolutionary Algorithm with Application to the Detection of Distributed Computer Network Intrusions

Today\u27s predominantly-employed signature-based intrusion detection systems are reactive in nature and storage-limited. Their operation depends upon catching an instance of an intrusion or virus after a potentially successful attack, performing post-mortem analysis on that instance and encoding it into a signature that is stored in its anomaly database. The time required to perform these tasks provides a window of vulnerability to DoD computer systems. Further, because of the current maximum size of an Internet Protocol-based message, the database would have to be able to maintain 25665535 possible signature combinations. In order to tighten this response cycle within storage constraints, this thesis presents an Artificial Immune System-inspired Multiobjective Evolutionary Algorithm intended to measure the vector of trade-off solutions among detectors with regard to two independent objectives: best classification fitness and optimal hypervolume size. Modeled in the spirit of the human biological immune system and intended to augment DoD network defense systems, our algorithm generates network traffic detectors that are dispersed throughout the network. These detectors promiscuously monitor network traffic for exact and variant abnormal system events, based on only the detector\u27s own data structure and the ID domain truth set, and respond heuristically. The application domain employed for testing was the MIT-DARPA 1999 intrusion detection data set, composed of 7.2 million packets of notional Air Force Base network traffic. Results show our proof-of-concept algorithm correctly classifies at best 86.48% of the normal and 99.9% of the abnormal events, attributed to a detector affinity threshold typically between 39-44%. Further, four of the 16 intrusion sequences were classified with a 0% false positive rate

From metaheuristics to learnheuristics: Applications to logistics, finance, and computing

Un gran nombre de processos de presa de decisions en sectors estratègics com el transport i la producció representen problemes NP-difícils. Sovint, aquests processos es caracteritzen per alts nivells d'incertesa i dinamisme. Les metaheurístiques són mètodes populars per a resoldre problemes d'optimització difícils en temps de càlcul raonables. No obstant això, sovint assumeixen que els inputs, les funcions objectiu, i les restriccions són deterministes i conegudes. Aquests constitueixen supòsits forts que obliguen a treballar amb problemes simplificats. Com a conseqüència, les solucions poden conduir a resultats pobres. Les simheurístiques integren la simulació a les metaheurístiques per resoldre problemes estocàstics d'una manera natural. Anàlogament, les learnheurístiques combinen l'estadística amb les metaheurístiques per fer front a problemes en entorns dinàmics, en què els inputs poden dependre de l'estructura de la solució. En aquest context, les principals contribucions d'aquesta tesi són: el disseny de les learnheurístiques, una classificació dels treballs que combinen l'estadística / l'aprenentatge automàtic i les metaheurístiques, i diverses aplicacions en transport, producció, finances i computació.Un gran número de procesos de toma de decisiones en sectores estratégicos como el transporte y la producción representan problemas NP-difíciles. Frecuentemente, estos problemas se caracterizan por altos niveles de incertidumbre y dinamismo. Las metaheurísticas son métodos populares para resolver problemas difíciles de optimización de manera rápida. Sin embargo, suelen asumir que los inputs, las funciones objetivo y las restricciones son deterministas y se conocen de antemano. Estas fuertes suposiciones conducen a trabajar con problemas simplificados. Como consecuencia, las soluciones obtenidas pueden tener un pobre rendimiento. Las simheurísticas integran simulación en metaheurísticas para resolver problemas estocásticos de una manera natural. De manera similar, las learnheurísticas combinan aprendizaje estadístico y metaheurísticas para abordar problemas en entornos dinámicos, donde los inputs pueden depender de la estructura de la solución. En este contexto, las principales aportaciones de esta tesis son: el diseño de las learnheurísticas, una clasificación de trabajos que combinan estadística / aprendizaje automático y metaheurísticas, y varias aplicaciones en transporte, producción, finanzas y computación.A large number of decision-making processes in strategic sectors such as transport and production involve NP-hard problems, which are frequently characterized by high levels of uncertainty and dynamism. Metaheuristics have become the predominant method for solving challenging optimization problems in reasonable computing times. However, they frequently assume that inputs, objective functions and constraints are deterministic and known in advance. These strong assumptions lead to work on oversimplified problems, and the solutions may demonstrate poor performance when implemented. Simheuristics, in turn, integrate simulation into metaheuristics as a way to naturally solve stochastic problems, and, in a similar fashion, learnheuristics combine statistical learning and metaheuristics to tackle problems in dynamic environments, where inputs may depend on the structure of the solution. The main contributions of this thesis include (i) a design for learnheuristics; (ii) a classification of works that hybridize statistical and machine learning and metaheuristics; and (iii) several applications for the fields of transport, production, finance and computing

From Parameter Tuning to Dynamic Heuristic Selection

The importance of balance between exploration and exploitation plays a crucial role while solving combinatorial optimization problems. This balance is reached by two general techniques: by using an appropriate problem solver and by setting its proper parameters. Both problems were widely studied in the past and the research process continues up until now. The latest studies in the field of automated machine learning propose merging both problems, solving them at design time, and later strengthening the results at runtime. To the best of our knowledge, the generalized approach for solving the parameter setting problem in heuristic solvers has not yet been proposed. Therefore, the concept of merging heuristic selection and parameter control have not been introduced. In this thesis, we propose an approach for generic parameter control in meta-heuristics by means of reinforcement learning (RL). Making a step further, we suggest a technique for merging the heuristic selection and parameter control problems and solving them at runtime using RL-based hyper-heuristic. The evaluation of the proposed parameter control technique on a symmetric traveling salesman problem (TSP) revealed its applicability by reaching the performance of tuned in online and used in isolation underlying meta-heuristic. Our approach provides the results on par with the best underlying heuristics with tuned parameters.:1 Introduction 1 1.1 Motivation 1 1.2 Research objective 2 1.3 Solution overview 2 2 Background and RelatedWork Analysis 3 2.1 Optimization Problems and their Solvers 3 2.2 Heuristic Solvers for Optimization Problems 9 2.3 Setting Algorithm Parameters 19 2.4 Combined Algorithm Selection and Hyper-Parameter Tuning Problem 27 2.5 Conclusion on Background and Related Work Analysis 28 3 Online Selection Hyper-Heuristic with Generic Parameter Control 31 3.1 Combined Parameter Control and Algorithm Selection Problem 31 3.2 Search Space Structure 32 3.3 Parameter Prediction Process 34 3.4 Low-Level Heuristics 35 3.5 Conclusion of Concept 36 4 Implementation Details 37 4.2 Search Space 40 4.3 Prediction Process 43 4.4 Low Level Heuristics 48 4.5 Conclusion 52 5 Evaluation 55 5.1 Optimization Problem 55 5.2 Environment Setup 56 5.3 Meta-heuristics Tuning 56 5.4 Concept Evaluation 60 5.5 Analysis of HH-PC Settings 74 5.6 Conclusion 79 6 Conclusion 81 7 FutureWork 83 7.1 Prediction Process 83 7.2 Search Space 84 7.3 Evaluations and Benchmarks 84 Bibliography 87 A Evaluation Results 99 A.1 Results in Figures 99 A.2 Results in numbers 10

Cognitive Security Framework For Heterogeneous Sensor Network Using Swarm Intelligence

Rapid development of sensor technology has led to applications ranging from academic to military in a short time span. These tiny sensors are deployed in environments where security for data or hardware cannot be guaranteed. Due to resource constraints, traditional security schemes cannot be directly applied. Unfortunately, due to minimal or no communication security schemes, the data, link and the sensor node can be easily tampered by intruder attacks. This dissertation presents a security framework applied to a sensor network that can be managed by a cohesive sensor manager. A simple framework that can support security based on situation assessment is best suited for chaotic and harsh environments. The objective of this research is designing an evolutionary algorithm with controllable parameters to solve existing and new security threats in a heterogeneous communication network. An in-depth analysis of the different threats and the security measures applied considering the resource constrained network is explored. Any framework works best, if the correlated or orthogonal performance parameters are carefully considered based on system goals and functions. Hence, a trade-off between the different performance parameters based on weights from partially ordered sets is applied to satisfy application specific requirements and security measures. The proposed novel framework controls heterogeneous sensor network requirements,and balance the resources optimally and efficiently while communicating securely using a multi-objection function. In addition, the framework can measure the affect of single or combined denial of service attacks and also predict new attacks under both cooperative and non-cooperative sensor nodes. The cognitive intuition of the framework is evaluated under different simulated real time scenarios such as Health-care monitoring, Emergency Responder, VANET, Biometric security access system, and Battlefield monitoring. The proposed three-tiered Cognitive Security Framework is capable of performing situation assessment and performs the appropriate security measures to maintain reliability and security of the system. The first tier of the proposed framework, a crosslayer cognitive security protocol defends the communication link between nodes during denial-of-Service attacks by re-routing data through secure nodes. The cognitive nature of the protocol balances resources and security making optimal decisions to obtain reachable and reliable solutions. The versatility and robustness of the protocol is justified by the results obtained in simulating health-care and emergency responder applications under Sybil and Wormhole attacks. The protocol considers metrics from each layer of the network model to obtain an optimal and feasible resource efficient solution. In the second tier, the emergent behavior of the protocol is further extended to mine information from the nodes to defend the network against denial-of-service attack using Bayesian models. The jammer attack is considered the most vulnerable attack, and therefore simulated vehicular ad-hoc network is experimented with varied types of jammer. Classification of the jammer under various attack scenarios is formulated to predict the genuineness of the attacks on the sensor nodes using receiver operating characteristics. In addition to detecting the jammer attack, a simple technique of locating the jammer under cooperative nodes is implemented. This feature enables the network in isolating the jammer or the reputation of node is affected, thus removing the malicious node from participating in future routes. Finally, a intrusion detection system using `bait\u27 architecture is analyzed where resources is traded-off for the sake of security due to sensitivity of the application. The architecture strategically enables ant agents to detect and track the intruders threateningthe network. The proposed framework is evaluated based on accuracy and speed of intrusion detection before the network is compromised. This process of detecting the intrusion earlier helps learn future attacks, but also serves as a defense countermeasure. The simulated scenarios of this dissertation show that Cognitive Security Framework isbest suited for both homogeneous and heterogeneous sensor networks

Air Force Institute of Technology Research Report 2000

This report summarizes the research activities of the Air Force Institute of Technology’s Graduate School of Engineering and Management. It describes research interests and faculty expertise; lists student theses/dissertations; identifies research sponsors and contributions; and outlines the procedures for contacting the school. Included in the report are: faculty publications, conference presentations, consultations, and funded research projects. Research was conducted in the areas of Aeronautical and Astronautical Engineering, Electrical Engineering and Electro-Optics, Computer Engineering and Computer Science, Systems and Engineering Management, Operational Sciences, and Engineering Physics