Performance and reliability in distributed systems

PhD ThesisThis thesis is devoted to the construction and analysis of models which can be used to evaluate the performance and reliability of distributed systems. The general object of the research therefore is to extend the types of queueing models with breakdowns which have been solved, with a particular interest in networking structures. The systems that are studied involve various collections of servers and their associated queues. These range from isolated nodes, though parallel nodes coupled by the effect of breakdowns on arrivals, to pipelines of such parallel stages and more general networks. The issues that are explored include the influence of breakdowns and repairs on delays, job losses and optimal routeing. Obtaining performance measures for interacting queues is difficult, however a degree of abstraction has been used here which allows long run averages to be calculated (exactly in many cases) for quite complex systems. A variety of different techniques are used in order to obtain solutions to these models, including exact equations, exact numerical and approximate numerical techniques

Integrated system based on the automation of the extraction and characterization, using natural processing language, of user ratings in sales platforms

[Abstract] At present, more and more importance is given to what people think, what they think and what are your preferences. With the rise of social networks and online stores, these data are accessible more than ever in a simple and global way, so it is increasingly important to know manage and analyze all this information. Thus, it is not surprising that over time more companies are focusing their interest on sentiment mining, which allows them to identify possible business opportunities, maintain a good reputation of the company in the networks or improve the marketing your products. For all these reasons, we are interested in building a tool capable of extracting from all the reviews that exists throughout the Internet, what is the opinion of a product/service (its most praised characteristics, its greatest shortcomings…), so that, through a good analysis, we can extract important information in an automated way. This paper will discuss how to build a tool capable of analyzing and retrieving reviews from the World Wide Web. Building two main systems, one for extracting reviews with the use of scraping tools and another for analyzing sentiment in texts with the use of machine learning and natural language processing.[Resumo] Na actualidade cada vez dáselle máis importancia a qué opinan as persoas, qué é o que pensan e cales son as suas preferencias. Coa subida da popularidade das redes sociais e as tendas online, estes datos estan máis accesibles que nunca, de manera sinxela e global, co que cada vez é máis importante saber xestionar e analizar toda esa información. Así, non é de extrañar que co tempo máis compañías estén centrando o seu interés na minaría de sentimentos, que lles permite posibles oportunidades de negocio, mantener unha boa reputación de empresa nas redes e mellorar o marketing. Por todo isto, interésanos construir unha ferramenta capaz de extraer de todas as reseñas que existen ao longo de internet, cal é a opinión que se ten dun produto/servizo, de forma que, a través dunha boa análise podamos obter información de importancia de maneira automatizada. Neste traballo falarase de como construir unha ferramenta capaz de analizar e obter as reseñas da World Wide Web. Construindo un sistema de extracción de reviews co uso de ferramentas de scraping e outro de análise de sentimentos en textos co uso de aprendizaxe automático e procesamento da linguaxe natural.Traballo fin de grao. Enxeñaría Informática. Curso 2021/202

Pooling and polling : creation of pooling in inventory and queueing models

The subject of the present monograph is the ‘Creation of Pooling in Inventory and Queueing Models’. This research consists of the study of sharing a scarce resource (such as inventory, server capacity, or production capacity) between multiple customer classes. This is called pooling, where the goal is to achieve cost or waiting time reductions. For the queueing and inventory models studied, both theoretical, scientific insights, are generated, as well as strategies which are applicable in practice. This monograph consists of two parts: pooling and polling. In both research streams, a scarce resource (inventory or server capacity, respectively production capacity) has to be shared between multiple users. In the first part of the thesis, pooling is applied to multi-location inventory models. It is studied how cost reduction can be achieved by the use of stock transfers between local warehouses, so-called lateral transshipments. In this way, stock is pooled between the warehouses. The setting is motivated by a spare parts inventory network, where critical components of technically advanced machines are kept on stock, to reduce down time durations. We create insights into the question when lateral transshipments lead to cost reductions, by studying several models. Firstly, a system with two stock points is studied, for which we completely characterize the structure of the optimal policy, using dynamic programming. For this, we formulate the model as a Markov decision process. We also derived conditions under which simple, easy to implement, policies are always optimal, such as a hold back policy and a complete pooling policy. Furthermore, we identified the parameter settings under which cost savings can be achieved. Secondly, we characterize the optimal policy structure for a multi-location model where only one stock point issues lateral transshipments, a so-called quick response warehouse. Thirdly, we apply the insights generated to the general multi-location model with lateral transshipments. We propose the use of a hold back policy, and construct a new approximation algorithm for deriving the performance characteristics. It is based on the use of interrupted Poisson processes. The algorithm is shown to be very accurate, and can be used for the optimization of the hold back levels, the parameters of this class of policies. Also, we study related inventory models, where a single stock point servers multiple customers classes. Furthermore, the pooling of server capacity is studied. For a two queue model where the head-of-line processor sharing discipline is applied, we derive the optimal control policy for dividing the servers attention, as well as for accepting customers. Also, a server farm with an infinite number of servers is studied, where servers can be turned off after a service completion in order to save costs. We characterize the optimal policy for this model. In the second part of the thesis polling models are studied, which are queueing systems where multiple queues are served by a single server. An application is the production of multiple types of products on a single machine. In this way, the production capacity is pooled between the product types. For the classical polling model, we derive a closedform approximation for the mean waiting time at each of the queues. The approximation is based on the interpolation of light and heavy traffic results. Also, we study a system with so-called smart customers, where the arrival rate at a queue depends on the position of the server. Finally, we invent two new service disciplines (the gated/exhaustive and the ??-gated discipline) for polling models, designed to yield ’fairness and efficiency’ in the mean waiting times. That is, they result in almost equal mean waiting times at each of the queues, without increasing the weighted sum of the mean waiting times too much

A vision-based optical character recognition system for real-time identification of tractors in a port container terminal

Automation has been seen as a promising solution to increase the productivity of modern sea port container terminals. The potential of increase in throughput, work efficiency and reduction of labor cost have lured stick holders to strive for the introduction of automation in the overall terminal operation. A specific container handling process that is readily amenable to automation is the deployment and control of gantry cranes in the container yard of a container terminal where typical operations of truck identification, loading and unloading containers, and job management are primarily performed manually in a typical terminal. To facilitate the overall automation of the gantry crane operation, we devised an approach for the real-time identification of tractors through the recognition of the corresponding number plates that are located on top of the tractor cabin. With this crucial piece of information, remote or automated yard operations can then be performed. A machine vision-based system is introduced whereby these number plates are read and identified in real-time while the tractors are operating in the terminal. In this paper, we present the design and implementation of the system and highlight the major difficulties encountered including the recognition of character information printed on the number plates due to poor image integrity. Working solutions are proposed to address these problems which are incorporated in the overall identification system.postprin

Evolutionary analyses of orphan genes in mouse lineages in the context of de novo gene birth

Gene birth is the process through which new genes appear. For a long time it was argued that the natural way of generating new genes was from copies of existing genes, and the possibility of de novo gene emergence was neglected. However, recent evidence has forced to reconsider old models and de novo gene birth gained recognition as a widespread phenomenon. De novo gene birth is the process by which a non-genic sequence is able to gain gene-like features through few mutations. The following work is a compilation of analyses that seek to highlight the importance and prevalence of de novo gene birth in genomes, suggesting that this is a process that is present at all times and which becomes very relevant upon ecological shifts. In the first chapter, I showed through phylostratigraphic analyses that new genes are substantially simpler than older, a trend which was consistent for several features and organisms, and suggestive of a frequent emergence of new genes through non-duplicative processes. In addition to this, I detected a strong association between gene birth and high transcriptional activity and chromosomal proximity. As part of this work, I was also able to use phylostratigraphy to evaluate a different model of gene birth, overprinting of alternative reading frames. In the following chapters of this dissertation, I made use of high-throughput sequencing of transcriptomes and genomes to ask questions about the origin and change of genes at closer time divergences than ever before, ranging from nearly 3000 years to 10 million years of divergence. I was able to detect the theoretically predicted effects of short time scale comparisons on the rate of protein evolution. Also, I contribute evidence that genes of different ages show different selective constraints even after only a few thousand years of divergence. Finally, in the last part of this thesis I evaluated the role of transcription in gene birth dynamics. Transcription seems to be a predominant feature of genomes, as most of the genome showed some level of transcription. In terms of de novo gene birth, I was able to identify 663 candidate loci from presence and absence of transcription. Analyses of these candidate loci indicated that gains are rather stable, meaning that subsequent losses were rarely found. In agreement with previous studies, I confirmed the role of testis as a driver of new genes. These results indicate that transcription is not a limiting factor in the emergence of new genes, and that our knowledge about the key regulatory elements of transcription and their turnover is still limited to explain why new genes seem to arise at a higher rate than they decay.Contents ......................................................................................................................................... 3 Summary of the thesis .................................................................................................................... 6 Zusammenfassung der Dissertation............................................................................................... 7 Acknowledgements ....................................................................................................................... 10 General introduction..................................................................................................................... 12 A brief historic perspective on the concepts of gene birth .................................................... 12 Gene duplication is the main source of new genes .............................................................. 12 Orphan genes and the genomics era .................................................................................... 14 Phylostratigraphy and the continuous emergence of new genes ......................................... 16 Not all genes come from other genes ................................................................................... 17 Considering gene birth from molecular and evolutionary perspectives ................................... 19 Overprinting: true innovation from existing genes .................................................................... 20 The life cycle of genes .............................................................................................................. 22 Overview................................................................................................................................... 24 Chapter 1: Phylogenetic patterns of emergence of new genes support a model of frequent de novo evolution ............................................................................................................................... 26 Introduction............................................................................................................................... 26 Results...................................................................................................................................... 27 Phylostratigraphy of mouse genes ........................................................................................ 27 Genomic features across ages.............................................................................................. 29 Chromosomal distribution ...................................................................................................... 33 Association with transcriptionally active sites ....................................................................... 33 Testis expressed genes......................................................................................................... 35 Alternative reading frames..................................................................................................... 36 Discussion ................................................................................................................................ 39 De novo evolution versus duplication-divergence ................................................................ 40 Regulatory evolution .............................................................................................................. 40 Overprinting ........................................................................................................................... 41 Conclusion................................................................................................................................ 42 Methods .................................................................................................................................... 43 Phylostratigraphy ................................................................................................................... 43 Gene structure analyses........................................................................................................ 43 Transcription associated regions........................................................................................... 44 Expression data for testis ...................................................................................................... 44 Secondary reading frames .................................................................................................... 44 Acknowledgements ................................................................................................................... 45 Chapter 2: Sequencing of genomes and transcriptomes of closely related mouse species....... 46 Introduction............................................................................................................................... 46 Using wild mice to understand gene birth at the transcriptome level ................................... 46 Phylogeographic distribution of the samples ........................................................................ 47 Methods .................................................................................................................................... 49 Biological material.................................................................................................................. 49 Transcriptome sequencing .................................................................................................... 49 Genome sequencing.............................................................................................................. 49 Raw data processing ............................................................................................................. 50 Transcriptome read mapping, annotation and quantification................................................ 50 Genome read mapping .......................................................................................................... 51 Available resources ................................................................................................................... 51 Chapter 3: Differential selective constrains across phylogenetic ages and their impact on the turnover of protein-coding genes. ................................................................................................. 53 Introduction............................................................................................................................... 53 Methods .................................................................................................................................... 53 Transcriptome assembly ....................................................................................................... 53 Generation of ortholog pairs and rate analyses .................................................................... 54 Overlapping genes................................................................................................................. 54 Reading frame polymorphism detection and annotation ...................................................... 55 Statistical analyses ................................................................................................................ 55 Results...................................................................................................................................... 55 Rate differences between genes of different ages ............................................................... 55 Overlapping genes are an unlikely source of bias ................................................................ 57 Impact of reading frame polymorphisms across phylogenetic time...................................... 59 Discussion ................................................................................................................................ 64 Acknowledgements ................................................................................................................... 66 Chapter 4: A transcriptomics approach to the gain and loss of de novo genes in mouse lineages...................................................................................................................................................... 67 Introduction............................................................................................................................... 67 How is a gene made? ............................................................................................................ 67 The early phase of new gene emergence............................................................................. 69 Pervasive transcription and junk-DNA as raw material for new genes ................................ 70 Methods .................................................................................................................................... 71 Transcriptome presence/absence matrix and mapping of gains and losses ....................... 71 Results...................................................................................................................................... 73 How much of the mouse genome has evidence of transcription? ........................................ 73 Genome-wide transcription: gain and loss dynamics ........................................................... 74 Phylogenetic patterns in genome-wide transcription ............................................................ 75 How much of the genome is transcribed in a lineage specific way? .................................... 77 Identification of cases of de novo transcripts ........................................................................ 81 Quantification of gain rates for curated genes ...................................................................... 84 What are the dynamics of transcription loss in known genes?............................................. 86 Where are new genes expressed?........................................................................................ 88 Discussion ................................................................................................................................ 89 Pervasive transcription can provide material for new genes ................................................ 89 Asymmetry in gains and losses of transcription.................................................................... 92 From transcribed protogenes to de novo genes ................................................................... 93 Differences in expression levels ............................................................................................ 95 Testis as a niche for new genes ............................................................................................ 95 Conclusion................................................................................................................................ 96 Concluding remarks ...................................................................................................................... 97 Perspectives................................................................................................................................. 98 References ................................................................................................................................... 99 Chapter contributions .................................................................................................................. 114 Appendices ................................................................................................................................ 115 Appendix A. Phylostratigraphic maps ..................................................................................... 115 Appendix B. Curation data from orphan genes ...................................................................... 115 Appendix C. Functional annotation clusters based on known genes with loss of expression ................................................................................................................................................ 117 Appendix D. Transcriptome information and statistics ........................................................... 118 Curriculum Vitae.......................................................................................................................... 119 Affidavit....................................................................................................................................... 12

System Integration Development and Testing Process

Organizations' decision making and business operations are more efficient when all of the data that is available for the organization can be used for them. System integrations are used to transfer the data between all of the relevant systems as a solution to this. Even though system integrations have a lot in common with other types of software development, they have differences and unique characteristics. An integration platform that consists of numerous integrations is developed to the City of Espoo and issues were discovered during the development and testing activities of them. This thesis was done to find solutions for those found problems and to make the processes smoother and more cost effective. Espoo has multiple integration platforms, the one studied in this thesis is only one of them. In this thesis the system integration specific challenges are looked into and reflected to other types of software development using literature concerning the subjects. After that a case study was done in the integration platform project and as a result process models of current development and testing processes were done. Using literature, interviews and personal experience improvement ideas were made and applied to form an improvement development process model for the project as it was discovered that the testing process itself had no significant flaws as the problems were inherited from the development process before it. The results can be applied to all kinds of system integration projects, as the problems seem to be universal and found in pretty much every integration project

Job shop scheduling with artificial immune systems

The job shop scheduling is complex due to the dynamic environment. When the information of the jobs and machines are pre-defined and no unexpected events occur, the job shop is static. However, the real scheduling environment is always dynamic due to the constantly changing information and different uncertainties. This study discusses this complex job shop scheduling environment, and applies the AIS theory and switching strategy that changes the sequencing approach to the dispatching approach by taking into account the system status to solve this problem. AIS is a biological inspired computational paradigm that simulates the mechanisms of the biological immune system. Therefore, AIS presents appealing features of immune system that make AIS unique from other evolutionary intelligent algorithm, such as self-learning, long-lasting memory, cross reactive response, discrimination of self from non-self, fault tolerance, and strong adaptability to the environment. These features of AIS are successfully used in this study to solve the job shop scheduling problem. When the job shop environment is static, sequencing approach based on the clonal selection theory and immune network theory of AIS is applied. This approach achieves great performance, especially for small size problems in terms of computation time. The feature of long-lasting memory is demonstrated to be able to accelerate the convergence rate of the algorithm and reduce the computation time. When some unexpected events occasionally arrive at the job shop and disrupt the static environment, an extended deterministic dendritic cell algorithm (DCA) based on the DCA theory of AIS is proposed to arrange the rescheduling process to balance the efficiency and stability of the system. When the disturbances continuously occur, such as the continuous jobs arrival, the sequencing approach is changed to the dispatching approach that involves the priority dispatching rules (PDRs). The immune network theory of AIS is applied to propose an idiotypic network model of PDRs to arrange the application of various dispatching rules. The experiments show that the proposed network model presents strong adaptability to the dynamic job shop scheduling environment.postprin

Virtualisation and Thin Client : A Survey of Virtual Desktop environments

This survey examines some of the leading commercial Virtualisation and Thin Client technologies. Reference is made to a number of academic research sources and to prominent industry specialists and commentators. A basic virtualisation Laboratory model is assembled to demonstrate fundamental Thin Client operations and to clarify potential problem areas