Asymptotic optimality of maximum pressure policies in stochastic processing networks

We consider a class of stochastic processing networks. Assume that the networks satisfy a complete resource pooling condition. We prove that each maximum pressure policy asymptotically minimizes the workload process in a stochastic processing network in heavy traffic. We also show that, under each quadratic holding cost structure, there is a maximum pressure policy that asymptotically minimizes the holding cost. A key to the optimality proofs is to prove a state space collapse result and a heavy traffic limit theorem for the network processes under a maximum pressure policy. We extend a framework of Bramson [Queueing Systems Theory Appl. 30 (1998) 89--148] and Williams [Queueing Systems Theory Appl. 30 (1998b) 5--25] from the multiclass queueing network setting to the stochastic processing network setting to prove the state space collapse result and the heavy traffic limit theorem. The extension can be adapted to other studies of stochastic processing networks.Comment: Published in at http://dx.doi.org/10.1214/08-AAP522 the Annals of Applied Probability (http://www.imstat.org/aap/) by the Institute of Mathematical Statistics (http://www.imstat.org

Traffic Optimization in Data Center and Software-Defined Programmable Networks

L'abstract è presente nell'allegato / the abstract is in the attachmen

Computation Offloading in Heterogeneous Vehicular Edge Networks:On-line and Off-policy Bandit Solutions

With the rapid advancement of Intelligent Transportation Systems (ITS) and vehicular communications, Vehicular Edge Computing (VEC) is emerging as a promising technology to support low-latency ITS applications and services. In this paper, we consider the computation offloading problem from mobile vehicles/users in a heterogeneous VEC scenario, and focus on the network and base station selection problems, where different networks have different traffic loads. In a fast-varying vehicular environment, computation offloading experience of users is strongly affected by the latency due to the congestion at the edge computing servers co-located with the base stations. However, as a result of the non-stationary property of such an environment and also information shortage, predicting this congestion is an involved task. To address this challenge, we propose an on-line learning algorithm and an off-policy learning algorithm based on multi-armed bandit theory. To dynamically select the least congested network in a piece-wise stationary environment, these algorithms predict the latency that the offloaded tasks experience using the offloading history. In addition, to minimize the task loss due to the mobility of the vehicles, we develop a method for base station selection. Moreover, we propose a relaying mechanism for the selected network, which operates based on the sojourn time of the vehicles. Through intensive numerical analysis, we demonstrate that the proposed learning-based solutions adapt to the traffic changes of the network by selecting the least congested network, thereby reducing the latency of offloaded tasks. Moreover, we demonstrate that the proposed joint base station selection and the relaying mechanism minimize the task loss in a vehicular environment

A dynamic DRR scheduling algorithm for flow level QOS assurances for elastic traffic

Cataloged from PDF version of article.Best effort service, used to transport the Internet traffic today, does not provide any QoS assurances. Intserv, DiffServ and recently proposed Proportional Diff- Serv architectures have been introduced to provide QoS. In these architectures, some applications with more stringent QoS requirement such as real time traffic are prioritized, while elastic flows share the remaining bandwidth. As opposed to the well studied differential treatment of delay and/or loss sensitive traffic to satisfy QoS constraints, our aim is satisfy QoS requirements of elastic traffic at the flow level. We intend to maintain different average rate levels for different classes of elastic traffic. For differential treatment of elastic flows, a dynamic variant of Deficit Round Robin Scheduler (DRR) is used as oppose to a FIFO queue. In this scheduling algorithm, all classes are served in a round robin fashion in proportion to their weights at each round. The main difference of our scheduler from the original DRR scheduler is that, we update the weights, which are called quantums of the scheduler at each round in response to the feedback from the network, which is in terms of the rate of phantom connection sharing capacity fairly with the other flows in the same queue. According to the rate measured in the last time interval, the controller updates the weights in proportion with the bandwidth requirements of each class to satisfy their QoS requirements, while the remaining bandwidth will be used by the best effort traffic. In order to find an optimal policy for the controller a simulation-based learning algorithm is performed using a processor sharing model of TCP, then the resultant policies are applied to a more realistic scenario to solve Dynamic DRR scheduling problem through ns-2 simulations.Kurugöl, SılaM.S

Optical flow switched networks

Thesis (Ph. D.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2009.Includes bibliographical references (p. 253-279).In the four decades since optical fiber was introduced as a communications medium, optical networking has revolutionized the telecommunications landscape. It has enabled the Internet as we know it today, and is central to the realization of Network-Centric Warfare in the defense world. Sustained exponential growth in communications bandwidth demand, however, is requiring that the nexus of innovation in optical networking continue, in order to ensure cost-effective communications in the future. In this thesis, we present Optical Flow Switching (OFS) as a key enabler of scalable future optical networks. The general idea behind OFS-agile, end-to-end, all-optical connections-is decades old, if not as old as the field of optical networking itself. However, owing to the absence of an application for it, OFS remained an underdeveloped idea-bereft of how it could be implemented, how well it would perform, and how much it would cost relative to other architectures. The contributions of this thesis are in providing partial answers to these three broad questions. With respect to implementation, we address the physical layer design of OFS in the metro-area and access, and develop sensible scheduling algorithms for OFS communication. Our performance study comprises a comparative capacity analysis for the wide-area, as well as an analytical approximation of the throughput-delay tradeoff offered by OFS for inter-MAN communication. Lastly, with regard to the economics of OFS, we employ an approximate capital expenditure model, which enables a throughput-cost comparison of OFS with other prominent candidate architectures. Our conclusions point to the fact that OFS offers significant advantage over other architectures in economic scalability.(cont.) In particular, for sufficiently heavy traffic, OFS handles large transactions at far lower cost than other optical network architectures. In light of the increasing importance of large transactions in both commercial and defense networks, we conclude that OFS may be crucial to the future viability of optical networking.by Guy E. Weichenberg.Ph.D

Dynamic scheduling in a multi-product manufacturing system

To remain competitive in global marketplace, manufacturing companies need to improve their operational practices. One of the methods to increase competitiveness in manufacturing is by implementing proper scheduling system. This is important to enable job orders to be completed on time, minimize waiting time and maximize utilization of equipment and machineries. The dynamics of real manufacturing system are very complex in nature. Schedules developed based on deterministic algorithms are unable to effectively deal with uncertainties in demand and capacity. Significant differences can be found between planned schedules and actual schedule implementation. This study attempted to develop a scheduling system that is able to react quickly and reliably for accommodating changes in product demand and manufacturing capacity. A case study, 6 by 6 job shop scheduling problem was adapted with uncertainty elements added to the data sets. A simulation model was designed and implemented using ARENA simulation package to generate various job shop scheduling scenarios. Their performances were evaluated using scheduling rules, namely, first-in-first-out (FIFO), earliest due date (EDD), and shortest processing time (SPT). An artificial neural network (ANN) model was developed and trained using various scheduling scenarios generated by ARENA simulation. The experimental results suggest that the ANN scheduling model can provided moderately reliable prediction results for limited scenarios when predicting the number completed jobs, maximum flowtime, average machine utilization, and average length of queue. This study has provided better understanding on the effects of changes in demand and capacity on the job shop schedules. Areas for further study includes: (i) Fine tune the proposed ANN scheduling model (ii) Consider more variety of job shop environment (iii) Incorporate an expert system for interpretation of results. The theoretical framework proposed in this study can be used as a basis for further investigation

Solving large structured Markov Decision Problems for perishable inventory management and traffic control

Published data revealed that Tagetes spp. suppress polyphagous endoparasitic root nematodes, that the effect varies, perhaps between Tagetes spp. and cultivars, certainly between nematode genera and perhaps between species and strains. The effect is sometimes striking but the picture in general is far from complete and not clear. This situation determined the three objectives of our investigation: occurrence and significance of Tagetes effect, interpretation, and possibilities of application in agriculture.For most of the trials plants were cultivated under controlled and field conditions, and their growth evaluated; nematode populations were collected, cultivated, maintained and transferred; nematodes in soil and plant tissues were counted and results were analysed statistically. Several special techniques were used occasionally as indicated in the relevant sections.The occurrence and significance of special Tagetes effects on plant nematode population were determined with species of Pratylenchus, Meloidogyne, Tylenchorhynchus, Rotylenchus and Helicotylenchus, other ectoparasitic nematodes, Ditylenchus and Aphelenchoides.Pratylenchus spp. were markedly suppressed by Tagetes spp. in tube cultures (Tables 1-5) and in field trials (Tables 6 and 7, Figs 1 and 2). This was true for P. penetrans, P. crenatus, P. neglectus and probably P. thornei. Soil type may be of influence on the result. There were great differences in effectiveness between Tagetes spp. and cultivars. The effectiveness against Pratylenchus spp. decreases in the order T. patula, T. erecta, T. minuta, with T. patula markedly better. T. patula Harmony suppresses field populations of Pratylenchus spp. in a few months and fallow requires a few years to reach a comparable low final density of these nematodes. Density never fell to zero, probably because of limited reproduction on weeds and limited reproduction on Tagetes itself if it is grown for a full season. There was no evidence that resistance of Tagetes was broken or that nematode strains resistant to Tagetes effects arose even after 7-10 successive crops of T. patula. Biennial rotations of T. patula and good hosts kept Pratylenchus spp. at a low density, except the population of P. thornei on heavy soil which fluctuated at a rather high level (Fig. 2).Tagetes spp. were generally as effective or better than fallow in suppressing Meloidogyne spp. in tube cultures (Tables 8-12), although with some notable exceptions. Meloidogyne larvae were less persistent than Pratylenchus larvae in fallow soil. T.patula severely suppressed M. hapla, M. incognita, M. arenaria and M.javanica. T. erecta was also suppressive but slightly more syncytia formed in all four Meloidogyne spp.; M. hapla reproduced and maintained a small population on T. erecta. T. minuta differed markedly from the earlier species in that it suppressed M. hapla and M. incognita completely and M.javanica almost completely, whereas M. arenaria could breed on this plant and reach considerable densities in both roots and soil. Any general effect by Tagetes on Meloidogyne is therefore complicated by certain exceptions which may account for conflicting published results.Tube and field trials showed that Tylenchorhynchus spp. were suppressed by T. patula in different soils (Table 7), that Tyl. dubius was suppressed better and more rapidly by T. patula than by fallow (Fig. 3A, B; literature), and that T. erecta and T. minuta were about as effective as T. patula against Tyl. dubius (Table 6). Tyl. dubius was suppressed slightly less effectively and less rapidly than Pratylenchus spp. in the same soil by T. patulaThe data on R.robustus supports the view that Tagetes spp., at any rate T. patula Golden Harmony and Harmony, maintains a rather high density. Unexplained peaks of the density under Tagetes and fallow make it difficult to indicate any Tagetes effect at all; factors other than the presence of higher plants may govern the population dynamics of this species.The genera Hemicycliophora, Paratylenchus and Trichodorus are not generally suppressed by Tagetes cultivars. Specific host-nematode relationships may vary as widely as is the case with these nematodes on other plants, and no special Tagetes effect can be demonstrated against these ectoparasites. Some species breed profusely on certain Tagetes spp. but are not affected by other Tagetes spp.The stem nematode D. dipsaci may reproduce to a limited extent and cause typical symptoms in T. patula and T. erecta, but not in T. minuta. The same is true for the foliar nematode A. ritzemabosi (Plate 1A and B).The data leave no doubt that certain Tagetes spp. suppress certain species of plant nematodes unusually strongly. The Tagetes effect manifested itself most clearly with Pratylenchus spp., Meloidogyne spp. and Tyl. dubius, but was evidently not present or not marked against R. robustus and several other ectoparasitic genera, nor against D. dipsaci and A. ritzemabosi. The results alone or combined with published data do not fully explain the mechanism. For a further analysis the exoradicular effects, the effects on the surface of the plant or during penetration, and the endoradicular effects were subsequently studied.Exoradicular effects may contribute to, but not explain the larger part of the Tagetes effect. As with the good host red clover, P. penetrans is not particularly attracted nor deterred by growing roots of T. patula on agar plates, but there was some aggregation around the roots in soil (Table 14), though it concerned only part of the soil population (Fig. 5). Percolates from pots of T. patula, red clover, apple or without a plant did not differ in effect on activity or mortality of P. penetrans in vitro. Survival of P. penetrans in water cultures of T. patula and T. minuta was only slightly less than in water culture of apple, control solution or distilled water, and the effect was at any rate slight and unspecific (Table 15). Tagetes soil was distinctly nematicidal for some days after the roots have been removed. This effect, however, was not very strong and was not specific for Tagetes , because red clover was equally and apple even more effective than T. patula and T. minuta (Table 16). Damage or kill of nematodes outside the root may therefore play a role, but it is apparently not the essential part of the Tagetes effect.Root systems of three different Tagetes spp. were penetrated by M. hapla larvae as much as root systems of a suitable host. The same holds for penetration by P. penetrans , except perhaps for T. patula which fewer nematodes entered than other Tagetes spp. or good hosts in most trials (Table 17). As a rule only a few of the larvae around the roots succeed in penetrating.Endoradicular influences comprise nematode survival and development, nematicidal effects and histological reactions in Tagetes tissues. M. hapla larvae enter Tagetes spp. to the same amount as tomato within a week. Only a few of the larvae around the roots succeed, although potential sites for penetration are present in excess. The unsuccessful nematodes outside the root decrease rapidly in number, more rapidly in the presence of growing plants than in fallow soil, which may be important for the population dynamics of nematodes in general. M. hapla survives for at least 4 weeks within roots of Tagetes spp., but development beyond the infective second larval stage is hardly noticeable in T. patula and T. minuta , whereas only a few larvae develop and reach adulthood in T. erecta (Table 18).The picture is similar with P. penetrans . The percentage penetration is generally low in all plants, but significantly lower in T. patula than in T. erecta , T. minuta or red clover (Table 19, Fig. 6). T. patula may resist penetration by this nematode The main difference between a suitable host such as red clover, and T. patula is that nematodes reproduce in the first plant and decline or remain few in T. patula. In T. minuta the nematodes survive and may even develop and multiply, though at slower rate than in red clover (Figs. 6, 7 and 8). T. patula , T. minuta and red clover represent degrees of host suitability from almost zero through low to very high (Fig. 7, Table 20). The equilibrium density under T. patula is very low but not zero because some reproduction occurs when the plants have grown a long while. The percentage males among the sexually differentiated nematodes is not higher in the very resistant T. patula and does not increase with ageing of host plants, as has often been published. T. patula allows very little escape or survival of P. penetrans once the nematodes have entered the roots, whereas the population in T. minuta roots does escape and may be infective as in suitable hosts (Tables 21 and 22).Root extracts of Tagetes spp., contrary to root exudates, contain a nematicidal principle which manifests itself against P. penetrans in vitro from the third day on (Fig. 9A, B, C). Extract of T. patula is more effective than extract of T. minuta and the latter is more effective than extract of potato or control solutions. Fractioning of root extracts of T. patula over a column of Sephadex G-75 indicate high mortality in a later fraction. No attempt was made to identify the active principle(s) in this fraction. This effect in vitro may be related to the Tagetes effect on nematodes in soil. It may be caused by the nematicides such as the thiophenes isolated from Tagetes roots by Uhlenbroek & Bijloo (1958, 1959). There was some nematode kill in potato root extract too, distinctly less than in Tagetes extract but distinctly higher than in the control solutions. This indicates the presence of a weak nematicidal effect in potato root extract and may support the indication recorded earlier that plants establish or induce in general an "antinemic potential" by means of their exudates or other substances in water or soil. The Tagetes effects are much stronger and evidently differ from it.Histological reactions as part of the endoradicular influences of Tagetes on penetrated nematodes are not conspicuous. Tagetes cultures, unlike most other plants, normally grow well and have well-developed root systems without discoloration in soils with dense populations of Meloidogyne and Pratylenchus spp. Despite this healthy appearance Tagetes roots may show barely visible histological reactions after such nematode infestations.M. hapla larvae penetrate the root apex of T. patula in much the same way as that of tomato. They do not normally develop nor cause marked necrosis or swelling in the roots of T. patula . Occasionally, however, small syncytia or sometimes even small galls occur associated with a developing larva. Unlike tomato, T.patula develops few and very small syncytia and galls and only slowly, and the nematode often dies and syncytia often abort in T. patula .P. penetrans penetrates young roots of T. patula , T. minuta and red clover at random sites on their surface except at the apex and causes cortical lesions in all three plants. The lesions in T. patula are small, dark and necrotic but do not abort from surrounding cortex tissue; they normally harbour only 1-3 nematodes, often dead, dying or twisted. In T. minuta the lesions appear slower, are larger and less dark than in T. patula ; the number of nematodes per lesion may be up to 38 and often occur outside lesions. Red clover lesions appear still slower, are usually larger and contain a large breeding population. Histological reactions, therefore, largely coincide with nematode development.The endoradicular effects are apparently instrumental in the nematode suppression by Tagetes spp. They are incorporated in or super-imposed upon the common plant-nematode relationships which are different for each association and may therefore influence the result. It is suggested that the special nematicidal principle in Tagetes is made up of more components, of which thiophenes recorded up to now from T. erecta , and that the components or their relative weights vary between Tagetes spp. Some other Compositae related closely to Tagetes spp. were also found to be effective against P. penetrans and also contained the same thiophenes as found in Tagetes or hitherto unidentified active principles. T. patula is probably superior to other Tagetes spp. in its effect against P. penetrans because necrosis appears earlier and more acutely. This may be a consequence of more rapid intoxication of the nematodes, as in root extracts.Ectoparasitic cortex feeders of the genus Tylenchorynchus are affected less and root-vessel feeders are evidently not influenced by the nematicidal principles in Tagetes roots, probably because they do not undergo the same type or degree of contact with the Tagetes tissue. The Tagetes effect, therefore, seems to be generally strong for endoparasitic root nematodes or cortical feeders, but varies even within this group with the different plant-nematode associations.The agricultural value of Tagetes as a source of organic matter, stains, therapeutics, or other chemicals, and as ornamentals is limited and has up to now supported only small-scale cultivation. The use of Tagetes crops for suppression of plant nematodes and the marked growth improvement obtained in main crops, is handicapped by the lack of value of the crop. Furthermore Tagetes spp. and cultivars are limited in their agricultural applicability. Nematode suppression is the primary determinant of their practical value so that T. patula particularly the cultivars Golden Harmony or Harmony are recommended. They are more effective against Pratylenchus and at least as effective against Meloidogyne populations, and appear to be as good or better than other Tagetes cultivars for growing and handling as a crop. Seed characteristics, rapid development as an autumn crop and winter-hardiness could all be improved, whereas a search for profitable use of the crop is desirable. Breeding of Tagetes spp. have resulted in a great assortment of ornamentals, and prospects seem good for the plant breeder of combining strong nematicidal effect with useful agricultural properties.The desired characters will depend on the type of agriculture. Tagetes grown as a full-season crop has been found effective by several workers and may cause striking growth and yield increments, it may find use in areas where valuable main crops are grown and where land rent is low, as in some tropical and subtropical countries. Tagetes as a spring crop does not seem promising in temperate climates due to its slow seedling growth. Even when sown densely, 10 cm apart on 15 May, minimum densities of Pratylenchus could be achieved only after about 2 months; these densities appear later when greater plant distances are taken (Fig. 10). Tagetes grown as an autumn crop has better possibilities. The degree of development of autumn Tagetes is evidently critical for its effect, and this varies strongly from year to year. Autumn Tagetes may be very effective in nematode suppression as well as in yield increase of main plants in certain years (Tables 23, 24 and 25), but the earliest sowing dates after an early pea crop varied in the period 1961-1968 from 6 July to 14 August, and a good development was obtained in only 4 out of 8 years (Table 6). Continuous autumn Tagetes , however, may be effective despite failure in certain years. Autumn Tagetes should be sown not later than the end of July and 10 rather than 25 cm apart is advisable.Simultaneous culture of Tagetes with a main crop appeared to be effective around and between trees and woody ornamentals and may be promising in more cases, especially since sowing at 60 cm apart completely suppressed nematodes, though slower than when closer sown. Sowing between rows is practized incidentally (Plate IIC and D). Sowing under cereals and other high crops to give the crop a quicker start after harvesting the main crop has not so far been promising, because only few weak plants survived.Autumn application after a main crop and application between rows of a main crop may be promising under certain conditions. The technical possibilities increase when climatic conditions allow a better growth in the autumn or when agricultural systems allow a full-season crop.Yield of main crops after Tagetes may be increased by nematological or other growth factors or both. Tagetes promoted growth of apple seedlings in soil with P. penetrans to 167 % of fallow infested soil, though Tagetes decreased growth in uninfested soil; allowance for the nitrogen used by Tagetes would lead to a higher growth promotion in infested soil (Table 27). The decline in nematode population continued markedly after the growth of Tagetes had been disrupted by removal of the tops. Direct mulch with a natural dosage of Tagetes roots suppresses P. penetrans much better than other mulches or fallow. It is probable that there is a significant aftereffect also under field conditions. Leaf mulches were also effective in suppressing P. penetrans , but Tagetes leaves were less effective than apple and no specific nematicidal effect of Tagetes leaves could be demonstrated. The results in uninfested soil show that cultivation of Tagetes and removal of tops decreases soil fertility in unfertilized soil (Table 27). The addition of root or leaf mulch generally increases soil fertility and therefore growth of test plants (Tables 30, 31, 32). These effects, however, are unspecific, and would have been smaller in fertilized soils, uninfested or infested. The dominance of nematicidal over non- nematicidal effects of Tagetes is clear for the growth of apple seedlings in soil infested with P. penetrans . The relative weight of these factors may of course be different in other plant nematode relationships.Our results, therefore, confirm or substantiate the marked Tagetes effect, but also the variation between plant-nematode relationships. The Tagetes effect is exceptional in nature, though not completely restricted to the genus Tagetes because it does occur in some genera ofthe Heleniae. The effect appears to be centred inside the roots and is evidently correlated with the presence in these plants of strongly nematicidal thiophenes, which are rare elsewhere in nature. The fact that it is conspicuous against endoparasitic polyphagous root nematodes and not against epidermal and vessel feeders is understandable from the difference in parasitic way of life.Histopathological reactions to nematode invasion vary between Tagetes spp. This may explain the stronger effect of T. patula on P. penetrans than of other Tagetes spp. The concept of nematode intoxication by nematicidal thiophenes and histopathological resistance phenomena may be interrelated and are not necessarily contradictory.Use of Tagetes spp. often markedly increases yield of main crops, but is limited by type of agriculture. In temperate regions Tagetes grown simultaneously with the main crop or grown in the autumn after the main crop is considered promising. Slow seedling growth, high light requirement and frost susceptibility of available cultivars are disadvantages. Tagetes has better prospects in tropical and subtropical agriculture

Quantum Reinforcement Learning for Dynamic Spectrum Access in Cognitive Radio Networks

Abstract This thesis proposes Quantum Reinforcement Learning (QRL) as an improvement to conventional reinforcement learning-based dynamic spectrum access used within cognitive radio networks. The aim is to overcome the slow convergence problem associated with exploration within reinforcement learning schemes. A literature review for the background of the carried out research work is illustrated. Review of research works on learning-based assignment techniques as well as quantum search techniques is provided. Modelling of three traditional dynamic channel assignment techniques is illustrated and the advantage characteristic of each technique is discussed. These techniques have been simulated to provide a comparison with learning based techniques, including QRL. Reinforcement learning techniques are used as a direct comparison with the Quantum Reinforcement Learning approaches. The elements of Quantum computation are then presented as an introduction to quantum search techniques. The Grover search algorithm is introduced. The algorithm is discussed from a theoretical perspective. The Grover algorithm is then used for the first time as a spectrum allocation scheme and compared to conventional schemes. Quantum Reinforcement Learning (QRL) is introduced as a natural evolution of the quantum search. The Grover search algorithm is combined as a decision making mechanism with conventional Reinforcement Learning (RL) algorithms resulting in a more efficient learning engine. Simulation results are provided and discussed. The convergence speed has been significantly increased. The beneficial effects of Quantum Reinforcement Learning (QRL) become more pronounced as the traffic load increases. The thesis shows that both system performance and capacity can be improved. Depending on the traffic load, the system capacity has improved by 9-84% from a number of users supported perspective. It also demonstrated file delay reduction for up to an average of 26% and 2.8% throughput improvement