R^3: On-device Real-Time Deep Reinforcement Learning for Autonomous Robotics

Autonomous robotic systems, like autonomous vehicles and robotic search and rescue, require efficient on-device training for continuous adaptation of Deep Reinforcement Learning (DRL) models in dynamic environments. This research is fundamentally motivated by the need to understand and address the challenges of on-device real-time DRL, which involves balancing timing and algorithm performance under memory constraints, as exposed through our extensive empirical studies. This intricate balance requires co-optimizing two pivotal parameters of DRL training -- batch size and replay buffer size. Configuring these parameters significantly affects timing and algorithm performance, while both (unfortunately) require substantial memory allocation to achieve near-optimal performance. This paper presents R^3, a holistic solution for managing timing, memory, and algorithm performance in on-device real-time DRL training. R^3 employs (i) a deadline-driven feedback loop with dynamic batch sizing for optimizing timing, (ii) efficient memory management to reduce memory footprint and allow larger replay buffer sizes, and (iii) a runtime coordinator guided by heuristic analysis and a runtime profiler for dynamically adjusting memory resource reservations. These components collaboratively tackle the trade-offs in on-device DRL training, improving timing and algorithm performance while minimizing the risk of out-of-memory (OOM) errors. We implemented and evaluated R^3 extensively across various DRL frameworks and benchmarks on three hardware platforms commonly adopted by autonomous robotic systems. Additionally, we integrate R^3 with a popular realistic autonomous car simulator to demonstrate its real-world applicability. Evaluation results show that R^3 achieves efficacy across diverse platforms, ensuring consistent latency performance and timing predictability with minimal overhead.Comment: Accepted by RTSS 202

Measurement-device-independent quantum key distribution with ensemble-based memories

Quantum memories are enabling devices for extending the reach of quantum key distribution (QKD) systems. The required specifications for memories are, however, often considered too demanding for available technologies. One can change this mindset by introducing memory-assisted measurement-device-independent QKD (MDI-QKD), which imposes less stringent conditions on the memory modules. It has been shown that, in the case of fast single-qubit memories, we can reach rates and distances not attainable by single no-memory QKD links. Single-qubit memories, such as single atoms or ions, have, currently, too slow of an access time to offer an advantage in practice. Here, we relax that assumption, and consider ensemble-based memories, which satisfy the main two requirements of having short access times and large storage-bandwidth products. Our results, however, suggest that the multiple-excitation effects in such memories can be so detrimental that they may wash away the scaling improvement offered by memory-equipped systems. We then propose an alternative setup that can in principle remedy the above problem. As a prelude to our main problem, we also obtain secret key generation rates for MDI-QKD systems that rely on imperfect single-photon sources with nonzero probabilities of emitting two photons

Self-service kiosk-based anamnesis system for emergency departments

Dissertação de mestrado integrado em Engenharia InformáticaEmergency departments have a higher number of visits compared to other hospital de partments. Technology has played a crucial role in promoting improvements in hospital management and clinical performance. The number of visits to emergency departments has increased considerably, giving rise to crowding situations that cause several adverse effects. This situation negatively affects the provision of emergency services, impairs the quality of health care and increases the time patients wait for medical check-up. One of the leading causes contributing to the crowding is the high number of patients with low severity clinical condition. These are referred to as non-urgent or inappropriate patients, whose clinical situation should be taken care through self-care or primary health care. It is the responsibility of the institutions to analyse and quantify the possible causes of crowding to find the best solution to mitigate the adverse effects caused. It is believed that non-urgent patients can use the time spent in the waiting room more productively, namely by using a self-service kiosk to which they can provide valuable information to facilitate and accelerate the clinical processing. This work proposes a solution to be used in the waiting room of emergency departments, which aims to reduce the period of medical check-up. The solution uses a self-service kiosk for the patient to provide relevant clinical data that would otherwise have to be collected by the physician during the clinical observation process. In particular, the kiosk will collect vital signs, past medical history, main complaint and usual medication. This data will be processed and provided to the physician in a structured and uniform way before each medical check-up. The primary purpose of this solution is to reduce the period of patients’ medical check-up and thus improve the response capacity of the emergency departments with the same resources. During the Master’s work period, an Android application was implemented for patients to enter the clinical data mentioned above, and a Web application for physicians to access it. Additionally, a data warehouse was implemented to store the data in a consolidated way to discover hidden relationships and patterns in the data. The first moment of evaluation, undertaken in a non-hospital facility, shows positive acceptability by participants, with a large majority considering the system user-friendly. Due to the pandemic, it was impossible to perform the second planned evaluation moment in a real emergency environment.Os serviços de urgência apresentam um número de visitas superior em comparação com outros serviços presentes nas instituições hospitalares. A afluência aos serviços de urgências tem vindo a aumentar consideravelmente, dando origem a situações de lotação que provocam diversos efeitos negativos nas instituições hospitalares. No geral, este fenômeno afeta negativamente a prestação dos serviços de urgência, prejudica a qualidade dos cuidados de saúde e faz aumentar o tempo que os doentes aguardam pela observação clínica na sala de espera. Uma das principais causas apontadas para o surgimento da lotação é o elevado número de doentes com condição clínica de baixa gravidade. Estes são designados como doentes não-urgentes ou inapropriados, cuja condição clínica poderia ser resolvida, idealmente, com recurso ao auto-cuidado ou a cuidados de saúde primários. É da responsabilidade das instituições analisar e quantificar as possíveis causas de lotação, de forma a encontrar a melhor solução para atenuar os efeitos negativos provocados. Acredita-se que os doentes não-urgentes tenham a capacidade de utilizar o tempo na sala de espera de forma mais produtiva, através da utilização de um quiosque self-service. Neste sentido e aliada à tecnologia, esta dissertação contextualiza uma solução para ser utilizada na sala de espera dos serviços de urgência, visando reduzir o período de observação clínico. Esta solução vem complementar a realização do procedimento inicial efetuado pelo médico, no consultório, através do uso de um quiosque. Assim, a recolha dos sinais vitais, história médica prévia, queixa principal e medicação habitual será efetuada pelos doentes no quiosque. Estes dados vão ser fornecidos de forma estruturada e organizada ao médico antes da realização da consulta. O objetivo principal desta solução é reduzir o período de observação clínico e assim melhorar a capacidade de resposta dos serviços de urgência com os mesmos recursos hospitalares. Durante o período da dissertação, foi implementada uma aplicação Android para os pacientes registarem os dados clínicos acima mencionados, e uma aplicação Web para os médicos acederem aos mesmos. Foi implementado também um data warehouse para a descoberta de relações e padrões escondidos nos dados. O primeiro momento de avaliação, realizado num ambiente não hospitalar, mostrou uma aceitabilidade positiva pelos partici pantes, com grande maioria a considerar o sistema user-friendly. Devido à pandemia, não foi possível realizar o segundo momento de avaliação planeado num serviço de urgências

Blockchained Post-Quantum Signatures

Inspired by the blockchain architecture and existing Merkle tree based signature schemes, we propose BPQS, an extensible post-quantum (PQ) resistant digital signature scheme best suited to blockchain and distributed ledger technologies (DLTs). One of the unique characteristics of the protocol is that it can take advantage of application-specific chain/graph structures in order to decrease key generation, signing and verification costs as well as signature size. Compared to recent improvements in the field, BPQS outperforms existing hash-based algorithms when a key is reused for reasonable numbers of signatures, while it supports a fallback mechanism to allow for a practically unlimited number of signatures if required. To our knowledge, this is the first signature scheme that can utilise an existing blockchain or graph structure to reduce the signature cost to one OTS, even when we plan to sign many times. This makes existing many-time stateful signature schemes obsolete for blockchain applications. We provide an open source implementation of the scheme and benchmark it

Bryn Mawr College College Catalogue and Calendar, 1978-1980

Volume contains calendars of undergraduate courses, calendars of The Graduate School of Social Work and Social Research, and calendars of The Graduate School of Arts and Sciences for 1978-79 and 1979-80.https://repository.brynmawr.edu/bmc_calendars/1040/thumbnail.jp

VIRTUAL MEMORY ON A MANY-CORE NOC

Many-core devices are likely to become increasingly common in real-time and embedded systems as computational demands grow and as expectations for higher performance can generally only be met by by increasing core numbers rather than relying on higher clock speeds. Network-on-chip devices, where multiple cores share a single slice of silicon and employ packetised communications, are a widely-deployed many-core option for system designers. As NoCs are expected to run larger and more complex programs, the small amount of fast, on-chip memory available to each core is unlikely to be sufficient for all but the simplest of tasks, and it is necessary to find an efficient, effective, and time-bounded, means of accessing resources stored in off-chip memory, such as DRAM or Flash storage. The abstraction of paged virtual memory is a familiar technique to manage similar tasks in general computing but has often been shunned by real-time developers because of concern about time predictability. We show it can be a poor choice for a many-core NoC system as, unmodified, it typically uses page sizes optimised for interaction with spinning disks and not solid state media, and transports significant volumes of subsequently unused data across already congested links. In this work we outline and simulate an efficient partial paging algorithm where only those memory resources that are locally accessed are transported between global and local storage. We further show that smaller page sizes add to efficiency. We examine the factors that lead to timing delays in such systems, and show we can predict worst case execution times at even safety-critical thresholds by using statistical methods from extreme value theory. We also show these results are applicable to systems with a variety of connections to memory

ON OPTIMIZATIONS OF VIRTUAL MACHINE LIVE STORAGE MIGRATION FOR THE CLOUD

Virtual Machine (VM) live storage migration is widely performed in the data cen- ters of the Cloud, for the purposes of load balance, reliability, availability, hardware maintenance and system upgrade. It entails moving all the state information of the VM being migrated, including memory state, network state and storage state, from one physical server to another within the same data center or across different data centers. To minimize its performance impact, this migration process is required to be transparent to applications running within the migrating VM, meaning that ap- plications will keep running inside the VM as if there were no migration operations at all. In this dissertation, a thorough literature review is conducted to provide a big picture of the VM live storage migration process, its problems and existing solutions. After an in-depth examination, we observe that a severe IO interference between the VM IO threads and migration IO threads exists and causes both types of the IO threads to suffer from performance degradation. This interference stems from the fact that both types of IO threads share the same critical IO path by reading from and writing to the same shared storage system. Owing to IO resource contention and requests interference between the two different types of IO requests, not only will the IO request queue lengthens in the storage system, but the time-consuming disk seek operations will also become more frequent. Based on this fundamental observation, this dissertation research presents three related but orthogonal solutions that tackle the IO interference problem in order to improve the VM live storage migration performance. First, we introduce the Workload-Aware IO Outsourcing scheme, called WAIO, to improve the VM live storage migration efficiency. Second, we address this problem by proposing a novel scheme, called SnapMig, to improve the VM live storage migration efficiency and eliminate its performance impact on user applications at the source server by effectively leveraging the existing VM snapshots in the backup servers. Third, we propose the IOFollow scheme to improve both the VM performance and migration performance simultaneously. Finally, we outline the direction for the future research work. Advisor: Hong Jian