Bounding Worst-Case Response Times of Tasks under PIP

Schedulability theory in real-time systems requires prior knowledge of the worst-case execution time (WCET) of every task in the system. One method to determine the WCET is known as static timing analysis. Determination of the priorities among tasks in such a system requires a scheduling policy, which could be either preemptive or nonpreemptive. While static timing analysis and data cache analysis are simplified by using a fully non-preemptive scheduling policy, it results in decreased schedulability. In prior work, a methodology was proposed to bound the data-cache related delay for real-time tasks that, beside having a non-preemptive region (critical section), can otherwise be scheduled preemptively. While the prior approach improves schedulability in comparison to fully non-preemptive methods, it is still conservative in its approach due to its fundamental assumption that a task executing in a critical section may not be preempted by any other task. In this paper, we propose a methodology that incorporates resource sharing policies such as the Priority Inheritance Protocol (PIP) into the calculation of data-cache related delay. In this approach, access to shared resources, which is the primary reason for critical sections within tasks, is controlled by the resource sharing policy. In addition to maintaining correctness of access, such policies strive to limit resource access conflicts, thereby improving the responsiveness of tasks. To the best of our knowledge, this is the first framework that integrates data-cache related delay calculations with resource sharing policies in the context of real-time systems

Bounding the Effects of Resource Access Protocols on Cache Behavior

The assumption of task independence has long been consubstantial with the formulation of many schedulability analysis techniques. That assumption is evidently advantageous for the mathematical formulation of the analysis equations, but ill fit to capture the actual behavior of the system. Resource sharing is one of the system design dimensions that break the assumption of task independence. By shaking the very foundations of the real-time analysis theory, the advent of multicore systems has caused resurgence of interest in resource sharing and synchronization protocols, and also dawned the fact that the assumption of task independence may be forever broken. Research in cache-aware schedulability analysis instead has paid very little attention to the impact that synchronization protocols may have on cache behavior. A blocked task may in fact incur time penalties similar in kind to those caused by preemption, in that some useful code or data already loaded in the cache may be evicted while the task is blocked. In this paper we characterize the sources of cache-related blocking delay (CRBD). We then provide a bound on the CRBD for three synchronization protocols of interest. The comparison between these bounds provides striking evidence that an informed choice of the synchronization protocol helps contain the perturbing effects of blocking on the cache state

Response-Time Analysis for Non-Preemptive Global Scheduling with {FIFO} Spin Locks

Motivated by the lack of response-time analyses for non-preemptive global scheduling that consider shared resources, this paper provides such an analysis for global job-level fixed-priority (JLFP) scheduling policies and FIFO-ordered spin locks. The proposed analysis computes response-time bounds for a set of resource-sharing jobs subject to release jitter and execution-time uncertainties by implicitly exploring all possible execution scenarios using state-abstraction and state-pruning techniques. A large-scale empirical evaluation of the proposed analysis shows it to be substantially less pessimistic than simple execution-time inflation methods, thanks to the explicit modeling of contention for shared resources and scenario-aware blocking analysis

A FIFO Spin-based Resource Control Framework for Symmetric Multiprocessing

Managing shared resources in multiprocessor real-time systems can often lead to considerable schedulability sacrifice, and currently there exist no optimal multiprocessor resource sharing solutions. In addition, the choice of task mapping and priority ordering algorithms also has a direct impact on the efficiency of multiprocessor resource sharing. This thesis argues that instead of adopting a single resource sharing protocol with the traditional task mapping (e.g., the task allocation schemes that are based on utilisation only) and priority ordering (e.g., the Deadline Monotonic Priority Ordering) algorithms, the schedulability loss for managing shared resources on multiprocessors can be effectively reduced by applying a combination of appropriately chosen resource sharing protocols with new resource-oriented task allocation schemes and a new search-based priority ordering algorithm (which are independent from multiprocessor resource sharing protocols and the corresponding schedulability tests). In this thesis, a Flexible Multiprocessor Resource Sharing (FMRS) framework is proposed that aims to provide feasible resource sharing, task allocation and priority assignment solutions to fully-partitioned systems with shared resources, where each resource is controlled by a designated locking protocol. To achieve this, the candidate resource sharing protocols for this framework are firstly determined with a new schedulability test developed to support the analysis of systems with multiple locking protocols in use. Then, besides the existing algorithms, three new resource-orientated task allocation schemes and a search-based priority ordering algorithm are developed for the FMRS framework as the task mapping and priority ordering solutions. The choices of which locking protocols, task allocation and priority ordering algorithm should be adopted to a given system are determined off-line via a genetic algorithm. As demonstrated by evaluations, the FMRS framework can facilitate multiprocessor resource sharing and has a better performance than the traditional resource control and task scheduling techniques for fully-partitioned systems

Composition and synchronization of real-time components upon one processor

Many industrial systems have various hardware and software functions for controlling mechanics. If these functions act independently, as they do in legacy situations, their overall performance is not optimal. There is a trend towards optimizing the overall system performance and creating a synergy between the different functions in a system, which is achieved by replacing more and more dedicated, single-function hardware by software components running on programmable platforms. This increases the re-usability of the functions, but their synergy requires also that (parts of) the multiple software functions share the same embedded platform. In this work, we look at the composition of inter-dependent software functions on a shared platform from a timing perspective. We consider platforms comprised of one preemptive processor resource and, optionally, multiple non-preemptive resources. Each function is implemented by a set of tasks; the group of tasks of a function that executes on the same processor, along with its scheduler, is called a component. The tasks of a component typically have hard timing constraints. Fulfilling these timing constraints of a component requires analysis. Looking at a single function, co-operative scheduling of the tasks within a component has already proven to be a powerful tool to make the implementation of a function more predictable. For example, co-operative scheduling can accelerate the execution of a task (making it easier to satisfy timing constraints), it can reduce the cost of arbitrary preemptions (leading to more realistic execution-time estimates) and it can guarantee access to other resources without the need for arbitration by other protocols. Since timeliness is an important functional requirement, (re-)use of a component for composition and integration on a platform must deal with timing. To enable us to analyze and specify the timing requirements of a particular component in isolation from other components, we reserve and enforce the availability of all its specified resources during run-time. The real-time systems community has proposed hierarchical scheduling frameworks (HSFs) to implement this isolation between components. After admitting a component on a shared platform, a component in an HSF keeps meeting its timing constraints as long as it behaves as specified. If it violates its specification, it may be penalized, but other components are temporally isolated from the malignant effects. A component in an HSF is said to execute on a virtual platform with a dedicated processor at a speed proportional to its reserved processor supply. Three effects disturb this point of view. Firstly, processor time is supplied discontinuously. Secondly, the actual processor is faster. Thirdly, the HSF no longer guarantees the isolation of an individual component when two arbitrary components violate their specification during access to non-preemptive resources, even when access is arbitrated via well-defined real-time protocols. The scientific contributions of this work focus on these three issues. Our solutions to these issues cover the system design from component requirements to run-time allocation. Firstly, we present a novel scheduling method that enables us to integrate the component into an HSF. It guarantees that each integrated component executes its tasks exactly in the same order regardless of a continuous or a discontinuous supply of processor time. Using our method, the component executes on a virtual platform and it only experiences that the processor speed is different from the actual processor speed. As a result, we can focus on the traditional scheduling problem of meeting deadline constraints of tasks on a uni-processor platform. For such platforms, we show how scheduling tasks co-operatively within a component helps to meet the deadlines of this component. We compare the strength of these cooperative scheduling techniques to theoretically optimal schedulers. Secondly, we standardize the way of computing the resource requirements of a component, even in the presence of non-preemptive resources. We can therefore apply the same timing analysis to the components in an HSF as to the tasks inside, regardless of their scheduling or their protocol being used for non-preemptive resources. This increases the re-usability of the timing analysis of components. We also make non-preemptive resources transparent during the development cycle of a component, i.e., the developer of a component can be unaware of the actual protocol being used in an HSF. Components can therefore be unaware that access to non-preemptive resources requires arbitration. Finally, we complement the existing real-time protocols for arbitrating access to non-preemptive resources with mechanisms to confine temporal faults to those components in the HSF that share the same non-preemptive resources. We compare the overheads of sharing non-preemptive resources between components with and without mechanisms for confinement of temporal faults. We do this by means of experiments within an HSF-enabled real-time operating system

Pushing the envelope for estimating poses and actions via full 3D reconstruction

Estimating poses and actions of human bodies and hands is an important task in the computer vision community due to its vast applications, including human computer interaction, virtual reality and augmented reality, medical image analysis. Challenges: There are many in-the-wild challenges in this task (see chapter 1). Among them, in this thesis, we focused on two challenges which could be relieved by incorporating the 3D geometry: (1) inherent 2D-to-3D ambiguity driven by the non-linear 2D projection process when capturing 3D objects. (2) lack of sufficient and quality annotated datasets due to the high-dimensionality of subjects' attribute space and inherent difficulty in annotating 3D coordinate values. Contributions: We first tried to jointly tackle the 2D-to-3D ambiguity and insufficient data issues by (1) explicitly reconstructing 2.5D and 3D samples and use them as new training data to train a pose estimator. Next, we tried to (2) encode 3D geometry in the training process of the action recognizer to reduce the 2D-to-3D ambiguity. In appendix, we proposed a (3) new hand pose synthetic dataset that can be used for more complete attribute changes and multi-modal experiments in the future. Experiments: Throughout experiments, we found interesting facts: (1) 2.5D depth map reconstruction and data augmentation can improve the accuracy of the depth-based hand pose estimation algorithm, (2) 3D mesh reconstruction can be used to generate a new RGB data and it improves the accuracy of RGB-based dense hand pose estimation algorithm, (3) 3D geometry from 3D poses and scene layouts could be successfully utilized to reduce the 2D-to-3D ambiguity in the action recognition problem.Open Acces

Deep Reinforcement Learning for 3D-based Object Grasping

Nowadays, collaborative robots based on Artificial Intelligence algorithms are very common to see in workstations and laboratories and they are expected to help their human colleagues in their everyday work. However, this type of robots can also assist in a domestic home, in tasks such as separate and organizing cutlery objects, but for that they need an algorithm to tell them which object to grasp and where to it. The main focus of this thesis is to create or improve an existing algorithm based on a Deep Reinforcement Learning for 3D-based Object Grasping, aiming to help collaborative robots on such tasks. Therefore, this work aims to present the state of the art and the study carried out, that enables the implementation of the proposed model that will help such robots to detect, grasp and separate each type of cutlery objects and consecutive experiments and results, as well as the retrospective of all the work done.Hoje em dia, ouve-se falar mais de robôs e do crescimento da robótica do que se ouviria há duas décadas atrás. A indústria da robótica tem vindo a evoluir imenso e a prova disso é a existência de robôs em estações de trabalho e laboratórios, cujo seu propósito é colaborar nas tarefas dos seus colegas trabalhadores humanos. A este tipo de robôs dá-se o nome de Cobot ou robô colaborativo. Estes robôs têm de suporte algoritmos da Inteligência Artificial para os ajudar a tomar as decisões mais corretas nas tarefas que têm de desempenhar. Contudo, este tipo de robôs já começa a ser adotado para tarefas domésticas. O tema desta dissertação envolve três grandes áreas: Inteligência Artificial, Visão Computacional e Robótica e tem como principal objetivo o desenvolvimento de um algoritmo de Aprendizagem por Reforço, que dê suporte a um robô universal, versão 3, na tomada de decisões para apanhar e separar objetos de cozinha por tipo. Assim sendo optou-se pelo uso de um algoritmo já desenvolvido, chamado Visual Pushing-for-Grasping, que permite simular robôs colaborativos a empurrar e apanhar objetos. Todavia, os objetos utilizados por este algoritmo em simulação não eram objetos de cozinha e o algoritmo apenas realiza apanha de objetos sem realizar a separação dos mesmos. Como tal, propomos uma nova abordagem com base no algoritmo anteriormente referido, e que passará a utilizar modelos 3D de objetos de cozinha, fará a deteção do tipo de objeto no cenário com recurso a um modelo de deteção de objetos exterior ao algoritmo base e que procederá à separação dos objetos por tipo. Os resultados experimentais permitem concluir que esta nova abordagem ainda precisa de ser melhorada, contudo e por ser uma abordagem nova tanto no ramo da Robótica como no ramo da Inteligència Artificial, para uso com o robôs universais da versão 3, afirmamos que os resultados estão melhores do que o esperado e expectamos que um dia esta possa ser aplicada a um robô físico em contexto real