On-Device Deep Learning Inference for System-on-Chip (SoC) Architectures

As machine learning becomes ubiquitous, the need to deploy models on real-time, embedded systems will become increasingly critical. This is especially true for deep learning solutions, whose large models pose interesting challenges for target architectures at the “edge” that are resource-constrained. The realization of machine learning, and deep learning, is being driven by the availability of specialized hardware, such as system-on-chip solutions, which provide some alleviation of constraints. Equally important, however, are the operating systems that run on this hardware, and specifically the ability to leverage commercial real-time operating systems which, unlike general purpose operating systems such as Linux, can provide the low-latency, deterministic execution required for embedded, and potentially safety-critical, applications at the edge. Despite this, studies considering the integration of real-time operating systems, specialized hardware, and machine learning/deep learning algorithms remain limited. In particular, better mechanisms for real-time scheduling in the context of machine learning applications will prove to be critical as these technologies move to the edge. In order to address some of these challenges, we present a resource management framework designed to provide a dynamic on-device approach to the allocation and scheduling of limited resources in a real-time processing environment. These types of mechanisms are necessary to support the deterministic behavior required by the control components contained in the edge nodes. To validate the effectiveness of our approach, we applied rigorous schedulability analysis to a large set of randomly generated simulated task sets and then verified the most time critical applications, such as the control tasks which maintained low-latency deterministic behavior even during off-nominal conditions. The practicality of our scheduling framework was demonstrated by integrating it into a commercial real-time operating system (VxWorks) then running a typical deep learning image processing application to perform simple object detection. The results indicate that our proposed resource management framework can be leveraged to facilitate integration of machine learning algorithms with real-time operating systems and embedded platforms, including widely-used, industry-standard real-time operating systems

Combined Scheduling of Time-Triggered Plans and Priority Scheduled Task Sets

© Owner/Author (2016). This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM SIGAda Ada Letters, 36(1), 68-76, http://dx.doi.org/10.1145/10.1145/2971571.2971580.[EN] Preemptive, priority-based scheduling on the one hand, and time-triggered scheduling on the other, are the two major techniques in use for development of real-time and embedded software. Both have their advantages and drawbacks with respect to the other, and are commonly adopted in mutual exclusion. In a previous paper, we proposed a software architecture that enables the combined and controlled execution of time-triggered plans and priority-scheduled tasks. The goal was to take advantage of the best of both approaches by providing deterministic, jitter-controlled execution of time-triggered tasks (e.g., control tasks), coexisting with a set of priority-scheduled tasks, with less demanding jitter requirements. In this paper, we briefly describe the approach, in which the time-triggered plan is executed at the highest priority level, controlled by scheduling decisions taken only at particular points in time, signalled by recurrent timing events. The rest of priority levels are used by a set of concurrent tasks scheduled by static or dynamic priorities. We also discuss several open issues such as schedulability analysis, use of the approach in multiprocessor architectures, usability in mixed-criticality systems and needed changes to make this approach Ravenscar compliant.This work has been partly supported by the Spanish Government’s project M2C2 (TIN2014-56158-C4-1-P-AR) and the European Commission’s project EMC2 (ARTEMIS-JU Call 2013 AIPP-5, Contract 621429).Real Sáez, JV.; Sáez Barona, S.; Crespo Lorente, A. (2016). Combined Scheduling of Time-Triggered Plans and Priority Scheduled Task Sets. Ada Letters. 36(1):68-76. https://doi.org/10.1145/2971571.2971580S6876361T. P. Baker and A. Shaw. The cyclic executive model and Ada. In Proceedings IEEE Real Time Systems Symposium 1988, Huntsville, Alabama, pages 120--129, 1988.P. Balbastre, I. Ripoll, J. Vidal, and A. Crespo. A Task Model to Reduce Control Delays. Real-Time Systems, 27(3):215--236, September 2004.A. Burns and R. Davis. Mixed Criticality Systems - A Review. Technical report, Depatment of Computer Science, University of York, 2013.A. Cervin. Integrated Control and Real-Time Scheduling. PhD thesis, Lund Institute of Technology, April 2003.R. Dobrin. Combining Offline Schedule Construction and Fixed Priority Scheduling in Real-Time Computer Systems. PhD thesis, Mälardalen University, 2005.S. Hong, X. Hu, and M. Lemmon. Reducing Delay Jitter of Real-Time Control Tasks through Adaptive Deadline Adjustments. In IEEE Computer Society, editor, 22nd Euromicro Conference on Real-Time Systems -- ECRTS, pages 229--238, 2010.J. W. S. Liu. Real-Time Systems. Prentice-Hall Inc., 2000.J. Palencia and M. González-Harbour. Schedulability Analysis for Tasks with Static and Dynamic Offsets. In 9th IEEE Real-Time Systems Symposium, 1998.M. J. Pont. The Engineering of Reliable Embedded Systems: LPC1769 edition. Number ISBN: 978-0-9930355-0-0. SafeTTy Systems Limited, 2014.J. Real and A. Crespo. Incorporating Operating Modes to an Ada Real-Time Framework. Ada Letters, 30(1):73--85, April 2010.J. Real, S. Sáez, and A. Crespo. Combining time-triggered plans with priority scheduled task sets. In M. Bertogna and L. M. Pinho, editors, Reliable Software Technologies -- Ada-Europe 2016, volume 9695 of Lecture Notes in Computer Science. Springer, June 2016.S. Sáez, J. Real, and A. Crespo. An integrated framework for multiprocessor, multimoded real-time applications. In M. Brorsson and L. Pinho, editors, Reliable Software Technologies -- Ada-Europe 2012, volume 7308, pages 18--34. Springer-Verlag, June 2012.S. Sáez, J. Real, and A. Crespo. Implementation of Timing-Event Anities in Ada/Linux. Ada Letters, 35(1), April 2015.A. J. Wellings and A. Burns. A Framework for Real-Time Utilities for Ada 2005. Ada Letters, XXVII(2), August 2007

A Hardware Time Manager Implementation for the Xenomai Real-Time Kernel of Embedded Linux

Nowadays, the use of embedded operating systems in different embedded projects is subject to a tremendous growth. Embedded Linux is becoming one of those most popular EOSs due to its modularity, efficiency, reliability, and cost. One way to make it hard real-time is to include a real-time kernel like Xenomai. One of the key characteristics of a Real-Time Operating System (RTOS) is its ability to meet execution time deadlines deterministically. So, the more precise and flexible the time management can be, the better it can handle efficiently the determinism for different embedded applications. RTOS time precision is characterized by a specific periodic interrupt service controlled by a software time manager. The smaller the period of the interrupt, the better the precision of the RTOS, the more it overloads the CPU, and though reduces the overall efficiency of the RTOS. In this paper, we propose to drastically reduce these overheads by migrating the time management service of Xenomai into a configurable hardware component to relieve the CPU. The hardware component is implemented in a Field Programmable Gate Array coupled to the CPU. This work was achieved in a Master degree project where students could apprehend many fields of embedded systems: RTOS programming, hardware design, performance evaluation, etc.Comment: Embed With Linux (EWiLi) workshop, Lorient : France (2012

A Comprehensive Experimental Comparison of Event Driven and Multi-Threaded Sensor Node Operating Systems

The capabilities of a sensor network are strongly influenced by the operating system used on the sensor nodes. In general, two different sensor network operating system types are currently considered: event driven and multi-threaded. It is commonly assumed that event driven operating systems are more suited to sensor networks as they use less memory and processing resources. However, if factors other than resource usage are considered important, a multi-threaded system might be preferred. This paper compares the resource needs of multi-threaded and event driven sensor network operating systems. The resources considered are memory usage and power consumption. Additionally, the event handling capabilities of event driven and multi-threaded operating systems are analyzed and compared. The results presented in this paper show that for a number of application areas a thread-based sensor network operating system is feasible and preferable

CSP channels for CAN-bus connected embedded control systems

Closed loop control system typically contains multitude of sensors and actuators operated simultaneously. So they are parallel and distributed in its essence. But when mapping this parallelism to software, lot of obstacles concerning multithreading communication and synchronization issues arise. To overcome this problem, the CT kernel/library based on CSP algebra has been developed. This project (TES.5410) is about developing communication extension to the CT library to make it applicable in distributed systems. Since the library is tailored for control systems, properties and requirements of control systems are taken into special consideration. Applicability of existing middleware solutions is examined. A comparison of applicable fieldbus protocols is done in order to determine most suitable ones and CAN fieldbus is chosen to be first fieldbus used. Brief overview of CSP and existing CSP based libraries is given. Middleware architecture is proposed along with few novel ideas

Towards an HLA Run-time Infrastructure with Hard Real-time Capabilities

Our work takes place in the context of the HLA standard and its application in real-time systems context. The HLA standard is inadequate for taking into consideration the different constraints involved in real-time computer systems. Many works have been invested in order to providing real-time capabilities to Run Time Infrastructures (RTI) to run real time simulation. Most of these initiatives focus on major issues including QoS guarantee, Worst Case Transit Time (WCTT) knowledge and scheduling services provided by the underlying operating systems. Even if our ultimate objective is to achieve real-time capabilities for distributed HLA federations executions, this paper describes a preliminary work focusing on achieving hard real-time properties for HLA federations running on a single computer under Linux operating systems. Our paper proposes a novel global bottom up approach for designing real-time Run time Infrastructures and a formal model for validation of uni processor to (then) distributed real-time simulation with CERTI

TASKers: A Whole-System Generator for Benchmarking Real-Time-System Analyses

Implementation-based benchmarking of timing and schedulability analyses requires system code that can be executed on real hardware and has defined properties, for example, known worst-case execution times (WCETs) of tasks. Traditional approaches for creating benchmarks with such characteristics often result in implementations that do not resemble real-world systems, either due to work only being simulated by means of busy waiting, or because tasks have no control-flow dependencies between each other. In this paper, we address this problem with TASKers, a generator that constructs realistic benchmark systems with predefined properties. To achieve this, TASKers composes patterns of real-world programs to generate tasks that produce known outputs and exhibit preconfigured WCETs when being executed with certain inputs. Using this knowledge during the generation process, TASKers is able to specifically introduce inter-task control-flow dependencies by mapping the output of one task to the input of another