Barbalace, Antonio

Boncagni, L.

Carvalho, B. B.

Carvalho, P. J.

De Tommasi, G.

Fernandes, H.

Manduchi, G.

McCullen, P.

Neto, A.

Sartori, F.

Stephen, A.

Valcárcel, D. F.

Vitelli, R.

Zabeo, L.

The Multi-threaded Application Real-Time executor (MARTe) is a C++framework that provides a development environment for the design and deployment of real-time applications, e.g. control systems. The kernel of MARTe comprises a set of data-driven independent blocks, connected using a shared bus. This modular design enforces a clear boundary between algorithms, hardware interaction and system configuration.The architecture, being multi-platform, facilitates the test and commissioning of new systems, enabling the execution of plant models in offline environments and with the hardware-in-the-loop, whilst also providing a set of non-intrusive introspection and logging facilities. Furthermore, applications can be developed in non real-time environments and deployed in a real-time operating system, using exactly the same code and configuration data. The framework is already being used in several fusion experiments, with control cycles ranging from 50 microseconds to 10 milliseconds exhibiting jitters of less than 2%, using VxWorks®, RTAI or Linux. Codes can also be developed and executed in Microsoft Windows® and Solaris®. This paper discusses the main design concepts of MARTe, in particular the architectural choices which enabled the combination of real-time accuracy, performance and robustness with complex and modular data driven applications

Edinburgh Research Explorer

     Edinburgh Research Explorer                                      MARTe Framework: A Middleware for Real-Time ApplicationsDevelopmentCitation for published version:Neto, A, Carvalho, BB, Carvalho, PJ, Fernandes, H, Valcárcel, DF, Sartori, F, Barbalace, A, Manduchi, G,Boncagni, L, De Tommasi, G, McCullen, P, Stephen, A, Vitelli, R, Zabeo, L & JET EFDA 2011, MARTeFramework: A Middleware for Real-Time Applications Development. in ICALEPCS2011 Contributions to theProceedings. Joint Accelerator Conferences Website, pp. 1277-1280, 13th International Conference onAccelerator and Large Experimental Physics Control Systems, Grenoble, France, 10/10/11.Link:Link to publication record in Edinburgh Research ExplorerDocument Version:Publisher's PDF, also known as Version of recordPublished In:ICALEPCS2011 Contributions to the ProceedingsGeneral rightsCopyright for the publications made accessible via the Edinburgh Research Explorer is retained by the author(s)and / or other copyright owners and it is a condition of accessing these publications that users recognise andabide by the legal requirements associated with these rights.Take down policyThe University of Edinburgh has made every reasonable effort to ensure that Edinburgh Research Explorercontent complies with UK legislation. If you believe that the public display of this file breaches copyright pleasecontact openaccess@ed.ac.uk providing details, and we will remove access to the work immediately andinvestigate your claim.Download date: 02. Jan. 2020MARTE FRAMEWORK: A MIDDLEWARE FOR REAL-TIMEAPPLICATIONS DEVELOPMENTA. Neto, D. Alves, B.B. Carvalho, P.J. Carvalho, H. Fernandes, D.F. Valca´rcel,Associac¸a˜o EURATOM/IST, Instituto de Plasmas e Fusa˜o Nuclear - Laborato´rio Associado,Instituto Superior Te´cnico, Universidade Te´cnica de Lisboa,1049-001 Lisboa, PortugalF. Sartori, Fusion for Energy, Barcelona, SpainA. Barbalace, G. Manduchi, Euratom-ENEA Association, Consorzio RFX,35127 Padova, ItalyL. Boncagni, Associazione EURATOM-ENEA sulla Fusione, C.R. ENEA Frascati,I-00044 Frascati-Rome, ItalyG. De Tommasi, Associazione EURATOM-ENEA-CREATE,Via Claudio 21, 80125, Napoli, ItalyP. McCullen, A. Stephen, EURATOM-CCFE Fusion Association, Culham Science Centre,Abingdon OX14 3DB, United KingdomR. Vitelli, Universita` di Roma, Tor Vergata, Via del Politecnico 1-00133, Roma, ItalyL. Zabeo, ITER Organisation, Cadarache, FranceJET EFDA Contributors∗AbstractThe Multi-threaded Application Real-Time executor(MARTe) is a C++ framework that provides a developmentenvironment for the design and deployment of real-time ap-plications, e.g. control systems. The kernel of MARTecomprises a set of data-driven independent blocks, con-nected using a shared bus. This modular design enforcesa clear boundary between algorithms, hardware interactionand system configuration.The architecture, being multi-platform, facilitates thetest and commissioning of new systems, enabling the exe-cution of plant models in oﬄine environments and with thehardware-in-the-loop, whilst also providing a set of non-intrusive introspection and logging facilities. Furthermore,applications can be developed in non real-time environ-ments and deployed in a real-time operating system, usingexactly the same code and configuration data.The framework is already being used in several fu-sion experiments, with control cycles ranging from 50 mi-croseconds to 10 milliseconds exhibiting jitters of less than2%, using VxWorks R©, RTAI or Linux. Codes can alsobe developed and executed in Microsoft Windows R© andSolaris R©.This paper discusses the main design concepts ofMARTe, in particular the architectural choices which en-abled the combination of real-time accuracy, performanceand robustness with complex and modular data driven ap-plications.∗ See the Appendix of F. Romanelli et al., Proceedings of the 23rdIAEA Fusion Energy Conference 2010, Daejeon, KoreaINTRODUCTIONMARTe [1] is a framework tailored at the design anddevelopment of real-time control systems. The kernelof MARTe uses a C++ multi-platform library namedBaseLib2 [2]. The main ideas behind the original designof the framework were the modularity and portability of itsapplications. In particular, a strong eﬀort was made in or-der to allow a robust simulation environment that allowsboth the models to be simulated with the hardware in theloop and the control algorithms to be validated oﬄine.This is achieved by providing a clear developmentboundary between the algorithms, hardware and systemconfiguration. Being multi-platform it also allows to debugand develop in non-real-time environments, where betterdeveloping tools are usually available. Currently the frame-work runs in Linux, Linux with RTAI [3], VxWorks R© forPowerPC R©, Solaris R© and Microsoft R© Windowstm.The framework components are configured using a com-mon language, designed to be as simple as possible, butcomplete enough to provide a clear way of describing theproblem. The structure is similar to XML, where the syn-tax rules are validated by a BaseLib2 parser, whereas theactual validity of the arguments is performed by the com-ponent (i.e. no validation schema is available). An exampleof a configuration is shown in Listing 1.The configuration file is translated into a database ofnamed objects that can be browsed using the object ad-dresses in the database. By parsing the configuration file,the framework automatically creates and configures in-stances of all the declared objects. A messaging mech-anism uses the database to provide a standard interfacefor communication between objects. This is the preferred, Culham Science Centre, OX14 3DB, Abingdon, UKProceedings of ICALEPCS2011, Grenoble, France THDAULT06Embedded + realtime software 1277 Copyrightc ○2011bytherespectiveauthors—ccCreativeCommonsAttribution3.0(CCBY3.0)+H t t p S e r v e r = {C l a s s = H t t p S e r v i c eP o r t = 8084}+MARTe = {C l a s s = MARTeContainer+RTThread1 = {C l a s s = RealTimeThread+SurfaceTemperature WOPL 14 = {C l a s s = S u r f a c e T e m p e r a t u r e 1 D C a l c u l a t i o nS p e c i f i c H e a t = 1 . 925000 e+03T h e r m a l C o n d u c t i v i t y = 1 . 900000 e+02Del taT = 0 . 0 1N s l i c e s = 40Inpu tS igna l N a m e s = {0 = {Q WOPL 14 }}Outpu tS igna lNam es = {0 = { SurfaceTemperature WOPL 14 }}}+D i v e r t o r T h e r m a l C a l c u l a t i o n s = {. . .W a l l s P o w e r P a r t i t i o n C l a s s = {Type = W a l l s P o w e r P a r t i t i o n C l a s sP a r t i t i o n T a b l e = {0 = {0 = { 0 . 3 0 . 3 0 . 3 0 . 3 }1 = { 0 . 7 0 . 7 0 . 7 0 . 7 }}. .}. . .}}. . .Listing 1: A small fraction of a configuration file from thereal-system responsible for the JET plasma wall load pro-tection system (WALLS).and protocols.MAIN COMPONENTSA MARTe application is designed by configuring andconnecting a series of blocks named Generic ApplicationModules (GAM). These modules contain an entry point toreceive data driven configuration and a set of data-drivenoptional input and output channels to interface with otherGAMs. Each of these channels has a unique name and isconnected to a generic memory data pipeline, named Dy-namic Data Buﬀer (DDB). Before starting the execution ofthe application, MARTe guarantees the coherency of theDDB by checking that all GAMs have the requested inputsbeing produced by another GAM. This scheme enables todesign interchangeable and generic modules, that can beused in diﬀerent projects without knowing any details orimposing any restrictions in the data producer. Moreover,this is also the key design concept which enables to replacea part of the system by a set of simulation GAMs, with-out changing the other modules, a very important featurewhen testing and designing a new control system, beforeintroducing the hardware in the loop, and later in the com-missioning of new hardware if good models of the plantare available. An example is shown in Fig. 1. A large set ofgeneric GAMs is available to be used in any MARTe appli-cation (e.g. PID, waveform generation, live data view, datacollection, data statistics).PersistenceControllerReference gen.ModelTiming Signal 2Signal 1Signal 3Signal NHardware I/OPersistenceControllerReference gen.Hardware I/OTiming Signal 2Signal 1Siganl 2Signal 4Figure 1: Example of a set of GAMs connected to theDDB. A timing and an hardware GAM provide the I/O in-terface to the outside world, whereas a generic waveformGAM inputs the reference for a PID controller. Finally, theoutput is sent to a DAC and the data is stored for analysis bya collection GAM. The picture in the right shows how thesame system can be developed using a model of the plant.It should be noticed that the reference generation and thecontroller GAM are not aware of the changes in the dataproviders and data consumers.A special GAM, named IOGAM, enables the connec-tion of any hardware to the DDB, as long as a MARTehigh level driver is developed to provide the connectionbetween the IOGAM and the hardware interface (usuallythrough an operating system low level driver). GAMs aresequentially executed, at a given frequency, by a real-timethread. Several real-time threads can be connected, bothsynchronously and asynchronously, and executed in paral-lel.A real-time thread cycle is triggered by an entity namedexternal time triggering service (ETTS). The ETTS is con-nected to a time provider and checks that the current time isa multiple of the configured MARTe cycle time. When thiscondition holds true a new cycle is signalled. Two typesof ETTS are available: one based in hardware interruptsand another based in the polling of a resource (e.g. sharedmemory entry). The synchronisation scheme will issue analarm if a timeout occurs, either due to a slow execution ofthe control cycle (e.g. a GAM is consuming too much timeand the cycle finishes after its period has already elapsed)or if the jitter in the timing source is very high. MARTemechanism to interface MARTe objects with other systemsTHDAULT06 Proceedings of ICALEPCS2011, Grenoble, France1278Copyrightc ○2011bytherespectiveauthors—ccCreativeCommonsAttribution3.0(CCBY3.0)Embedded + realtime softwareprovides some ready to be used synchronisation schemesbased in CPU timers and in network inputs.Taking advantage of the new multi-core processors, theframework enables all of its tasks to be allocated to a spe-cific subset of cores. In Linux, using the special isolcpuskernel parameter, it enables to have an isolated real-timeexecution environment for the real-time threads with jittersin the order of the microseconds [4].INTERFACING WITH MARTEAll MARTe objects are setup using the configurationmechanism described above. The framework makes no as-sumptions on how these files are produced and does not im-pose any communication protocol on how these should betransmitted. A standard C++ interface, with all the infras-tructure from the MARTe side implemented, is availablefor the development of new communication mechanisms.Currently, for systems outside JET, the only ready to beused configuration mechanism is based in the HTTP proto-col, although the preferred way of interfacing with MARTecomponents is to use the message mechanism introducedbefore. In order to integrate the framework in diﬀerent en-vironments, the required protocols are translated into mes-sages and broadcast to the destination objects. An exampleof such integration can be found in [5].The framework also provides its own HTTP server, ca-pable of browsing and introspecting any of the installedobjects. The server was designed to minimise any impactwith the real-time activities, by carefully executing all ofits activities in low-priority tasks and, in the case of multi-core environments, in cores not allocated for the real-timethreading. This scheme enables GAMs to publish run-timeexecution information about the internal state of algorithmsand data.MARTE SYSTEMSAs shown in Table 1, a large number of control systemsis already using MARTe to solve diﬀerent control prob-lems [6]. These systems are key to the operation of largeexperiments, and some have an active role providing a firstline of defence for the protection of the machine itself. Theframework is installed in diﬀerent experiments, with diﬀer-ent configuration and data retrieving protocols.The most frequent operating systems are Linux, runningin Intel R© and AMD R© processors with isolated cores, andVxWorks R© running in PowerPC R©. The frequency of exe-cution spans from 100 Hz to 20 kHz. Some modules, likethe waveform generation, data collection and data statisticsare shared by all projects.The COMPASS control system [8], depicted in Fig. 2,was the first to exploit the capability of running multi-ple real-time threads at diﬀerent frequencies. The fastestthread executes at 20 kHz and starts by reading the datafrom the analogue input channels, followed by a drift com-pensation (due to the use of integrators). Data is thenTable 1: Systems using MARTeName Cycle time O.S.JET Vertical 50 µs Linux-RTAIStabilisation [1]JET Error 200 µs VxWorks R©Field CorrectionCoils [7]COMPASS [8] 500 µs LinuxShape ControllerCOMPASS [8] 50 µs LinuxVertical StabilisationISTTOK [9] 100 µs LinuxTomographyFTU [10] 500 µs Linux-RTAIPlasma ControlRFX [11] 125 µs LinuxMHD ControlJET 2 ms VxWorks R©Real-time ProtectionSequencer [12]JET 10 ms LinuxVessel ThermalMap [13]JET 10 ms LinuxPlasma WallLoad Systemfiltered and sent to the second, slower, thread running at2 kHz. In parallel, the fastest thread controls the horizon-tal and vertical magnetic fields, while the slower thread isresponsible for the control of the plasma current and posi-tion. Since the hardware is the same used by the JET ver-tical stabilisation, the only GAMs that had to be developedconcerned the plasma control.CONCLUSIONSMARTe is C++ real-time framework designed for thedevelopment and deployment of control-systems. It is al-ready being used in several experiments to solve diﬀerentproblems that require diﬀerent operational frequencies andhardware interfaces. Currently it is only being used in mag-netic confinement nuclear fusion experiments, but there isnot reason why it cannot be deployed in any context wherea real-time control system is required.ACKNOWLEDGEMENTSThis work was supported by the European Communitiesunder the contract of Association between EURATOM/ISTand was carried out within the framework of the EuropeanProceedings of ICALEPCS2011, Grenoble, France THDAULT06Embedded + realtime software 1279 Copyrightc ○2011bytherespectiveauthors—ccCreativeCommonsAttribution3.0(CCBY3.0)DDBPlasma currentPlasma positionFast InputsTimingControllersDACsPersistenceStatisticsDDBOﬀset removalOutput to slowHardware I/OTimingControllersDACsPersistenceStatisticsFigure 2: The COMPASS plasma control system was thefirst system to exploit the multi real-time threading capa-bilities of MARTe. A fast thread (20 kHz) provides theADC data to a slower thread (2 kHz).Fusion Development Agreement. See the Appendix of F.Romanelli et al., Proceedings of the 23rd IAEA Fusion En-ergy Conference 2010, Daejeon, Korea. The views andopinions expressed herein do not necessarily reflect thoseof the European Commission.REFERENCES[1] A. Neto et al., “MARTe: A Multiplatform Real-TimeFramework”, IEEE Transactions on Nuclear Science, Vol.57, pp. 479-486, 2010.[2] G. De Tommasi et al., “A flexible software for real-timecontrol in nuclear fusion experiments”, Control EngineeringPractice, Vol. 14, pp. 1387-1393, 2006.[3] A. Neto et al., “Linux real-time framework for fusion de-vices”, Fusion Engineering and Design, Vol. 84, pp. 1408-1411, 2009.[4] D.F. Valcarcel et al., “The COMPASS Tokamak PlasmaControl Software Performance”, IEEE Transactions on Nu-clear Science, Vol. 58, pp. 1490-1496, 2011.[5] D.F. Valcarcel et al., “EPICS as a MARTe Configuration En-vironment”, IEEE Transactions on Nuclear Science, Vol. 58,pp. 1472-1476, 2011.[6] A. Neto et al., “A Survey of Recent MARTe Based Sys-tems”, IEEE Transactions on Nuclear Science, Vol. 58, pp.1482-1489, 2011.[7] D. Alves et al., “The new Error Field Correction Coil con-troller system in the Joint European Torus tokamak”, ac-cepted for publication in Fusion Engineering and Design.[8] D.F. Valcarcel et al., “Real-time software for the COMPASStokamak plasma control”, Fusion Engineering and Design,Vol. 85, pp. 470-473, 2010.[9] P.J. Carvalho et al.,“ISTTOK plasma control with the to-mography diagnostic”, Fusion Engineering and Design, Vol.85, pp. 266-271, 2010.[10] L. Boncagni et al., “First Steps in the FTU Migration To-wards a Modular and Distributed Real-Time Control Archi-tecture Based on MARTe”, IEEE Transactions on NuclearScience, Vol. 58, pp. 1778-1783, 2011.[11] A. Barbalace et al., “Concepts, Design, and Development ofa Multiplatform Framework for Real-Time Control in Nu-clear Fusion”, IEEE Transactions on Nuclear Science, Vol.57, pp. 688-695, 2010.[12] A. Stephen et al., “Centralised Coordinated Control To Pro-tect The JET ITER-like Wall”, ICALEPCS2011, Grenoble,France.[13] D. Alves et al., “The Software and Hardware ArchitecturalDesign of the Vessel Thermal Map Real-Time System inJET”, ICALEPCS2011, Grenoble, France.THDAULT06 Proceedings of ICALEPCS2011, Grenoble, France1280Copyrightc ○2011bytherespectiveauthors—ccCreativeCommonsAttribution3.0(CCBY3.0)Embedded + realtime software

MARTe Framework: A Middleware for Real-Time Applications Development

https://www.pure.ed.ac.uk/ws/files/124993737/MARTe_Framework_NETO_DOA12122019_VOR_CC_BY.pdf

MARTe Framework: A Middleware for Real-Time Applications Development

Abstract

Similar works

Full text

Available Versions

Edinburgh Research Explorer