A flexible readout board for HEP experiments

This thesis will present my contributions to the development of the PiLUP board along with a general overview of its features and capabilities. The PiLUP board is a general-purpose FPGA-based readout board for data acquisition systems under development by the University of Bologna and the Instituto Nazionale Fisica Nucleare (INFN) and intended for high energy physics experiments, where the sheer amount of data generated by detectors often requires custom hardware solutions. This board was initially proposed for the next upgrade of the ATLAS Pixel detector. In this context its purpose would be to interface the Front-End readout chip RD53A with the FELIX card and provide advanced testing features such as an emulator for the RD53A that will help the development of the other parts of the data acquisition chain. Nonetheless, since the early stages of development, the hardware has been designed to offer great flexibility so that the same hardware platform could be directly used in other applications. To this purpose an important feature of the board is the great extendibility offered by the presence of different interfaces, such as and 3 FMC connectors (two low density and one high density), a PCI Express x8 interface, gigabit ethernet and an integrated SFP connector. The computing power of the PiLUP is provided by of two FPGAs, a Zynq-7 SoC and a Kintex-7 produced by Xilinx, intended to be used in master-slave configuration. In this case the Zynq, with its dual-core ARM processor and the possibility to run an embedded linux distribution, would be used as main interface with the other functionalities in the board. The main objective of this thesis is the development of such software and firmware control infrastructure, starting from the firmware solutions for the inter-FPGA communication to the low-level software to control the system

PET System Synchronization and Timing Resolution Using High-Speed Data Links

Current PET systems with fully digital trigger rely on early digitization of detector signals and the use of digital processors, usually FPGAs, for recognition of valid gamma events on single detectors. Timestamps are assigned and later used for coincidence analysis. In order to maintain a decent timing resolution for events detected on different acquisition boards, it is necessary that local timestamps on different FPGAs be synchronized. Sub-nanosecond accuracy is mandatory if we want this effect to be negligible on overall timing resolution. This is usually achieved by connecting all boards to a common backplane with a precise clock delivery network; however, this approach forces a rigid structure on the whole PET system and may pose scalability problems. © 2006 IEEE.Manuscript received June 14, 2010; revised November 18, 2010; accepted March 31, 2011. Date of publication April 21, 2011; date of current version August 17, 2011. This work was supported in part by the Spanish Ministry of Science and Innovation under FPU Grant AP2006-04275 and CICYT Grant FIS2010-21216-C02-02.Aliaga Varea, RJ.; Monzó Ferrer, JM.; Spaggiari, M.; Ferrando Jódar, N.; Gadea Gironés, R.; Colom Palero, RJ. (2011). PET System Synchronization and Timing Resolution Using High-Speed Data Links. IEEE Transactions on Nuclear Science. 58(4):1596-1605. https://doi.org/10.1109/TNS.2011.2140130S1596160558

Timing Signals and Radio Frequency Distribution Using Ethernet Networks for High Energy Physics Applications

Timing networks are used around the world in various applications from telecommunications systems to industrial processes, and from radio astronomy to high energy physics. Most timing networks are implemented using proprietary technologies at high operation and maintenance costs. This thesis presents a novel timing network capable of distributed timing with subnanosecond accuracy. The network, developed at CERN and codenamed “White- Rabbit”, uses a non-dedicated Ethernet link to distribute timing and data packets without infringing the sub-nanosecond timing accuracy required for high energy physics applications. The first part of this thesis proposes a new digital circuit capable of measuring time differences between two digital clock signals with sub-picosecond time resolution. The proposed digital circuit measures and compensates for the phase variations between the transmitted and received network clocks required to achieve the sub-nanosecond timing accuracy. Circuit design, implementation and performance verification are reported. The second part of this thesis investigates and proposes a new method to distribute radio frequency (RF) signals over Ethernet networks. The main goal of existing distributed RF schemes, such as Radio-Over-Fibre or Digitised Radio-Over-Fibre, is to increase the bandwidth capacity taking advantage of the higher performance of digital optical links. These schemes tend to employ dedicated and costly technologies, deemed unnecessary for applications with lower bandwidth requirements. This work proposes the distribution of RF signals over the “White-Rabbit” network, to convey phase and frequency information from a reference base node to a large numbers of remote nodes, thus achieving high performance and cost reduction of the timing network. Hence, this thesis reports the design and implementation of a new distributed RF system architecture; analysed and tested using a purpose-built simulation environment, with results used to optimise a new bespoke FPGA implementation. The performance is evaluated through phase-noise spectra, the Allan-Variance, and signalto- noise ratio measurements of the distributed signals

Design, implementation and testing of SRAM based neutron detectors

Neutrons of thermal and high energies can change the value of a bit stored in a Static Random Access Memory (SRAM) memory chip. The effect is non destructive and linearly dependent on the amount of incoming particles, which makes it exploitable for use as a neutron detector. Detection is done by writing a known pattern to the memory and continuously reading it back checking for wrong values. As the SRAM memory is immune to gamma radiation it is ideal for use in for instance medical linear accelerators for detection of neutron dose to a patient. The intention of this work has been twofold: (1) Testing of different SRAM devices of different bit-sizes, manufacturers, feature sizes and voltages for their sensitivity to neutrons of different energies from thermal to high energies. (2) Design and implement detector hardware, firmware and its accompanying readout system for successful use in irradiation testing. The work has been done in close collaboration with Eivind Larsen, whose main contributions has been related to the nuclear physics aspect of the work in addition to arrangements in regard to beam setup and experimentation. Testing have been done at the Physikalisch-Technische Bundesanstalt (PTB) facility in Braunschweig Germany in a quasi-monochromatic neutron beam of 5:8MeV, 8:5MeV and 14:8MeV, finding a dependence of the sensitivity on the energy. In addition there have been testing conducted in the high energy hadron field at CERF at CERN, finding that by using the results from the other experiments an estimated range of the saturation cross section could be determined. Testing was also conducted at two occasions in the 29MeV proton beam at Oslo Cyclotron Laboratory (OCL) in Oslo Norway, where it was found that the detector could be used as a reference detector for beam monitoring and for beam profile characterization. The cross sections of the detectors were found to be comparable to the 14:8MeV cross section found at PTB. Thermal neutron testing of the devices was done in the thermal neutron field of the nuclear reactor at Institute for Energy Technology (IFE) at Kjeller Norway. All the devices were found to be sensitive to the field. Detector electronics, adapted to the different devices, has been built which can withstand the same radiation as the memory device without malfunctioning. There has been a focus on using Commercial Off The Shelf (COTS) components for reducing the total cost of the detector to about 100-200$US. The use of COTS SRAM memory devices also simplifies the reproducibility and availability of spares. The detector currently uses a two way communication between the detector and iv Abstract the readout computer over two pair of cables reducing the amount of cabling needed for experiments. The detectors can be connected to the communication link in a bus fashion, currently enabling a total of 14 detectors to be tested simultaneously from 100m away, over the same cable. Single Event Latch-up (SEL) and problems with irregular count rate of SRAMs created in the 90nm fabrication node has created problems during testing. Some solutions and techniques to mitigate these in hardware and firmware are presented in this work.Master i FysikkMAMN-PHYSPHYS39

Design and Development of a Multi-Purpose Input Output Controller Board for the SPES Control System

This PhD work has been carried out at the Legnaro National Laboratories (LNL), one of the four national labs of the National Institute for Nuclear Physics (INFN). The mission of LNL is to perform research in the field of nuclear physics and nuclear astrophysics together with emerging technologies. Technological research and innovation are the key to promote excellence in science, to excite competitive industries and to establish a better society. The research activities concerning electronics and computer science are an essential base to develop the control system of the Selective Production of Exotic Species (SPES) project. Nowadays, SPES is the most important project commissioned at LNL and represents the future of the Lab. It is a second generation Isotope Separation On-Line (ISOL) radioactive ion beam facility intended for fundamental nuclear physics research as well as experimental applications in different fields of science, such as nuclear medicine; radio-pharmaceutical production for therapy and diagnostic. The design of the SPES control system demands innovative technologies to embed the control of several appliances with different requirements and performing different tasks spanning from data sharing and visualization, data acquisition and storage, networking, security and surveillance operations, beam transport and diagnostic. The real time applications and fast peripherals control commonly found in the distributed control network of particle accelerators are accompanied by the challenge of developing custom embedded systems. In this context, the proposed PhD work describes the design and development of a multi-purpose Input Output Controller (IOC) board capable of embedding the control of typical accelerator instrumentation involved in the automatic beam transport system foreseen for the SPES project. The idea behind this work is to extend the control reach to the single device level without losing in modularity and standardization. The outcome of the research work is a general purpose embedded computer that will be the base for standardizing the hardware layer of the frontend computers in the SPES distributed control system. The IOC board is a Computer-on-Module (COM) carrier board designed to host any COM Express type 6 module and is equipped with a Field Programmable Gate Array (FPGA) and user application specific I/O connection solutions not found in a desktop pc. All the generic pc functionalities are readily available in off-the-shelf modules and the result is a custom motherboard that bridges the gap between custom developments and commercial personal computers. The end user can deal with a general-purpose pc with a high level of hardware abstraction besides being able to exploit the on-board FPGA potentialities in terms of fast peripherals control and real time digital data processing. This document opens with an introductory chapter about the SPES project and its control system architecture and technology before to describe the IOC board design, prototyping, and characterization. The thesis ends describing the installation in the field of the IOC board which is the core of the new diagnostics data readout and signal processing system. The results of the tests performed under real beam conditions prove that the new hardware extends the current sensitivity to the pA range, addressing the SPES requirements, and prove that the IOC board is a reliable solution to standardize the control of several appliances in the SPES accelerators complex where it will be embedded into physical equipment, or in their proximity, and will control and monitor their operation replacing the legacy VME technology. The installation in the field of the IOC board represents a great personal reward and crowns these years of busy time during which I turned what was just an idea in 2014, into a working embedded computer today

Dynamic Partial Reconfiguration for Dependable Systems

Moore’s law has served as goal and motivation for consumer electronics manufacturers in the last decades. The results in terms of processing power increase in the consumer electronics devices have been mainly achieved due to cost reduction and technology shrinking. However, reducing physical geometries mainly affects the electronic devices’ dependability, making them more sensitive to soft-errors like Single Event Transient (SET) of Single Event Upset (SEU) and hard (permanent) faults, e.g. due to aging effects. Accordingly, safety critical systems often rely on the adoption of old technology nodes, even if they introduce longer design time w.r.t. consumer electronics. In fact, functional safety requirements are increasingly pushing industry in developing innovative methodologies to design high-dependable systems with the required diagnostic coverage. On the other hand commercial off-the-shelf (COTS) devices adoption began to be considered for safety-related systems due to real-time requirements, the need for the implementation of computationally hungry algorithms and lower design costs. In this field FPGA market share is constantly increased, thanks to their flexibility and low non-recurrent engineering costs, making them suitable for a set of safety critical applications with low production volumes. The works presented in this thesis tries to face new dependability issues in modern reconfigurable systems, exploiting their special features to take proper counteractions with low impacton performances, namely Dynamic Partial Reconfiguration

The Fifth NASA Symposium on VLSI Design

The fifth annual NASA Symposium on VLSI Design had 13 sessions including Radiation Effects, Architectures, Mixed Signal, Design Techniques, Fault Testing, Synthesis, Signal Processing, and other Featured Presentations. The symposium provides insights into developments in VLSI and digital systems which can be used to increase data systems performance. The presentations share insights into next generation advances that will serve as a basis for future VLSI design

Multi-gigabit CMOS analog-to-digital converter and mixed-signal demodulator for low-power millimeter-wave communication systems

The objective of the research is to develop high-speed ADCs and mixed-signal demodulator for multi-gigabit communication systems using millimeter-wave frequency bands in standard CMOS technology. With rapid advancements in semiconductor technologies, mobile communication devices have become more versatile, portable, and inexpensive over the last few decades. However, plagued by the short lifetime of batteries, low power consumption has become an extremely important specification in developing mobile communication devices. The ever-expanding demand of consumers to access and share information ubiquitously at faster speeds requires higher throughputs, increased signal-processing functionalities at lower power and lower costs. In today’s technology, high-speed signal processing and data converters are incorporated in almost all modern multi-gigabit communication systems. They are key enabling technologies for scalable digital design and implementation of baseband signal processors. Ultimately, the merits of a high performance mixed-signal receiver, such as data rate, sensitivity, signal dynamic range, bit-error rate, and power consumption, are directly related to the quality of the embedded ADCs. Therefore, this dissertation focuses on the analysis and design of high-speed ADCs and a novel broadband mixed-signal demodulator with a fully-integrated DSP composed of low-cost CMOS circuitry. The proposed system features a novel dual-mode solution to demodulate multi-gigabit BPSK and ASK signals. This approach reduces the resolution requirement of high-speed ADCs, while dramatically reducing its power consumption for multi-gigabit wireless communication systems.PhDGee-Kung Chang - Committee Chair; Chang-Ho Lee - Committee Member; Geoffrey Ye Li - Committee Member; Paul A. Kohl - Committee Member; Shyh-Chiang Shen - Committee Membe

Firmware Development and Integration for ALICE TPC and PHOS Front-end Electronics: A Trigger Based Readout and Control System operating in a Radiation Environment

The readout electronics in PHOS and TPC - two of the major detectors of the ALICE experiment at the LHC - consist of a set of Front End Cards (FECs) that digitize, process and buffer the data from the detector sensors. The FECs are connected to a Readout Control Unit (RCU) via two sets of custom made PCB backplanes. For PHOS, 28 FECs are connected to one RCU, while for TPC the number is varying from 18 to 25 FECs depending on location. The RCU is in charge of the data readout, including reception and distribution of triggers and in moving the data from the FECs to the Data Acquisition System. In addition it does low level control tasks. The RCU consists of an RCU Motherboard that hosts a Detector Control System (DCS) board and a Source Interface Unit. The DCS board is an embedded computer running Linux that controls the readout electronics. All the mentioned devices are implemented in commercial grade SRAM based Field Programmable Gate Arrays (FPGAs). Even if these devices are not very radiation tolerant, they are chosen because of their cost and flexibility, and most importantly the possibility to easily do future upgrades of the electronics. Since physical shielding of the electronics is not possible in ALICE due to the architecture of the detector, the radiation related errors need to be handled with other techniques such as firmware mitigation techniques. The main objective of this thesis has been to make firmware modules for the FPGAs reciding in different parts of the readout electronics. Because of the flexibility of the designs, some of them have, with minor adaptations, been applied in different devices surrounding the readout electronics. Additionally, effort has been put into testing and integration of the system. In detail, the work presented in this thesis can be summarized as follows: - Firmware design for radiation environments. All firmware modules that are designed are to be used in a radiation environment, and then special precautions need to be taken. Additionally, a state-of-the-art solution has been designed for protecting the main FPGA on the RCU Motherboard against radiation induced functional failures. - Implementation of Trigger Handling for the TPC/PHOS Readout Electronics. The triggers are received from the global trigger system via an optical link and are handled by an Application Spesific Integrated Circuit (ASIC) on the DCS board. The problem is that the DCS board might have occasional down time 6 due to radiation related errors, so a special interface module is designed for the main FPGA on the RCU Motherboard. This module decodes and verifies the information received from the trigger system. As it is a generic design it has also been implemented as part of the BusyBox. The BusyBox is an important device in the trigger path of the TPC and PHOS sub-detectors. - Testing and Verification of all firmware modules. All firmware modules have been extensively verified with computer simulation before being tested in real hardware. - Maintenance of the DCS board for TPC/PHOS and of the different Fee firmware modules in general. - System Integration and System Level Tests. A big contribution has been done integrating and testing all the modules and sub-systems. This concern both locally on the RCU and the BusyBox, as well as making all the devices play together on a larger scale. - Testing and Verification of all firmware modules. All firmware modules have been extensively verified with computer simulation before being tested in real hardware. - Maintenance of the DCS board for TPC/PHOS and of the different Fee firmware modules in general. - System Integration and System Level Tests. A big contribution has been done integrating and testing all the modules and sub-systems. This concern both locally on the RCU and the BusyBox, as well as making all the devices play together on a larger scale. As the presented electronics are located in a radiation environment and are physically unavailable after commissioning, effort has been put into making designs that are reliable, scalable and possible to upgrade. This has been ensured by following a systematic design approach where testability, version management and documentation are key elements. Some parts of the work described in this thesis have been published and presented in international peer reviewed publications and conferences