SARA: Self-Aware Resource Allocation for Heterogeneous MPSoCs

In modern heterogeneous MPSoCs, the management of shared memory resources is crucial in delivering end-to-end QoS. Previous frameworks have either focused on singular QoS targets or the allocation of partitionable resources among CPU applications at relatively slow timescales. However, heterogeneous MPSoCs typically require instant response from the memory system where most resources cannot be partitioned. Moreover, the health of different cores in a heterogeneous MPSoC is often measured by diverse performance objectives. In this work, we propose a Self-Aware Resource Allocation (SARA) framework for heterogeneous MPSoCs. Priority-based adaptation allows cores to use different target performance and self-monitor their own intrinsic health. In response, the system allocates non-partitionable resources based on priorities. The proposed framework meets a diverse range of QoS demands from heterogeneous cores.Comment: Accepted by the 55th annual Design Automation Conference 2018 (DAC'18

Real-time audio spectrum analyser research, design, development and implementation using the 32 bit ARMR Cortex-M4 microcontroller

This thesis describes the design and testing of a low-cost hand-held real-time audio analyser (RTAA). This includes the design of an embedded system, the development of the firmware executed by the embedded system, and the implementation of a real-time signal processing algorithms. One of the objectives of this project was to design an alternative low-cost audio analyser to the current commercially available solutions. The device was tested with the audio standard test signal (pink noise) and was compared to the expected at-spectrum response corresponding to a balanced audio system. The design makes use of an 32-bit Reduced Instruction Set Computer (RISC) processor core (ARM Cortex-M4), namely the STM32F4 family of microcontrollers. Due to the pin compatibility of the microcontroller (designed and manufactured by STMicroelectronics), the new development board can also be upgraded with the newly released Cortex-M7 microcontroller, namely the STM32F7 family of microcontrollers. Moreover, the low-cost hardware design features 256kB Random Access Memory (RAM); on-board Micro-Electro-Mechanical System (MEMS) microphone; on-chip 12-bit Analogue-to-Digital (A/D) and Digital-to-Analogue (D/A) Converters; 3.2" Thin-Film-Transistor Liquid-Crystal Display (TFT-LCD) with a resistive touch screen sensor and SD-Card Socket. Furthermore, two additional expansion modules were designed and can extend the functionality of the designed real-time audio analyser. Firstly, an audio/video module featuring a professional 24-bit 192kHz sampling rate audio CODEC; balanced audio microphone input; unbalanced line output; three MEMS microphone inputs; headphone output; and a Video Graphics Array (VGA) controller allowing the display of the analysed audio spectrum on either a projector or monitor. The second expansion module features two external memories: 1MB Static Random Access Memory (SRAM) and 16MB Synchronous Dynamic Random Access Memory (SDRAM). While the two additional expansion modules were not completely utilised by the firmware presented in this thesis, upgrades of the real-time audio analyser firmware in future revisions will provide a higher performing and more accurate analysis of the audio spectrum. The full research and design process for the real-time audio analyser is discussed and both Problems and pitfalls with the final implemented design are highlighted and possible resolutions were investigated. The development costs (excluding labour) are given in the form of a bill of materials (BOM) with the total costs averaging around R1000. Moreover, the additional VGA controller could further decrease the overall costs with the removal of the TFT-LCD screen from the audio analyser and provided the external display was not included in the BOM

Pyramic array: An FPGA based platform for many-channel audio acquisition

Array processing of audio data has many interesting applications: acoustic beamforming, source separation, indoor localization, room geometry estimation, etc. Recent advances in MEMS has produced tiny microphones, analog or even with digital converter integrated. This opens the door to create arrays with a massive number of microphones. We dub such an array many-channel by analogy to many-core processors.Microphone arrays techniques present compelling applications for robotic implementations. Those techniques can allow robots to listen to their environment and infer clues from it. Such features might enable capabilities such as natural interaction with humans, interpreting spoken commands or the localization of victims during search and rescue tasks. However, under noisy conditions robotic implementations of microphone arrays might degrade their precision when localizing sound sources. For practical applications, human hearing still leaves behind microphone arrays. Daniel Kisch is an example of how humans are able to efficiently perform echo-localization to recognize their environment, even in noisy and reverberant environments. For ubiquitous computing, another limitation of acoustic localization algorithms is within their capabilities of performing real-time Digital Signal Processing (DSP) operations. To tackle those problems, tradeoffs between size, weight, cost and power consumption compromise the design of acoustic sensors for practical applications. This work presents the design and operation of a large microphone array for DSP applications in realistic environments. To address those problems this project introduces the Pyramic sound capture system designed at LAP in EPFL. Pyramic is a custom hardware which possesses 48 microphones dis- tributed in the edges of a tetrahedron. The microphone arrays interact with a Terasic DE1-SoC board from Altera Cyclone V family devices, which combines a Hard Processor System (HPS) and a Field Programmable Gate Array (FPGA) in the same die. The HPS part integrates a dual- core ARM-based Cortex-A9 processor, which combined with the power of FPGA design suitable for processing multichannel microphone signals. This thesis explains the implementation of the Pyramic array. Moreover, FPGA-based hardware accelerators have been designed to imple- ment a Master SPI communication with the array and a parallel 48 channels FIR filters cascade of the audio data for delay-and-sum beamforming applications. Additionally, the configura- tion of the HPS part allows the Pyramic array to be controlled through a Linux based OS. The main purpose of the project is to develop a flexible platform in which real-time echo-location algorithms can be implemented. The effectiveness of the Pyramic array design is illustrated by testing the recorded data with offline direction of arrival algorithms developed at LCAV in EPFL

Interactive Real-Time Embedded Systems Education Infused with Applied Internet Telephony

The transition from traditional circuit-switched phone systems to modern packet-based Internet telephony networks demands tools to support Voice over Internet Protocol (VoIP) development. In this paper, we introduce the XinuPhone, an integrated hardware/software approach for educating users about VoIP technology on a real-time embedded platform. We propose modular course topics for design-oriented, hands-on laboratory exercises: filter design, timing, serial communications, interrupts and resource budgeting, network transmission, and system benchmarking. Our open-source software platform encourages development and testing of new CODECs alongside existing standards, unlike similar commercial solutions. Furthermore, the supporting hardware features inexpensive, readily available components designed specifically for educational and research users on a limited budget. The XinuPhone is especially good for experimenting with design trade-offs as well as interactions between real-time software and hardware components

Design of an image acquisition and processing system using configurable devices

This thesis consists of the evaluation of the possibility to implement a Neural Network in an FPGA instead on the more used GPU. Theoretically, an FPGA is a better choice in terms of processing power, latency, or flexibility but its configuration is harder. In this report, the implementation process for an FPGA is followed, including the creation of an embedded Operating System, video capture and display pipelines, testing of the chosen model and the final implementation of the model in the board. As a result of the evaluation, the conclusion is that nowadays the way to implement a neural network in an FPGA is not mature enough to compete with GPU alternative. The tools needed to achieve this implementation are very limited and the process is confusing. In the other hand, the GPU implementations has a huge catalogue of HW options and one can choose the better solution for its model

Development and Evaluation of a Real-Time Framework for a Portable Assistive Hearing Device

Testing and verification of digital hearing aid devices, and the embedded software and algorithms can prove to be a challenging task especially taking into account time-to-market considerations. This thesis describes a PC based, real-time, highly configurable framework for the evaluation of audio algorithms. Implementation of audio processing algorithms on such a platform can provide hearing aid designers and manufacturers the ability to test new and existing processing techniques and collect data about their performance in real-life situations, and without the need to develop a prototype device. The platform is based on the Eurotech Catalyst development kit and the Fedora Linux OS, and it utilizes the JACK audio engine to facilitate reliable real-time performance Additionally, we demonstrate the capabilities of this platform by implementing an audio processing chain targeted at improving speech intelligibility for people suffering from auditory neuropathy. Evaluation is performed for both noisy and noise-free environments. Subjective evaluation of the results, using normal hearing listeners and an auditory neuropathy simulator, demonstrates improvement in some conditions

The Design of a System Architecture for Mobile Multimedia Computers

This chapter discusses the system architecture of a portable computer, called Mobile Digital Companion, which provides support for handling multimedia applications energy efficiently. Because battery life is limited and battery weight is an important factor for the size and the weight of the Mobile Digital Companion, energy management plays a crucial role in the architecture. As the Companion must remain usable in a variety of environments, it has to be flexible and adaptable to various operating conditions. The Mobile Digital Companion has an unconventional architecture that saves energy by using system decomposition at different levels of the architecture and exploits locality of reference with dedicated, optimised modules. The approach is based on dedicated functionality and the extensive use of energy reduction techniques at all levels of system design. The system has an architecture with a general-purpose processor accompanied by a set of heterogeneous autonomous programmable modules, each providing an energy efficient implementation of dedicated tasks. A reconfigurable internal communication network switch exploits locality of reference and eliminates wasteful data copies

Utilizing DSP for IP telephony applications in mobile terminals

Tässä diplomityössä etsitään ja määritellään optimaalinen ohjelmistoarkkitehtuuri reaaliaikaisen puheenkoodauksen mahdollistamiseksi mobiilin laitteen Internet-puheluohjelmistossa. Arkkitehtuurille asetettiin vaatimus, jonka mukaan puhelu ja siihen liittyvä puheen reaaliaikaisuus ei saa rajoittaa tai liikaa kuormittaa laitteen muuta toiminnallisuutta. Työssä käytetty mobiili laite tarjoaa mahdollisuuden hyödyntää kahta prosessoria. Toinen prosessoreista on tarkoitettu yleisille käyttöjärjestelmille sekä ohjelmistoille ja toinen signaalinkäsittelyoperaatioille. Suunniteltu arkkitehtuuri yhdistää näiden kahden prosessorin toiminnallisuuden ja mahdollistaa reaaliaikaisen puheenkoodauksen (sekä toisto että äänitys) mobiliisissa laitteessa. Arkkitehtuuri toteutettiin ja sen suorituskykyä arvioitiin erilaisilla mittauksilla ja parametreilla. Havaittiin, että toteutus suoriutuu erinomaisesti sille asetetuista vaatimuksista. Todettiin myös, että käytettäessä ainoastaan laitteen yhtä prosessoria reaaliaikavaatimus ei täyty. Tämä johtuu puhekoodekin matemaattisesta kompleksisuudesta ja laitteen rajoitetuista ominaisuuksista. Työn aikana jätettiin kaksi patenttihakemusta.In this thesis, an optimal software architecture is studied and defined for enabling a real-time speech coding scheme in an Internet telephony application of a mobile terminal. According to a requirement set for the architecture, a phone call and the related real-time speech coding shall not limit or overload other functionality of the terminal. The mobile terminal utilized in this thesis provides a potential to take advantage of the efficiency of a dual core processor. One of the processors is designed for general purpose operating systems, and the other one for signal processing operations. The designed software architecture combines the functionality of these processors and enables real-time speech coding (both playback and capture) in the device. The architecture was implemented and its performance was evaluated with different measurements and parameters. It was observed that the implementation outperforms the requirements set. It was also confirmed that the performance of the general purpose processor is inadequate for real-time operations with the chosen speech coder/decoder. Two patent applications were filed by the author during the writing of this thesis