An efficient FPGA implementation of versatile video coding intra prediction

Versatile Video Coding (VVC) is a new international video compression standard offering much better compression efficiency than previous video compression standards at the expense of much higher computational complexity. In this paper, an efficient FPGA implementation of VVC intra prediction for angular prediction modes of 4x4, 8x8, 16x16 and 32x32 prediction unit sizes is proposed. In the proposed FPGA implementation, four constant multiplications used in one intra angular prediction equation are implemented using two DSP blocks and two adders in FPGA. The proposed FPGA implementation of VVC intra prediction, in the worst case, can process 34 full HD (1920x1080) frames per second

FPGA implementation of HEVC intra prediction using high-level synthesis

Intra prediction algorithm in the recently developed High Efficiency Video Coding (HEVC) standard has very high computational complexity. High-level synthesis (HLS) tools are started to be successfully used for FPGA implementations of digital signal processing algorithms. Therefore, in this paper, the first FPGA implementation of HEVC intra prediction algorithm using a HLS tool in the literature is proposed. The proposed HEVC intra prediction hardware, in the worst case, can process 35 full HD (1920×1080) video frames per second. Using HLS tool significantly reduced the FPGA development time. Therefore, HLS tools can be used for FPGA implementation of HEVC video encoder

FPGA implementations of HEVC sub-pixel interpolation using high-level synthesis

Sub-pixel interpolation is one of the most computationally intensive parts of High Efficiency Video Coding (HEVC) video encoder and decoder. High-level synthesis (HLS) tools are started to be successfully used for FPGA implementations of digital signal processing algorithms. Therefore, in this paper, the first FPGA implementation of HEVC sub-pixel (half-pixel and quarter-pixel) interpolation algorithm using a HLS tool in the literature is proposed. The proposed HEVC sub-pixel interpolation hardware is implemented on Xilinx FPGAs using Xilinx Vivado HLS tool. It, in the worst case, can process 45 quad full HD (3840×2160) video frames per second. Using HLS tool significantly reduced the FPGA development time. Therefore, HLS tools can be used for FPGA implementation of HEVC video encoder

Pengkodean Video 3D Pada FPGA Berbasiskan Xilinx Zynq-7000

Kebutuhan konsumen terhadap teknologi multimedia yang baru dan lebih handal, menggiring pihak industri untuk meningkatkan pelayanan di bidang pemasaran entertainment, sehingga pada muaranya mendorong popularisasi konten video 3D, perangkat pendukung yang berkemampuan 3D, dan aplikasi-aplikasi 3D. Sebagai fenomena yang terjadi saat ini, smartphone, tablet, dan perangkat mobile lainnya sudah melampaui nilai penjualan PC. Bersamaan dengan semakin populernya video 3D dan diaplikasikan ke perangkat mobile tersebut, mengakibatkan kebutuhan akan penyimpanan, transmisi data, dan tampilan membutuhkan pengkodean yang efisien. High Efficiency Video Coding (HEVC) adalah teknik pengkodean video yang telah didesain menjadi standar untuk banyak aplikasi video dan memiliki kehandalan yang cukup signifikan dari generasi pendahulunya seperti teknik pengkodean H.264. Meskipun HEVC memiliki pengkodean yang sangat efiesien, namun disamping itu memerlukan beban prosesor yang berat dan menjalankan beban yang paralel pada saat pengkodean data yang berisi video. Untuk meningkatkan kehandalan dalam proses encoder, salah satunya dapat dilakukan dengan mengimplementasikan kode HEVC ke Zynq 7000 AP SoC. Diaplikasikan dalam tiga desain yaitu pertama dengan mengimplementasikan kedalam Zynq PS sebagai operasi standalone. Kedua yaitu dengan mengimplementasikan HEVC encoder dalam hardware/software co-design. Dan ketiga, implementasi code HEVC ke Zynq PL, tanpa PS. Dalam implementasi ini digunakan perangkat Xilinx Vivado HLS untuk mengembangkan kode yang dibutuhkan. Nilai hasil yang akan didapatkan adalah waktu yang dibutuhkan untuk pengkodean, PSNR, dan ukuran file hasil pengkodean kemudian akan dibandingkan antara kinerja PC berbasis Linux dengan FPGA Xilinx Zynq-7000. ======================================================================================================================== Consumer demand for new multimedia technologies and more reliable, drove the industry to improve services in the field of entertainment marketing, so that the estuary encourage the popularization of 3D video content, supporting devices 3D capabilities, and 3D applications. As a phenomenon that occurs at this time, smartphones, tablets, and other mobile devices has surpassed the value of PC sales. Along with the growing popularity of 3D video and be applied to the mobile device, resulting in the need for storage, data transmission, and display requires an efficient coding. High Efficiency Video Coding (HEVC) is a video coding technique that has been designed to become the standard for many video applications and has the reliability significantly from the preceding generation such as H.264 coding techniques. Although HEVC has very efiesien coding, but besides that it requires a heavy processor load and run parallel load at the time of encoding data containing the video. To improve reliability in the process of encoder, one of which can be done by implementing a code HEVC to Zynq 7000 AP SoC. Applied in three designs into Zynq first to implement PS as a standalone operation. The second is to implement HEVC encoder in hardware / software co-design. And third, the implementation code HEVC to Zynq PL, without PS. In this implementation Xilinx device is used Vivado HLS to develop the code needed. The value of the results to be obtained is the time required for video encoding, PSNR, and encoded file size that will be compared between the performance of Linux-based PCs with Xilinx Zynq-7000 FPGA

FGPA implementations of motion estimation algorithms using Vivado high level synthesis

Joint collaborative team on video coding (JCT-VC) recently developed a new international video compression standard called High Efficiency Video Coding (HEVC). HEVC has 50% better compression efficiency than previous H.264 video compression standard. HEVC achieves this video compression efficiency by significantly increasing the computational complexity. Motion estimation is the most computationally complex part of video encoders. Integer motion estimation and fractional motion estimation account for 70% of the computational complexity of an HEVC video encoder. High-level synthesis (HLS) tools are started to be successfully used for FPGA implementations of digital signal processing algorithms. They significantly decrease design and verification time. Therefore, in this thesis, we proposed the first FPGA implementation of HEVC full search motion estimation using Vivado HLS. Then, we proposed the first FPGA implementations of two fast search (diamond search and TZ search) algorithms using Vivado HLS. Finally, we proposed the first FPGA implementations of HEVC fractional interpolation and motion estimation using Vivado HLS. We used several HLS optimization directives to increase performance and decrease area of these FPGA implementations

IMPLEMENTASI HEVC CODEC PADA PLATFORM BERBASIS FPGA

High Efficiency Video Coding (HEVC) telah di desain sebagai standar baru untuk beberapa aplikasi video dan memiliki peningkatan performa dibanding dengan standar sebelumnya. Meskipun HEVC mencapai efisiensi coding yang tinggi, namun HEVC memiliki kekurangan pada beban pemrosesan tinggi dan loading yang berat ketika melakukan proses encoding video. Untuk meningkatkan performa encoder, kami bertujuan untuk mengimplementasikan HEVC codec pada Zynq 7000 AP SoC. Kami mencoba mengimplementasikan HEVC menggunakan tiga desain sistem. Pertama, HEVC codec di implementasikan pada Zynq PS. Kedua, encoder HEVC di implementasikan dengan hardware/software co-design. Ketiga, mengimplementasikan sebagian dari encoder HEVC pada Zynq PL. Pada implementasi kami menggunakan Xilinx Vivado HLS untuk mengembangkan codec. Hasil menunjukkan bahwa HEVC codec dapat di implementasikan pada Zynq PS. Codec dapat mengurangi ukuran video dibanding ukuran asli video pada format H.264. Kualitas video hampir sama dengan format H.264. Sayangnya, kami tidak dapat menyelesaikan desain dengan hardware/software co-design karena kompleksitas coding untuk validasi kode C pada Vivado HLS. Hasil lain, sebagian dari encoder HEVC dapat di implementasikan pada Zynq PL, yaitu HEVC 2D IDCT. Dari implementasi kami dapat mengoptimalkan fungsi loop pada HEVC 2D dan 1D IDCT menggunakan pipelining. Perbandingan hasil antara pipelining inner-loop dan outer-loop menunjukkan bahwa pipelining di outer-loop dapat meningkatkan performa dilihat dari nilai latency

High-Level Synthesis Implementation of HEVC Intra Encoder

High Efficiency Video Coding (HEVC) is the latest video coding standard that aims to alleviate the increasing transmission and storage needs of modern video applications. Compared with its predecessor, HEVC is able to halve the bit rate required for high quality video, but at the cost of increased complexity. High complexity makes HEVC video encoding slow and resource intensive but also ideal for hardware acceleration. With increasingly more complex designs, the effort required for traditional hardware development at register-transfer level (RTL) grows substantially. High-Level Synthesis (HLS) aims to solve this by raising the abstraction level through automatic tools that generate RTL-level code from general programming languages like C or C++. In this Thesis, we made use of Catapult-C HLS tool to create an intra coding accelerator for an HEVC encoder on a Field Programmable Gate Array (FPGA). We used the C source code of Kvazaar open-source HEVC encoder as a reference model for accelerator implementation. Over 90 % of the implementation including all major intra coding tools were implemented with HLS, with the rest being ready made IP blocks and hand-written RTL components. The accelerator was synthesized into an Arria 10 FPGA chip that was able to accommodate three accelerators and associated interface components. With two FPGAs connected to a high-end PC, our encoder was able to encode 2160p Ultra-High definition (UHD) video at 123 fps. Total FPGA resource usage was around 80 % with 346k Adaptive logic modules (ALMs) and 1227 Digital signal processors (DSPs)

Feasibility Study of High-Level Synthesis : Implementation of a Real-Time HEVC Intra Encoder on FPGA

High-Level Synthesis (HLS) on automatisoitu suunnitteluprosessi, joka pyrkii parantamaan tuottavuutta perinteisiin suunnittelumenetelmiin verrattuna, nostamalla suunnittelun abstraktiota rekisterisiirtotasolta (RTL) käyttäytymistasolle. Erilaisia kaupallisia HLS-työkaluja on ollut markkinoilla aina 1990-luvulta lähtien, mutta vasta äskettäin ne ovat alkaneet saada hyväksyntää teollisuudessa sekä akateemisessa maailmassa. Hidas käyttöönottoaste on johtunut pääasiassa huonommasta tulosten laadusta (QoR) kuin mitä on ollut mahdollista tavanomaisilla laitteistokuvauskielillä (HDL). Uusimmat HLS-työkalusukupolvet ovat kuitenkin kaventaneet QoR-aukkoa huomattavasti. Tämä väitöskirja tutkii HLS:n soveltuvuutta videokoodekkien kehittämiseen. Se esittelee useita HLS-toteutuksia High Efficiency Video Coding (HEVC) -koodaukselle, joka on keskeinen mahdollistava tekniikka lukuisille nykyaikaisille mediasovelluksille. HEVC kaksinkertaistaa koodaustehokkuuden edeltäjäänsä Advanced Video Coding (AVC) -standardiin verrattuna, saavuttaen silti saman subjektiivisen visuaalisen laadun. Tämä tyypillisesti saavutetaan huomattavalla laskennallisella lisäkustannuksella. Siksi reaaliaikainen HEVC vaatii automatisoituja suunnittelumenetelmiä, joita voidaan käyttää rautatoteutus- (HW ) ja varmennustyön minimoimiseen. Tässä väitöskirjassa ehdotetaan HLS:n käyttöä koko enkooderin suunnitteluprosessissa. Dataintensiivisistä koodaustyökaluista, kuten intra-ennustus ja diskreetit muunnokset, myös enemmän kontrollia vaativiin kokonaisuuksiin, kuten entropiakoodaukseen. Avoimen lähdekoodin Kvazaar HEVC -enkooderin C-lähdekoodia hyödynnetään tässä työssä referenssinä HLS-suunnittelulle sekä toteutuksen varmentamisessa. Suorituskykytulokset saadaan ja raportoidaan ohjelmoitavalla porttimatriisilla (FPGA). Tämän väitöskirjan tärkein tuotos on HEVC intra enkooderin prototyyppi. Prototyyppi koostuu Nokia AirFrame Cloud Server palvelimesta, varustettuna kahdella 2.4 GHz:n 14-ytiminen Intel Xeon prosessorilla, sekä kahdesta Intel Arria 10 GX FPGA kiihdytinkortista, jotka voidaan kytkeä serveriin käyttäen joko peripheral component interconnect express (PCIe) liitäntää tai 40 gigabitin Ethernettiä. Prototyyppijärjestelmä saavuttaa reaaliaikaisen 4K enkoodausnopeuden, jopa 120 kuvaa sekunnissa. Lisäksi järjestelmän suorituskykyä on helppo skaalata paremmaksi lisäämällä järjestelmään käytännössä minkä tahansa määrän verkkoon kytkettäviä FPGA-kortteja. Monimutkaisen HEVC:n tehokas mallinnus ja sen monipuolisten ominaisuuksien mukauttaminen reaaliaikaiselle HW HEVC enkooderille ei ole triviaali tehtävä, koska HW-toteutukset ovat perinteisesti erittäin aikaa vieviä. Tämä väitöskirja osoittaa, että HLS:n avulla pystytään nopeuttamaan kehitysaikaa, tarjoamaan ennen näkemätöntä suunnittelun skaalautuvuutta, ja silti osoittamaan kilpailukykyisiä QoR-arvoja ja absoluuttista suorituskykyä verrattuna olemassa oleviin toteutuksiin.High-Level Synthesis (HLS) is an automated design process that seeks to improve productivity over traditional design methods by increasing design abstraction from register transfer level (RTL) to behavioural level. Various commercial HLS tools have been available on the market since the 1990s, but only recently they have started to gain adoption across industry and academia. The slow adoption rate has mainly stemmed from lower quality of results (QoR) than obtained with conventional hardware description languages (HDLs). However, the latest HLS tool generations have substantially narrowed the QoR gap. This thesis studies the feasibility of HLS in video codec development. It introduces several HLS implementations for High Efficiency Video Coding (HEVC) , that is the key enabling technology for numerous modern media applications. HEVC doubles the coding efficiency over its predecessor Advanced Video Coding (AVC) standard for the same subjective visual quality, but typically at the cost of considerably higher computational complexity. Therefore, real-time HEVC calls for automated design methodologies that can be used to minimize the HW implementation and verification effort. This thesis proposes to use HLS throughout the whole encoder design process. From data-intensive coding tools, like intra prediction and discrete transforms, to more control-oriented tools, such as entropy coding. The C source code of the open-source Kvazaar HEVC encoder serves as a design entry point for the HLS flow, and it is also utilized in design verification. The performance results are gathered with and reported for field programmable gate array (FPGA) . The main contribution of this thesis is an HEVC intra encoder prototype that is built on a Nokia AirFrame Cloud Server equipped with 2.4 GHz dual 14-core Intel Xeon processors and two Intel Arria 10 GX FPGA Development Kits, that can be connected to the server via peripheral component interconnect express (PCIe) generation 3 or 40 Gigabit Ethernet. The proof-of-concept system achieves real-time. 4K coding speed up to 120 fps, which can be further scaled up by adding practically any number of network-connected FPGA cards. Overcoming the complexity of HEVC and customizing its rich features for a real-time HEVC encoder implementation on hardware is not a trivial task, as hardware development has traditionally turned out to be very time-consuming. This thesis shows that HLS is able to boost the development time, provide previously unseen design scalability, and still result in competitive performance and QoR over state-of-the-art hardware implementations