Video Compression and Optimization Technologies - Review

The use of video streaming is constantly increasing. High-resolution video requires resources on both the sender and the receiver side. There are many compression techniques that can be utilized to compress the video and simultaneously maintain quality. The main goal of this paper is to provide an overview of video streaming and QoE. This paper describes the basic concepts and discusses existing methodologies to measure QoE. Subjective, objective, and video compression technologies are discussed. This review paper gathers the codec implementation developed by MPEG, Google, and Apple. This paper outlines the challenges and future research directions that should be considered in the measurement and assessment of quality of experience for video services

First season MWA EoR power spectrum results at redshift 7

The Murchison Widefield Array (MWA) has collected hundreds of hours of Epoch of Reionization (EoR) data and now faces the challenge of overcoming foreground and systematic contamination to reduce the data to a cosmological measurement. We introduce several novel analysis techniques, such as cable reflection calibration, hyper-resolution gridding kernels, diffuse foreground model subtraction, and quality control methods. Each change to the analysis pipeline is tested against a two-dimensional power spectrum figure of merit to demonstrate improvement. We incorporate the new techniques into a deep integration of 32 hours of MWA data. This data set is used to place a systematic-limited upper limit on the cosmological power spectrum of ∆2 ≤ 2.7×104 mK2 at k =0.27 h Mpc-1 and z = 7.1, consistent with other published limits, and a modest improvement (factor of 1.4) over previous MWA results. From this deep analysis, we have identified a list of improvements to be made to our EoR data analysis strategies. These improvements will be implemented in the future and detailed in upcoming publications

Design of an image acquisition and processing system using configurable devices

This thesis consists of the evaluation of the possibility to implement a Neural Network in an FPGA instead on the more used GPU. Theoretically, an FPGA is a better choice in terms of processing power, latency, or flexibility but its configuration is harder. In this report, the implementation process for an FPGA is followed, including the creation of an embedded Operating System, video capture and display pipelines, testing of the chosen model and the final implementation of the model in the board. As a result of the evaluation, the conclusion is that nowadays the way to implement a neural network in an FPGA is not mature enough to compete with GPU alternative. The tools needed to achieve this implementation are very limited and the process is confusing. In the other hand, the GPU implementations has a huge catalogue of HW options and one can choose the better solution for its model

LOCATOR: Low-power ORB accelerator for autonomous cars

Simultaneous Localization And Mapping (SLAM) is crucial for autonomous navigation. ORB-SLAM is a state-of-the-art Visual SLAM system based on cameras used for self-driving cars. In this paper, we propose a high-performance, energy-efficient, and functionally accurate hardware accelerator for ORB-SLAM, focusing on its most time-consuming stage: Oriented FAST and Rotated BRIEF (ORB) feature extraction. The Rotated BRIEF (rBRIEF) descriptor generation is the main bottleneck in ORB computation, as it exhibits highly irregular access patterns to local on-chip memories causing a high-performance penalty due to bank conflicts. We introduce a technique to find an optimal static pattern to perform parallel accesses to banks based on a genetic algorithm. Furthermore, we propose the combination of an rBRIEF pixel duplication cache, selective ports replication, and pipelining to reduce latency without compromising cost. The accelerator achieves a reduction in energy consumption of 14597× and 9609×, with respect to high-end CPU and GPU platforms, respectively.This work has been supported by the CoCoUnit ERC Advanced Grant of the EU’s Horizon 2020 program (grant No 833057), the Spanish State Research Agency (MCIN/AEI) under grant PID2020- 113172RB-I00, the ICREA Academia program and the FPU grant FPU18/04413Peer ReviewedPostprint (published version

ENERGY-EFFICIENT FEATURE EXTRACTION ENGINE AND SECURE CHIP IDENTIFICATION FOR UBIQUITOUS SURVEILLANCE

Ph.DDOCTOR OF PHILOSOPH

First Season MWA EoR Power Spectrum Results at Redshift 7

The Murchison Widefield Array (MWA) has collected hundreds of hours of Epoch of Reionization (EoR) data and now faces the challenge of overcoming foreground and systematic contamination to reduce the data to a cosmological measurement. We introduce several novel analysis techniques such as cable reflection calibration, hyper-resolution gridding kernels, diffuse foreground model subtraction, and quality control methods. Each change to the analysis pipeline is tested against a two dimensional power spectrum figure of merit to demonstrate improvement. We incorporate the new techniques into a deep integration of 32 hours of MWA data. This data set is used to place a systematic-limited upper limit on the cosmological power spectrum of Δ2≤2.7×104 mK2 at k=0.27 h~Mpc−1 and z=7.1, consistent with other published limits, and a modest improvement (factor of 1.4) over previous MWA results. From this deep analysis we have identified a list of improvements to be made to our EoR data analysis strategies. These improvements will be implemented in the future and detailed in upcoming publications.Astronom

Computational Approaches to Simulation and Analysis of Large Conformational Transitions in Proteins

abstract: In a typical living cell, millions to billions of proteins—nanomachines that fluctuate and cycle among many conformational states—convert available free energy into mechanochemical work. A fundamental goal of biophysics is to ascertain how 3D protein structures encode specific functions, such as catalyzing chemical reactions or transporting nutrients into a cell. Protein dynamics span femtosecond timescales (i.e., covalent bond oscillations) to large conformational transition timescales in, and beyond, the millisecond regime (e.g., glucose transport across a phospholipid bilayer). Actual transition events are fast but rare, occurring orders of magnitude faster than typical metastable equilibrium waiting times. Equilibrium molecular dynamics (EqMD) can capture atomistic detail and solute-solvent interactions, but even microseconds of sampling attainable nowadays still falls orders of magnitude short of transition timescales, especially for large systems, rendering observations of such "rare events" difficult or effectively impossible. Advanced path-sampling methods exploit reduced physical models or biasing to produce plausible transitions while balancing accuracy and efficiency, but quantifying their accuracy relative to other numerical and experimental data has been challenging. Indeed, new horizons in elucidating protein function necessitate that present methodologies be revised to more seamlessly and quantitatively integrate a spectrum of methods, both numerical and experimental. In this dissertation, experimental and computational methods are put into perspective using the enzyme adenylate kinase (AdK) as an illustrative example. We introduce Path Similarity Analysis (PSA)—an integrative computational framework developed to quantify transition path similarity. PSA not only reliably distinguished AdK transitions by the originating method, but also traced pathway differences between two methods back to charge-charge interactions (neglected by the stereochemical model, but not the all-atom force field) in several conserved salt bridges. Cryo-electron microscopy maps of the transporter Bor1p are directly incorporated into EqMD simulations using MD flexible fitting to produce viable structural models and infer a plausible transport mechanism. Conforming to the theme of integration, a short compendium of an exploratory project—developing a hybrid atomistic-continuum method—is presented, including initial results and a novel fluctuating hydrodynamics model and corresponding numerical code.Dissertation/ThesisDoctoral Dissertation Physics 201

On-chip memory reduction in CNN hardware design for image super-resolution

학위논문 (박사)-- 서울대학교 대학원 : 공과대학 전기·컴퓨터공학부, 2019. 2. 이혁재.Single image super-resolution (SISR) 을 위한 convolutional neural network (CNN) 는 영상 분류용 CNN과 달리 고해상도의 영상을 입력 받아 고해상도의 중간 연산 결과인 feature map을 생성 한다. SISR용 CNN을 가속하기 위한 하드웨어는 주로 디스플레이 장치에 적용이 되며 외부 메모리 접근이 불가능한 스트리밍 구조를 갖는다. 이는 on-chip 메모리의 용량이 제한적인 하드웨어의 특성상 구현의 어려움을 야기한다. 기존의 연구들은 on-chip 메모리를 감소하기 위해 성능 저하 또는 압축 모듈을 추가한다. 본 논문은 성능 저하 없이 SISR용 CNN 하드웨어의 on-chip 메모리 감소 및 하드웨어를 설계하기 위한 방법을 제안한다. CNN 하드웨어는 VDSR (Very deep neural network for super-resolution) 구조를 기반으로 한다. 기존 CNN 하드웨어의 SRAM에 읽기 및 쓰기 접근이 동시에 발생하는 래스터 스캔 순서를 부분적 수직 순서로 변경 함으로 읽기 및 쓰기 접근 타이밍을 분리한다. 부분적 수직 순서는 기존의 CNN 하드웨어가 사용하는 듀얼 포트 SRAM 대신 싱글 포트 SRAM을 사용하도록 하며 이는 on-chip 메모리를 절반으로 감소한다. 두 번째 방법으로 VDSR의 필터의 형태를 변경하는 방법을 적용한다. On-chip 메모리의 크기는 컨볼루션 필터의 높이에 비례한다. 그러나 VDSR의 필터는 대칭 구조 중 가장 작은 필터 모양이므로 해당 문제를 해결하기 위해 컨텍스트 보존 1D 필터 구성 방법 및 컨텍스트를 기반한 세로 필터 감소 방법을 적용하여 SRAM의 크기를 절반으로 추가적으로 감소한다. CNN 하드웨어 구조가 확정 된 이후 CNN의 SISR 성능을 개선 하기 위한 CNN학습 방법을 자연 영상 (natural image)와 텍스트 영상 (text image)에 대해 각각 제안한다. SRGAN (Super-resolution generative adversarial networks) 는 판별자 네트워크 (discriminator network)로부터 발생하는 손실으로 SISR용 CNN이 실제 영상처럼 보이는 자연 영상을 출력하도록 한다. 그러나 SRGAN은 과선명화로 인한 시각적 결함을 발생하는 문제가 있다. 본 논문은 SRGAN의 시각적 결함을 제거하는 두 가지 방법을 제안한다. 첫 번째는 판별자 네트워크의 구조를 변경하여 판별자 네트워크 내에서 영상의 세부 정보 손실을 방지하는 해상도 유지 판별자 네트워크 구조를 제안 한다. 두 번째는 콘텐트 손실을 발생하는 VGG 네트워크의 구조상 영상의 세부적인 정보를 손실하는 문제를 해결하기 위한 해상도 유지 콘텐트 손실 방법을 제안한다. 텍스트 영상은 자연 영상이 아닌 합성 영상으로 영상 내 폰트와 배경의 색상 조합을 다양하게 변경될 수 있다. 기존의 CNN 학습 방법은 네트워크의 일반화를 위해 다양한 종류의 영상을 학습 시키는 방법을 사용한다. 그러나 모든 종류의 색상 조합을 CNN에 학습 시키는 것은 불가능하다. 본 논문은 영상 압축에 사용되는 De-colorization 방법을 차용하여 CNN이 학습할 영상을 검은 폰트와 흰색 배경으로 이루어진 영상으로 한정 함으로 학습되지 않은 영상의 폰트 및 배경 색상 조합에도 시각적 결함 없이 SISR 연산을 수행 하는 방법을 제안 한다.Unlike convolutional neural network (CNN) for image classification, CNN for single image super-resolution (SISR) receives high-resolution image and generates feature maps which are high-resolution intermediate results. The hardware for accelerating the CNN for SISR is mainly applied to the display device, and the CNN hardware has a streaming architecture in which external memory access is impossible. This causes implementation difficulties due to the limited hardware capacity of the on-chip memory. This paper proposes two methods for designing CNN hardware for SISR using limited hardware resources. CNN hardware is based on a very deep neural network for super-resolution (VDSR) architecture. By using the partially-vertical order for the convolution layers, simultaneous read and write accesses to SRAM are prevented. The proposed order makes CNN use single-port SRAM instead of dual-port SRAM, and it reduces on-chip memory area by half. The second method is to change the shape of the filter in VDSR. The size of the on-chip memory is proportional to the height of the convolution filter. However, since the filter of VDSR is the smallest of the symmetric shape, it is impossible to reduce the filter height of the VDSR. To solve this problem, a method of constructing a context-preserving 1D filter and a method of decreasing a vertical filter based on the context are proposed. These proposed methods reduce the size of the SRAM in half. Two CNN training methods for SISR of natural image and that of text image are proposed. These methods improve SISR performance after the CNN hardware architecture is confirmed. SRGAN (super-resolution generative adversarial networks) is trained by the help of discriminator network to generate realistic natural images. However, SRGAN has the problem of causing visual defects due to over-sharpening. This paper proposes two methods to eliminate the visual defects of SRGAN. First, the resolution-preserving discriminator network structure is proposed. This discriminator network prevents detailed information loss in the network by changing the structure of it. Second, the resolution-preserving content loss is proposed to solve the problem of loss of detailed information of image due to the structure of VGG19 network that causes content loss. The text image is not a natural image but a synthetic image. The color combination of the font and the background in the image can be variously changed. The existing CNN learning method uses a method of learning various kinds of images to generalize the network. However, it is impossible to learn all kinds of color combinations on CNN. This paper uses the de-colorization method used in image compression to limit the image to be learned by CNN to a black font and a white background image. As a result, CNN performs SISR operation without visual flaws in the font and background color combination image of the trained image.제 1 장 서 론 1 1.1 연구의 배경 1 1.2 연구의 내용 5 1.3 논문의 구성 8 제 2 장 이전 연구 9 2.1 SISR CNN 알고리즘 9 2.2 스트리밍 구조의 SISR 하드웨어 14 2.3 기존 CNN 하드웨어의 on-chip 메모리 감소 방법 15 2.4 De-colorization 17 제 3 장 컨볼루션 뉴럴 네트워크의 SRAM 면적 감소를 위한 연산 순서 변경 20 3.1 부분적 수직 순서 컨볼루션 연산 20 3.2 ifmap을 저장하기 위한 레지스터 24 3.3 CNN의 첫 번째 및 마지막 컨볼루션 레이어 SRAM 구성 26 3.4 fmap의 SRAM 다채널 공유를 위한 부분적 수직 순서 28 3.5 부분적 수직 순서의 적용 가능 CNN 구조 33 3.5 실험 결과 36 제 4 장 영상의 컨텍스트 보존을 위한 필터 재구성 및 CNN 하드웨어 설계 42 4.1 SRAM 감소를 위한 제안 알고리즘 43 4.2 SISR용 CNN 하드웨어 구조 49 4.3 실험 결과 55 제 5 장 SISR을 위한 해상도 보존 생산적 적대 신경망 구조 64 5.1 해상도 보존 판별 신경망 구조 64 5.2 해상도 보존 콘텐트 손실 68 5.3 실험 결과 70 제 6 장 De-colorization을 적용한 text SISR 84 6.1 Text de-colorization을 적용한 CNN 학습 84 6.2 실험 결과 86 제 7 장 결론 95 참고문헌 98 Abstract 105Docto