44 research outputs found
Energy-efficient circuits and systems for computational imaging and vision on mobile devices
Thesis: Ph. D., Massachusetts Institute of Technology, Department of Electrical Engineering and Computer Science, 2018.Cataloged from PDF version of thesis.Includes bibliographical references (pages 125-127).Eighty five percent of images today are taken by cell phones. These images are not merely projections of light from the scene onto the camera sensor but result from a deep calculation. This calculation involves a number of computational imaging algorithms such as high dynamic range (HDR) imaging, panorama stitching, image deblurring and low-light imaging that compensate for camera limitations, and a number of deep learning based vision algorithms such as face recognition, object recognition and scene understanding that make inference on these images for a variety of emerging applications. However, because of their high computational complexity, mobile CPU or GPU based implementations of these algorithms do not achieve real-time performance. Moreover, offloading these algorithms to the cloud is not a viable solution because wirelessly transmitting large amounts of image data results in long latency and high energy consumption, making them unsuitable for mobile devices. This work solves these problems by designing energy-efficient hardware accelerators targeted at these applications. It presents the architecture of two complete computational imaging systems for energy-constrained mobile environments: (1) an energy-scalable accelerator for blind image deblurring, with an on-chip implementation and (2) a low-power processor for real-time motion magnification in videos, with an FPGA implementation. It also presents a 3D imaging platform and image processing workflow for 3D surface area assessment of dermatologic lesions. It demonstrates that such accelerator-based systems can enable energy-efficient integration of computational imaging and vision algorithms into mobile and wearable devices.by Priyanka Raina.Ph. D
Architectures for computational photography
Thesis (S.M.)--Massachusetts Institute of Technology, Dept. of Electrical Engineering and Computer Science, 2013.Cataloged from PDF version of thesis.Includes bibliographical references (p. 93-94).Computational photography refers to a wide range of image capture and processing techniques that extend the capabilities of digital photography and allow users to take photographs that could not have been taken by a traditional camera. Since its inception less than a decade ago, the field today encompasses a wide range of techniques including high dynamic range (HDR) imaging, low light enhancement, panorama stitching, image deblurring and light field photography. These techniques have so far been software based, which leads to high energy consumption and typically no support for real-time processing. This work focuses on hardware architectures for two algorithms - (a) bilateral filtering which is commonly used in computational photography applications such as HDR imaging, low light enhancement and glare reduction and (b) image deblurring. In the first part of this work, digital circuits for three components of a multi-application bilateral filtering processor are implemented - the grid interpolation block, the HDR image creation and contrast adjustment blocks, and the shadow correction block. An on-chip implementation of the complete processor, designed with other team members, performs HDR imaging, low light enhancement and glare reduction. The 40 nm CMOS test chip operates from 98 MHz at 0.9 V to 25 MHz at 0.9 V and processes 13 megapixels/s while consuming 17.8 mW at 98 MHz and 0.9 V, achieving significant energy reduction compared to previous CPU/GPU implementations. In the second part of this work, a complete system architecture for blind image deblurring is proposed. Digital circuits for the component modules are implemented using Bluespec SystemVerilog and verified to be bit accurate with a reference software implementation. Techniques to reduce power and area cost are investigated and synthesis results in 40nm CMOS technology are presentedby Priyanka Raina.S.M
LightNorm: Area and Energy-Efficient Batch Normalization Hardware for On-Device DNN Training
When training early-stage deep neural networks (DNNs), generating
intermediate features via convolution or linear layers occupied most of the
execution time. Accordingly, extensive research has been done to reduce the
computational burden of the convolution or linear layers. In recent
mobile-friendly DNNs, however, the relative number of operations involved in
processing these layers has significantly reduced. As a result, the proportion
of the execution time of other layers, such as batch normalization layers, has
increased. Thus, in this work, we conduct a detailed analysis of the batch
normalization layer to efficiently reduce the runtime overhead in the batch
normalization process. Backed up by the thorough analysis, we present an
extremely efficient batch normalization, named LightNorm, and its associated
hardware module. In more detail, we fuse three approximation techniques that
are i) low bit-precision, ii) range batch normalization, and iii) block
floating point. All these approximate techniques are carefully utilized not
only to maintain the statistics of intermediate feature maps, but also to
minimize the off-chip memory accesses. By using the proposed LightNorm
hardware, we can achieve significant area and energy savings during the DNN
training without hurting the training accuracy. This makes the proposed
hardware a great candidate for the on-device training.Comment: The paper is going to appearin the 40th IEEE International Conference
on Computer Design (ICCD), 202
eCNN: A Block-Based and Highly-Parallel CNN Accelerator for Edge Inference
Convolutional neural networks (CNNs) have recently demonstrated superior
quality for computational imaging applications. Therefore, they have great
potential to revolutionize the image pipelines on cameras and displays.
However, it is difficult for conventional CNN accelerators to support
ultra-high-resolution videos at the edge due to their considerable DRAM
bandwidth and power consumption. Therefore, finding a further memory- and
computation-efficient microarchitecture is crucial to speed up this coming
revolution.
In this paper, we approach this goal by considering the inference flow,
network model, instruction set, and processor design jointly to optimize
hardware performance and image quality. We apply a block-based inference flow
which can eliminate all the DRAM bandwidth for feature maps and accordingly
propose a hardware-oriented network model, ERNet, to optimize image quality
based on hardware constraints. Then we devise a coarse-grained instruction set
architecture, FBISA, to support power-hungry convolution by massive
parallelism. Finally,we implement an embedded processor---eCNN---which
accommodates to ERNet and FBISA with a flexible processing architecture. Layout
results show that it can support high-quality ERNets for super-resolution and
denoising at up to 4K Ultra-HD 30 fps while using only DDR-400 and consuming
6.94W on average. By comparison, the state-of-the-art Diffy uses dual-channel
DDR3-2133 and consumes 54.3W to support lower-quality VDSR at Full HD 30 fps.
Lastly, we will also present application examples of high-performance style
transfer and object recognition to demonstrate the flexibility of eCNN.Comment: 14 pages; appearing in IEEE/ACM International Symposium on
Microarchitecture (MICRO), 201
Low-Power Computer Vision: Improve the Efficiency of Artificial Intelligence
Energy efficiency is critical for running computer vision on battery-powered systems, such as mobile phones or UAVs (unmanned aerial vehicles, or drones). This book collects the methods that have won the annual IEEE Low-Power Computer Vision Challenges since 2015. The winners share their solutions and provide insight on how to improve the efficiency of machine learning systems
Comparative Study With New Accuracy Metrics for Target Volume Contouring in PET Image Guided Radiation Therapy
[EN] The impact of positron emission tomography (PET)
on radiation therapy is held back by poor methods of defining functional
volumes of interest. Many new software tools are being proposed
for contouring target volumes but the different approaches
are not adequately compared and their accuracy is poorly evaluated
due to the ill-definition of ground truth. This paper compares
the largest cohort to date of established, emerging and proposed
PET contouring methods, in terms of accuracy and variability.
We emphasize spatial accuracy and present a new metric
that addresses the lack of unique ground truth. Thirty methods
are used at 13 different institutions to contour functional volumes
of interest in clinical PET/CT and a custom-built PET phantom representing typical problems in image guided radiotherapy. Contouring
methods are grouped according to algorithmic type, level
of interactivity and how they exploit structural information in hybrid
images. Experiments reveal benefits of high levels of user interaction,
as well as simultaneous visualization of CT images and
PET gradients to guide interactive procedures. Method-wise evaluation
identifies the danger of over-automation and the value of
prior knowledge built into an algorithm.For retrospective patient data and manual ground truth delineation, the authors wish to thank S. Suilamo, K. Lehtio, M. Mokka, and H. Minn at the Department of Oncology and Radiotherapy, Turku University Hospital, Finland. This study was funded by the Finnish Cancer Organisations.Shepherd, T.; TerÀs, M.; Beichel, RR.; Boellaard, R.; Bruynooghe, M.; Dicken, V.; Gooding, MJ.... (2012). Comparative Study With New Accuracy Metrics for Target Volume Contouring in PET Image Guided Radiation Therapy. IEEE Transactions on Medical Imaging. 31(12):2006-2024. doi:10.1109/TMI.2012.2202322S20062024311
Event-based Vision: A Survey
Event cameras are bio-inspired sensors that differ from conventional frame
cameras: Instead of capturing images at a fixed rate, they asynchronously
measure per-pixel brightness changes, and output a stream of events that encode
the time, location and sign of the brightness changes. Event cameras offer
attractive properties compared to traditional cameras: high temporal resolution
(in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low
power consumption, and high pixel bandwidth (on the order of kHz) resulting in
reduced motion blur. Hence, event cameras have a large potential for robotics
and computer vision in challenging scenarios for traditional cameras, such as
low-latency, high speed, and high dynamic range. However, novel methods are
required to process the unconventional output of these sensors in order to
unlock their potential. This paper provides a comprehensive overview of the
emerging field of event-based vision, with a focus on the applications and the
algorithms developed to unlock the outstanding properties of event cameras. We
present event cameras from their working principle, the actual sensors that are
available and the tasks that they have been used for, from low-level vision
(feature detection and tracking, optic flow, etc.) to high-level vision
(reconstruction, segmentation, recognition). We also discuss the techniques
developed to process events, including learning-based techniques, as well as
specialized processors for these novel sensors, such as spiking neural
networks. Additionally, we highlight the challenges that remain to be tackled
and the opportunities that lie ahead in the search for a more efficient,
bio-inspired way for machines to perceive and interact with the world
The Design of Hotel Performance Management System in Padang
As a tourist place, Indonesia is supported by its beautiful natural scenaries and unique cultures. Actually most of Indonesia incomes came from tourism sectors. Padang as the administrative center of West Sumatra is one of tourism places in Indonesia. Unfortunately, all the facilities and touris actractions here need improvement, for example the hotels. Hotels in Padang need attention on the performance Hotels in Padang need attention on the performance This hotel depends on profit targets and classification of IHRA (Indonesian Hotel & Restaurant Association). For the increasement of this hotel, SWOT (Strength, Weakness, Opportunity, Threats) analysis and balanced scorecard method were applied. It began with the strategic information gathering based on interviewing the company, then continue processing it into a questionnaire which based on SWOT research. At this point, it is known that Premier Basko Hotel is in quadrant II (strength-threat) SWOT analysis diagram. So, this hotel needs to implement a diversification strategy. It also has 14 types of alternative strategies with strategic goals, 14 factors on Critical Success Factors (CSF), 38 indicators on Key Performance Indicators (KPI), and 38 pieces forms of performance management system. All of these are as the form of guidelines for the performance management system Premier Basko Hotel