9,508 research outputs found
DeepWear: Adaptive Local Offloading for On-Wearable Deep Learning
Due to their on-body and ubiquitous nature, wearables can generate a wide
range of unique sensor data creating countless opportunities for deep learning
tasks. We propose DeepWear, a deep learning (DL) framework for wearable devices
to improve the performance and reduce the energy footprint. DeepWear
strategically offloads DL tasks from a wearable device to its paired handheld
device through local network. Compared to the remote-cloud-based offloading,
DeepWear requires no Internet connectivity, consumes less energy, and is robust
to privacy breach. DeepWear provides various novel techniques such as
context-aware offloading, strategic model partition, and pipelining support to
efficiently utilize the processing capacity from nearby paired handhelds.
Deployed as a user-space library, DeepWear offers developer-friendly APIs that
are as simple as those in traditional DL libraries such as TensorFlow. We have
implemented DeepWear on the Android OS and evaluated it on COTS smartphones and
smartwatches with real DL models. DeepWear brings up to 5.08X and 23.0X
execution speedup, as well as 53.5% and 85.5% energy saving compared to
wearable-only and handheld-only strategies, respectively
Modelling the Scene Dependent Imaging in Cameras with a Deep Neural Network
We present a novel deep learning framework that models the scene dependent
image processing inside cameras. Often called as the radiometric calibration,
the process of recovering RAW images from processed images (JPEG format in the
sRGB color space) is essential for many computer vision tasks that rely on
physically accurate radiance values. All previous works rely on the
deterministic imaging model where the color transformation stays the same
regardless of the scene and thus they can only be applied for images taken
under the manual mode. In this paper, we propose a data-driven approach to
learn the scene dependent and locally varying image processing inside cameras
under the automode. Our method incorporates both the global and the local scene
context into pixel-wise features via multi-scale pyramid of learnable histogram
layers. The results show that we can model the imaging pipeline of different
cameras that operate under the automode accurately in both directions (from RAW
to sRGB, from sRGB to RAW) and we show how we can apply our method to improve
the performance of image deblurring.Comment: To appear in ICCV 201
Hardware-Aware Machine Learning: Modeling and Optimization
Recent breakthroughs in Deep Learning (DL) applications have made DL models a
key component in almost every modern computing system. The increased popularity
of DL applications deployed on a wide-spectrum of platforms have resulted in a
plethora of design challenges related to the constraints introduced by the
hardware itself. What is the latency or energy cost for an inference made by a
Deep Neural Network (DNN)? Is it possible to predict this latency or energy
consumption before a model is trained? If yes, how can machine learners take
advantage of these models to design the hardware-optimal DNN for deployment?
From lengthening battery life of mobile devices to reducing the runtime
requirements of DL models executing in the cloud, the answers to these
questions have drawn significant attention.
One cannot optimize what isn't properly modeled. Therefore, it is important
to understand the hardware efficiency of DL models during serving for making an
inference, before even training the model. This key observation has motivated
the use of predictive models to capture the hardware performance or energy
efficiency of DL applications. Furthermore, DL practitioners are challenged
with the task of designing the DNN model, i.e., of tuning the hyper-parameters
of the DNN architecture, while optimizing for both accuracy of the DL model and
its hardware efficiency. Therefore, state-of-the-art methodologies have
proposed hardware-aware hyper-parameter optimization techniques. In this paper,
we provide a comprehensive assessment of state-of-the-art work and selected
results on the hardware-aware modeling and optimization for DL applications. We
also highlight several open questions that are poised to give rise to novel
hardware-aware designs in the next few years, as DL applications continue to
significantly impact associated hardware systems and platforms.Comment: ICCAD'18 Invited Pape
IoT for Green Building Management
Buildings consume 60% of global electricity. However, current building
management systems (BMSs) are highly expensive and difficult to justify for
small to medium-sized buildings. As such, the Internet of Things (IoT), which
can monitor and collect a large amount of data on different contexts of a
building and feed the data to the processor of the BMS, provides a new
opportunity to integrate intelligence into the BMS to monitor and manage the
energy consumption of the building in a cost-effective manner. Although an
extensive literature is available on IoT based BMS and applications of signal
processing techniques for some aspects of building energy management
separately, detailed study on their integration to address the overall BMS is
quite limited. As such, the proposed paper will address this gap by providing
an overview of an IoT based BMS leveraging signal processing and machine
learning techniques. It is demonstrated how to extract high-level building
occupancy information through simple and low-cost IoT sensors and studied the
impact of human activities on energy usage of a building, which can be
exploited to design energy conservation measures to reduce the building's
energy consumption.Comment: 20 pages, 7 figures, 1 table, accepted journal pape
Local Light Field Fusion: Practical View Synthesis with Prescriptive Sampling Guidelines
We present a practical and robust deep learning solution for capturing and
rendering novel views of complex real world scenes for virtual exploration.
Previous approaches either require intractably dense view sampling or provide
little to no guidance for how users should sample views of a scene to reliably
render high-quality novel views. Instead, we propose an algorithm for view
synthesis from an irregular grid of sampled views that first expands each
sampled view into a local light field via a multiplane image (MPI) scene
representation, then renders novel views by blending adjacent local light
fields. We extend traditional plenoptic sampling theory to derive a bound that
specifies precisely how densely users should sample views of a given scene when
using our algorithm. In practice, we apply this bound to capture and render
views of real world scenes that achieve the perceptual quality of Nyquist rate
view sampling while using up to 4000x fewer views. We demonstrate our
approach's practicality with an augmented reality smartphone app that guides
users to capture input images of a scene and viewers that enable realtime
virtual exploration on desktop and mobile platforms.Comment: SIGGRAPH 2019. Project page with video and code:
http://people.eecs.berkeley.edu/~bmild/llff
Leveraging Machine Learning and Big Data for Smart Buildings: A Comprehensive Survey
Future buildings will offer new convenience, comfort, and efficiency
possibilities to their residents. Changes will occur to the way people live as
technology involves into people's lives and information processing is fully
integrated into their daily living activities and objects. The future
expectation of smart buildings includes making the residents' experience as
easy and comfortable as possible. The massive streaming data generated and
captured by smart building appliances and devices contains valuable information
that needs to be mined to facilitate timely actions and better decision making.
Machine learning and big data analytics will undoubtedly play a critical role
to enable the delivery of such smart services. In this paper, we survey the
area of smart building with a special focus on the role of techniques from
machine learning and big data analytics. This survey also reviews the current
trends and challenges faced in the development of smart building services
Personalized Exposure Control Using Adaptive Metering and Reinforcement Learning
We propose a reinforcement learning approach for real-time exposure control
of a mobile camera that is personalizable. Our approach is based on Markov
Decision Process (MDP). In the camera viewfinder or live preview mode, given
the current frame, our system predicts the change in exposure so as to optimize
the trade-off among image quality, fast convergence, and minimal temporal
oscillation. We model the exposure prediction function as a fully convolutional
neural network that can be trained through Gaussian policy gradient in an
end-to-end fashion. As a result, our system can associate scene semantics with
exposure values; it can also be extended to personalize the exposure
adjustments for a user and device. We improve the learning performance by
incorporating an adaptive metering module that links semantics with exposure.
This adaptive metering module generalizes the conventional spot or matrix
metering techniques. We validate our system using the MIT FiveK and our own
datasets captured using iPhone 7 and Google Pixel. Experimental results show
that our system exhibits stable real-time behavior while improving visual
quality compared to what is achieved through native camera control.Comment: 17 pages, 20 figure
Handheld Multi-Frame Super-Resolution
Compared to DSLR cameras, smartphone cameras have smaller sensors, which
limits their spatial resolution; smaller apertures, which limits their light
gathering ability; and smaller pixels, which reduces their signal-to noise
ratio. The use of color filter arrays (CFAs) requires demosaicing, which
further degrades resolution. In this paper, we supplant the use of traditional
demosaicing in single-frame and burst photography pipelines with a multiframe
super-resolution algorithm that creates a complete RGB image directly from a
burst of CFA raw images. We harness natural hand tremor, typical in handheld
photography, to acquire a burst of raw frames with small offsets. These frames
are then aligned and merged to form a single image with red, green, and blue
values at every pixel site. This approach, which includes no explicit
demosaicing step, serves to both increase image resolution and boost signal to
noise ratio. Our algorithm is robust to challenging scene conditions: local
motion, occlusion, or scene changes. It runs at 100 milliseconds per
12-megapixel RAW input burst frame on mass-produced mobile phones.
Specifically, the algorithm is the basis of the Super-Res Zoom feature, as well
as the default merge method in Night Sight mode (whether zooming or not) on
Google's flagship phone.Comment: 24 pages, accepted to Siggraph 2019 Technical Papers progra
Impact of Physical Activity on Sleep:A Deep Learning Based Exploration
The importance of sleep is paramount for maintaining physical, emotional and
mental wellbeing. Though the relationship between sleep and physical activity
is known to be important, it is not yet fully understood. The explosion in
popularity of actigraphy and wearable devices, provides a unique opportunity to
understand this relationship. Leveraging this information source requires new
tools to be developed to facilitate data-driven research for sleep and activity
patient-recommendations.
In this paper we explore the use of deep learning to build sleep quality
prediction models based on actigraphy data. We first use deep learning as a
pure model building device by performing human activity recognition (HAR) on
raw sensor data, and using deep learning to build sleep prediction models. We
compare the deep learning models with those build using classical approaches,
i.e. logistic regression, support vector machines, random forest and adaboost.
Secondly, we employ the advantage of deep learning with its ability to handle
high dimensional datasets. We explore several deep learning models on the raw
wearable sensor output without performing HAR or any other feature extraction.
Our results show that using a convolutional neural network on the raw
wearables output improves the predictive value of sleep quality from physical
activity, by an additional 8% compared to state-of-the-art non-deep learning
approaches, which itself shows a 15% improvement over current practice.
Moreover, utilizing deep learning on raw data eliminates the need for data
pre-processing and simplifies the overall workflow to analyze actigraphy data
for sleep and physical activity research
An iterative scheme for feature based positioning using a weighted dissimilarity measure
We propose an iterative scheme for feature-based positioning using a new
weighted dissimilarity measure with the goal of reducing the impact of large
errors among the measured or modeled features. The weights are computed from
the location-dependent standard deviations of the features and stored as part
of the reference fingerprint map (RFM). Spatial filtering and kernel smoothing
of the kinematically collected raw data allow efficiently estimating the
standard deviations during RFM generation. In the positioning stage, the
weights control the contribution of each feature to the dissimilarity measure,
which in turn quantifies the difference between the set of online measured
features and the fingerprints stored in the RFM. Features with little
variability contribute more to the estimated position than features with high
variability. Iterations are necessary because the variability depends on the
location, and the location is initially unknown when estimating the position.
Using real WiFi signal strength data from extended test measurements with
ground truth in an office building, we show that the standard deviations of
these features vary considerably within the region of interest and are neither
simple functions of the signal strength nor of the distances from the
corresponding access points. This is the motivation to include the empirical
standard deviations in the RFM. We then analyze the deviations of the estimated
positions with and without the location-dependent weighting. In the present
example the maximum radial positioning error from ground truth are reduced by
40% comparing to kNN without the weighted dissimilarity measure.Comment: 18 pages, 9 figures, and 1 tabl
- …