297 research outputs found
HDR-VDP-3: A multi-metric for predicting image differences, quality and contrast distortions in high dynamic range and regular content
High-Dynamic-Range Visual-Difference-Predictor version 3, or HDR-VDP-3, is a
visual metric that can fulfill several tasks, such as full-reference
image/video quality assessment, prediction of visual differences between a pair
of images, or prediction of contrast distortions. Here we present a high-level
overview of the metric, position it with respect to related work, explain the
main differences compared to version 2.2, and describe how the metric was
adapted for the HDR Video Quality Measurement Grand Challenge 2023
Event-based Vision: A Survey
Event cameras are bio-inspired sensors that differ from conventional frame
cameras: Instead of capturing images at a fixed rate, they asynchronously
measure per-pixel brightness changes, and output a stream of events that encode
the time, location and sign of the brightness changes. Event cameras offer
attractive properties compared to traditional cameras: high temporal resolution
(in the order of microseconds), very high dynamic range (140 dB vs. 60 dB), low
power consumption, and high pixel bandwidth (on the order of kHz) resulting in
reduced motion blur. Hence, event cameras have a large potential for robotics
and computer vision in challenging scenarios for traditional cameras, such as
low-latency, high speed, and high dynamic range. However, novel methods are
required to process the unconventional output of these sensors in order to
unlock their potential. This paper provides a comprehensive overview of the
emerging field of event-based vision, with a focus on the applications and the
algorithms developed to unlock the outstanding properties of event cameras. We
present event cameras from their working principle, the actual sensors that are
available and the tasks that they have been used for, from low-level vision
(feature detection and tracking, optic flow, etc.) to high-level vision
(reconstruction, segmentation, recognition). We also discuss the techniques
developed to process events, including learning-based techniques, as well as
specialized processors for these novel sensors, such as spiking neural
networks. Additionally, we highlight the challenges that remain to be tackled
and the opportunities that lie ahead in the search for a more efficient,
bio-inspired way for machines to perceive and interact with the world
Event-Based Motion Segmentation by Motion Compensation
In contrast to traditional cameras, whose pixels have a common exposure time,
event-based cameras are novel bio-inspired sensors whose pixels work
independently and asynchronously output intensity changes (called "events"),
with microsecond resolution. Since events are caused by the apparent motion of
objects, event-based cameras sample visual information based on the scene
dynamics and are, therefore, a more natural fit than traditional cameras to
acquire motion, especially at high speeds, where traditional cameras suffer
from motion blur. However, distinguishing between events caused by different
moving objects and by the camera's ego-motion is a challenging task. We present
the first per-event segmentation method for splitting a scene into
independently moving objects. Our method jointly estimates the event-object
associations (i.e., segmentation) and the motion parameters of the objects (or
the background) by maximization of an objective function, which builds upon
recent results on event-based motion-compensation. We provide a thorough
evaluation of our method on a public dataset, outperforming the
state-of-the-art by as much as 10%. We also show the first quantitative
evaluation of a segmentation algorithm for event cameras, yielding around 90%
accuracy at 4 pixels relative displacement.Comment: When viewed in Acrobat Reader, several of the figures animate. Video:
https://youtu.be/0q6ap_OSBA
Lighting in the third dimension : laser scanning as an architectural survey and representation method
This paper proposes tridimensional (3D) laser scanning to architects and lighting designers as a lighting
enquiry and visualization method for existing built environments. The method constitutes a complement
to existing lighting methods by responding to limitations of photometric measurements, computer
simulation and HDR imagery in surveying and visualizing light in actual buildings. The research explores
advantages and limitations of 3D laser scanning in a case study addressing a vast, geometrically complex
and fragmented naturally and artificially lit space. Lighting patterns and geometry of the case study are
captured with a 3D laser scanner through a series of four scans. A single 3D model of the entire space is
produced from the aligned and fused scans. Lighting distribution patterns are showcased in relation to
the materiality, geometry and position of windows, walls, lighting fixtures and day lighting sources.
Results and presented through images similar to architectural presentation drawings. More specifically,
the lighting distribution patterns are illustrated in a floor plan, a reflected ceiling plan, an axonometry
and a cross-section. The point cloud model of the case study is also generated into a video format
representing the entire building as well as different viewpoints. The study shows that the proposed
method provides powerful visualization results due to the unlimited number of images that can be
generated from a point cloud and facilitates understanding of existing lighting conditions in spaces
Beyond the Pixel: a Photometrically Calibrated HDR Dataset for Luminance and Color Prediction
Light plays an important role in human well-being. However, most computer
vision tasks treat pixels without considering their relationship to physical
luminance. To address this shortcoming, we introduce the Laval Photometric
Indoor HDR Dataset, the first large-scale photometrically calibrated dataset of
high dynamic range 360{\deg} panoramas. Our key contribution is the calibration
of an existing, uncalibrated HDR Dataset. We do so by accurately capturing RAW
bracketed exposures simultaneously with a professional photometric measurement
device (chroma meter) for multiple scenes across a variety of lighting
conditions. Using the resulting measurements, we establish the calibration
coefficients to be applied to the HDR images. The resulting dataset is a rich
representation of indoor scenes which displays a wide range of illuminance and
color, and varied types of light sources. We exploit the dataset to introduce
three novel tasks, where: per-pixel luminance, per-pixel color and planar
illuminance can be predicted from a single input image. Finally, we also
capture another smaller photometric dataset with a commercial 360{\deg} camera,
to experiment on generalization across cameras. We are optimistic that the
release of our datasets and associated code will spark interest in physically
accurate light estimation within the community. Dataset and code are available
at https://lvsn.github.io/beyondthepixel/
Cuboid-maps for indoor illumination modeling and augmented reality rendering
This thesis proposes a novel approach for indoor scene illumination modeling and augmented reality rendering. Our key observation is that an indoor scene is well represented by a set of rectangular spaces, where important illuminants reside on their boundary faces, such as a window on a wall or a ceiling light. Given a perspective image or a panorama and detected rectangular spaces as inputs, we estimate their cuboid shapes, and infer illumination components for each face of the cuboids by a simple convolutional neural architecture. The process turns an image into a set of cuboid environment maps, each of which is a simple extension of a traditional cube-map. For augmented reality rendering, we simply take a linear combination of inferred environment maps and an input image, producing surprisingly realistic illumination effects. This approach is simple and efficient, avoids flickering, and achieves quantitatively more accurate and qualitatively more realistic effects than competing substantially more complicated systems
Recommended from our members
Active sampling, scaling and dataset merging for large-scale image quality assessment
The field of subjective assessment is concerned with eliciting human judgements about a set of stimuli. Collecting such data is costly and time-consuming, especially when the subjective study is to be conducted in a controlled environment and using a specialized equipment. Thus, data from these studies are usually scarce. One of the areas, for which obtaining subjective measurements is difficult is image quality assessment. The results from these studies are used to develop and train automated or objective image quality metrics, which, with the advent of deep learning, require large amounts of versatile and heterogeneous data.
I present three main contributions in this dissertation. First, I propose a new active sampling method for efficient collection of pairwise comparisons in subjective assessment experiments. In these experiments observers are asked to express a preference between two conditions. However, many pairwise comparison protocols require a large number of comparisons to infer accurate scores, which may be unfeasible when each comparison is time-consuming (e.g. videos) or expensive (e.g. medical imaging). This motivates the use of an active sampling algorithm that chooses only the most informative pairs for comparison. I demonstrate, with real and synthetic data, that my algorithm offers the highest accuracy of inferred scores given a fixed number of measurements compared to the existing methods. Second, I propose a probabilistic framework to fuse the outcomes of different psychophysical experimental protocols, namely rating and pairwise comparisons experiments. Such a method can be used for merging existing datasets of subjective nature and for experiments in which both measurements are collected. Third, with a new dataset merging technique and by collecting additional cross-dataset quality comparisons I create a Unified Photometric Image Quality (UPIQ) dataset with over 4,000 images by realigning and merging existing high-dynamic-range (HDR) and standard-dynamic-range (SDR) datasets. The realigned quality scores share the same unified quality scale across all datasets. I then use the new dataset to retrain existing HDR metrics and show that the dataset is sufficiently large for training deep architectures. I show the utility of the dataset and metrics in an application to image compression that accounts for viewing conditions, including screen brightness and the viewing distance
- …