8 research outputs found
A Temporally Consistent Image-based Sun Tracking Algorithm for Solar Energy Forecasting Applications
Improving irradiance forecasting is critical to further increase the share of
solar in the energy mix. On a short time scale, fish-eye cameras on the ground
are used to capture cloud displacements causing the local variability of the
electricity production. As most of the solar radiation comes directly from the
Sun, current forecasting approaches use its position in the image as a
reference to interpret the cloud cover dynamics. However, existing Sun tracking
methods rely on external data and a calibration of the camera, which requires
access to the device. To address these limitations, this study introduces an
image-based Sun tracking algorithm to localise the Sun in the image when it is
visible and interpolate its daily trajectory from past observations. We
validate the method on a set of sky images collected over a year at SIRTA's
lab. Experimental results show that the proposed method provides robust smooth
Sun trajectories with a mean absolute error below 1% of the image size.Comment: Accepted as a workshop paper at NeurIPS 202
Improving data-driven global weather prediction using deep convolutional neural networks on a cubed sphere
We present a significantly-improved data-driven global weather forecasting
framework using a deep convolutional neural network (CNN) to forecast several
basic atmospheric variables on a global grid. New developments in this
framework include an offline volume-conservative mapping to a cubed-sphere
grid, improvements to the CNN architecture, and the minimization of the loss
function over multiple steps in a prediction sequence. The cubed-sphere
remapping minimizes the distortion on the cube faces on which convolution
operations are performed and provides natural boundary conditions for padding
in the CNN. Our improved model produces weather forecasts that are indefinitely
stable and produce realistic weather patterns at lead times of several weeks
and longer. For short- to medium-range forecasting, our model significantly
outperforms persistence, climatology, and a coarse-resolution dynamical
numerical weather prediction (NWP) model. Unsurprisingly, our forecasts are
worse than those from a high-resolution state-of-the-art operational NWP
system. Our data-driven model is able to learn to forecast complex surface
temperature patterns from few input atmospheric state variables. On annual time
scales, our model produces a realistic seasonal cycle driven solely by the
prescribed variation in top-of-atmosphere solar forcing. Although it is
currently less accurate than operational weather forecasting models, our
data-driven CNN executes much faster than those models, suggesting that machine
learning could prove to be a valuable tool for large-ensemble forecasting.Comment: Manuscript submitted to Journal of Advances in Modeling Earth System
Orientation-aware semantic segmentation on icosahedron spheres
© 2019 IEEE. We address semantic segmentation on omnidirectional images, to leverage a holistic understanding of the surrounding scene for applications like autonomous driving systems. For the spherical domain, several methods recently adopt an icosahedron mesh, but systems are typically rotation invariant or require significant memory and parameters, thus enabling execution only at very low resolutions. In our work, we propose an orientation-aware CNN framework for the icosahedron mesh. Our representation allows for fast network operations, as our design simplifies to standard network operations of classical CNNs, but under consideration of north-aligned kernel convolutions for features on the sphere. We implement our representation and demonstrate its memory efficiency up-to a level-8 resolution mesh (equivalent to 640 x 1024 equirectangular images). Finally, since our kernels operate on the tangent of the sphere, standard feature weights, pretrained on perspective data, can be directly transferred with only small need for weight refinement. In our evaluation our orientation-aware CNN becomes a new state of the art for the recent 2D3DS dataset, and our Omni-SYNTHIA version of SYNTHIA. Rotation invariant classification and segmentation tasks are additionally presented for comparison to prior art
Learning Equivariant Representations
State-of-the-art deep learning systems often require large amounts of data
and computation. For this reason, leveraging known or unknown structure of the
data is paramount. Convolutional neural networks (CNNs) are successful examples
of this principle, their defining characteristic being the shift-equivariance.
By sliding a filter over the input, when the input shifts, the response shifts
by the same amount, exploiting the structure of natural images where semantic
content is independent of absolute pixel positions. This property is essential
to the success of CNNs in audio, image and video recognition tasks. In this
thesis, we extend equivariance to other kinds of transformations, such as
rotation and scaling. We propose equivariant models for different
transformations defined by groups of symmetries. The main contributions are (i)
polar transformer networks, achieving equivariance to the group of similarities
on the plane, (ii) equivariant multi-view networks, achieving equivariance to
the group of symmetries of the icosahedron, (iii) spherical CNNs, achieving
equivariance to the continuous 3D rotation group, (iv) cross-domain image
embeddings, achieving equivariance to 3D rotations for 2D inputs, and (v)
spin-weighted spherical CNNs, generalizing the spherical CNNs and achieving
equivariance to 3D rotations for spherical vector fields. Applications include
image classification, 3D shape classification and retrieval, panoramic image
classification and segmentation, shape alignment and pose estimation. What
these models have in common is that they leverage symmetries in the data to
reduce sample and model complexity and improve generalization performance. The
advantages are more significant on (but not limited to) challenging tasks where
data is limited or input perturbations such as arbitrary rotations are present
Coordinate Independent Convolutional Networks -- Isometry and Gauge Equivariant Convolutions on Riemannian Manifolds
Motivated by the vast success of deep convolutional networks, there is a
great interest in generalizing convolutions to non-Euclidean manifolds. A major
complication in comparison to flat spaces is that it is unclear in which
alignment a convolution kernel should be applied on a manifold. The underlying
reason for this ambiguity is that general manifolds do not come with a
canonical choice of reference frames (gauge). Kernels and features therefore
have to be expressed relative to arbitrary coordinates. We argue that the
particular choice of coordinatization should not affect a network's inference
-- it should be coordinate independent. A simultaneous demand for coordinate
independence and weight sharing is shown to result in a requirement on the
network to be equivariant under local gauge transformations (changes of local
reference frames). The ambiguity of reference frames depends thereby on the
G-structure of the manifold, such that the necessary level of gauge
equivariance is prescribed by the corresponding structure group G. Coordinate
independent convolutions are proven to be equivariant w.r.t. those isometries
that are symmetries of the G-structure. The resulting theory is formulated in a
coordinate free fashion in terms of fiber bundles. To exemplify the design of
coordinate independent convolutions, we implement a convolutional network on
the M\"obius strip. The generality of our differential geometric formulation of
convolutional networks is demonstrated by an extensive literature review which
explains a large number of Euclidean CNNs, spherical CNNs and CNNs on general
surfaces as specific instances of coordinate independent convolutions.Comment: The implementation of orientation independent M\"obius convolutions
is publicly available at https://github.com/mauriceweiler/MobiusCNN