9,400 research outputs found
Kernel computations from large-scale random features obtained by Optical Processing Units
Approximating kernel functions with random features (RFs)has been a
successful application of random projections for nonparametric estimation.
However, performing random projections presents computational challenges for
large-scale problems. Recently, a new optical hardware called Optical
Processing Unit (OPU) has been developed for fast and energy-efficient
computation of large-scale RFs in the analog domain. More specifically, the OPU
performs the multiplication of input vectors by a large random matrix with
complex-valued i.i.d. Gaussian entries, followed by the application of an
element-wise squared absolute value operation - this last nonlinearity being
intrinsic to the sensing process. In this paper, we show that this operation
results in a dot-product kernel that has connections to the polynomial kernel,
and we extend this computation to arbitrary powers of the feature map.
Experiments demonstrate that the OPU kernel and its RF approximation achieve
competitive performance in applications using kernel ridge regression and
transfer learning for image classification. Crucially, thanks to the use of the
OPU, these results are obtained with time and energy savings.Comment: 5 pages, 3 figures, submitted to ICASSP 202
Neural system identification for large populations separating "what" and "where"
Neuroscientists classify neurons into different types that perform similar
computations at different locations in the visual field. Traditional methods
for neural system identification do not capitalize on this separation of 'what'
and 'where'. Learning deep convolutional feature spaces that are shared among
many neurons provides an exciting path forward, but the architectural design
needs to account for data limitations: While new experimental techniques enable
recordings from thousands of neurons, experimental time is limited so that one
can sample only a small fraction of each neuron's response space. Here, we show
that a major bottleneck for fitting convolutional neural networks (CNNs) to
neural data is the estimation of the individual receptive field locations, a
problem that has been scratched only at the surface thus far. We propose a CNN
architecture with a sparse readout layer factorizing the spatial (where) and
feature (what) dimensions. Our network scales well to thousands of neurons and
short recordings and can be trained end-to-end. We evaluate this architecture
on ground-truth data to explore the challenges and limitations of CNN-based
system identification. Moreover, we show that our network model outperforms
current state-of-the art system identification models of mouse primary visual
cortex.Comment: NIPS 201
A fast GPU Monte Carlo Radiative Heat Transfer Implementation for Coupling with Direct Numerical Simulation
We implemented a fast Reciprocal Monte Carlo algorithm, to accurately solve
radiative heat transfer in turbulent flows of non-grey participating media that
can be coupled to fully resolved turbulent flows, namely to Direct Numerical
Simulation (DNS). The spectrally varying absorption coefficient is treated in a
narrow-band fashion with a correlated-k distribution. The implementation is
verified with analytical solutions and validated with results from literature
and line-by-line Monte Carlo computations. The method is implemented on GPU
with a thorough attention to memory transfer and computational efficiency. The
bottlenecks that dominate the computational expenses are addressed and several
techniques are proposed to optimize the GPU execution. By implementing the
proposed algorithmic accelerations, a speed-up of up to 3 orders of magnitude
can be achieved, while maintaining the same accuracy
LightOn Optical Processing Unit: Scaling-up AI and HPC with a Non von Neumann co-processor
We introduce LightOn's Optical Processing Unit (OPU), the first photonic AI
accelerator chip available on the market for at-scale Non von Neumann
computations, reaching 1500 TeraOPS. It relies on a combination of free-space
optics with off-the-shelf components, together with a software API allowing a
seamless integration within Python-based processing pipelines. We discuss a
variety of use cases and hybrid network architectures, with the OPU used in
combination of CPU/GPU, and draw a pathway towards "optical advantage".Comment: Proceedings IEEE Hot Chips 33, 202
Invariance of visual operations at the level of receptive fields
Receptive field profiles registered by cell recordings have shown that
mammalian vision has developed receptive fields tuned to different sizes and
orientations in the image domain as well as to different image velocities in
space-time. This article presents a theoretical model by which families of
idealized receptive field profiles can be derived mathematically from a small
set of basic assumptions that correspond to structural properties of the
environment. The article also presents a theory for how basic invariance
properties to variations in scale, viewing direction and relative motion can be
obtained from the output of such receptive fields, using complementary
selection mechanisms that operate over the output of families of receptive
fields tuned to different parameters. Thereby, the theory shows how basic
invariance properties of a visual system can be obtained already at the level
of receptive fields, and we can explain the different shapes of receptive field
profiles found in biological vision from a requirement that the visual system
should be invariant to the natural types of image transformations that occur in
its environment.Comment: 40 pages, 17 figure
- …