2 research outputs found
Decoding Generic Visual Representations From Human Brain Activity using Machine Learning
Among the most impressive recent applications of neural decoding is the
visual representation decoding, where the category of an object that a subject
either sees or imagines is inferred by observing his/her brain activity. Even
though there is an increasing interest in the aforementioned visual
representation decoding task, there is no extensive study of the effect of
using different machine learning models on the decoding accuracy. In this paper
we provide an extensive evaluation of several machine learning models, along
with different similarity metrics, for the aforementioned task, drawing many
interesting conclusions. That way, this paper a) paves the way for developing
more advanced and accurate methods and b) provides an extensive and easily
reproducible baseline for the aforementioned decoding task.Comment: Accepted at 1st Workshop on Brain-Driven Computer Vision - ECCV 201
Category decoding of visual stimuli from human brain activity using a bidirectional recurrent neural network to simulate bidirectional information flows in human visual cortices
Recently, visual encoding and decoding based on functional magnetic resonance
imaging (fMRI) have realized many achievements with the rapid development of
deep network computation. Despite the hierarchically similar representations of
deep network and human vision, visual information flows from primary visual
cortices to high visual cortices and vice versa based on the bottom-up and
top-down manners, respectively. Inspired by the bidirectional information
flows, we proposed a bidirectional recurrent neural network (BRNN)-based method
to decode the categories from fMRI data. The forward and backward directions in
the BRNN module characterized the bottom-up and top-down manners, respectively.
The proposed method regarded the selected voxels of each visual cortex region
(V1, V2, V3, V4, and LO) as one node in the sequence fed into the BRNN module
and combined the output of the BRNN module to decode the categories with the
subsequent fully connected layer. This new method allows the efficient
utilization of hierarchical information representations and bidirectional
information flows in human visual cortices. Experiment results demonstrated
that our method improved the accuracy of three-level category decoding than
other methods, which implicitly validated the hierarchical and bidirectional
human visual representations. Comparative analysis revealed that the category
representations of human visual cortices were hierarchical, distributed,
complementary, and correlative.Comment: submitted to the Frontiers in neuroscienc