Dojat, Michel

English

Archive Ouverte en Sciences de l'Information et de la Communication

Cliquez et modifiez le titre 
en Arial 24 pt gras
Machine Learning & Neurosciences
Michel Dojat
michel.dojat@inserm.fr
/55
Artificial Intelligence
2
• How to build an artificial intelligent agent?
• Test our models of natural intelligent agents?
Intelligent Agent: an entity that takes the best possible 
action in a situation
NeurosciencesComputer science
Integration of heterogeneous datasets
Management of large repositories of data & 
knowledge
Knowledge discovery
Learning
ConsciousPerception
Memory
Attention
/55
Computer & Brain
3
1958
Yale University Press, New Haven
1963
1982
1988
/55
Two main approaches
4
• Machine Learning
• Bio-inspired
⎼Artificial life
⎼Neural Networks
⎼ WMcCulloch & W Pitts (1943)
Artificial neurons
⎼ D Hebb (1949)
Learning by modification of connections
⎼ F Rosenblatt (1963)
Convergence theorem
⎼ M Minsky & S Paper
Perceptrons (1969)
• Classification (SVM,…)
• Symbolic Processing
⎼Problem-solving
⎼Planning
⎼Logic
⎼Knowledge representation
⎼ Common knowledge
⎼ Meta-knowledge
⎼ Ontology
⎼Multi-agents
⎼Co-construction
Operations on large vectors Rule-based manipulation of symbols
/555
But …
Mycin
Shortliffe 1976
/55
ML: Natural extension of traditional statistical approaches
6
Deep learning
Classic machine learning
Risk calculators
Knowledge-Based systems
Other
Beam & Kohane Nature 2018
/55
ML Approaches
7
Learns known patterns
Predicts outcome
Learns unknown patterns
Find hidden patterns
Labeled
Unlabeled
Reinforcement learning Generates data
Interacts with environment
Labeled
Transfert learning al
Applies learned patterns to
a different but related task
[From Choy et al. 
Radiology 2017]
/55
ML Approaches
8
Learns known patterns
Predicts outcome
Learns unknown patterns
Find hidden patterns
Labeled
Unlabeled
Reinforcement learning Generates data
Interacts with environment
Labeled
Transfert learning
Applies learned patterns to
a different but related task
[From Choy et al. 
Radiology 2017]
/55
Neurosciences - ML Cross fertilization 
9
• The Human Visual System
• Encoding / Decoding
• Deep Learning in NN
• CNN as Models of the Visual System
• Discussion - Perspectives
/55
Mammalian visual system
10
[adapted from Cox & Dean curr bio 2014]
Simultaneous and parallel computing
J. Fabre 2012
J. Fabre 2012
/5511
Human visual system
[Dakin & Frith Neuron 2005]
CORTEX
[Escher, Balcony 1945]
/55
Brain connectivity
12
[Serre et al 2005 Tech Report]
V1
V2
V3
V4
< 2%
V5
- Local features
- Global grouping
- Hierarchical model
- Several areas interconnected (layers)
- Receptive field sizes increasing
/55
MAX pooling
13
[Riesenhuber & Poggio Nature 1999]
MAX pooling: invariance to scale and translation; key mechanism for object recognition
[Serre et al 2005 Tech Report]
/55
Visual representation in the Human Brain
14
Bold
responses
Model :
- Features
- Processing
/55
Functional imaging: Encoding models
15
1
0
Temps
p<0.05 è 2500 false positive !
Univariate Voxel Analysis
/55
Some examples
16
t = 20s
Localizers
t = 20s
V5
V4
LOC
PPA
FFA
[F Bacon 1976]
/5517
Cortical activation – Space localisation
/5518
[Polimeni et al. NeuroIm 2010]
Cortical activation – Space localisation
/55
Pattern recognition
19
[Haynes and Rees Nat Neuro 2006]
/55
Pattern recognition
20
Risque d’overfitting
f(x) =w+b
f(x*) = wx*+B
[Haynes and Rees Nat Neuro 2006]
/55
Mind Reading
21
[Wandell Nature 2008]
Identifying natural images from
human brain activity
Kendrick N. Kay, Thomas Naselaris, 
Ryan J. Prenger & Jack L. Gallant
Nature Vol 452|20 March 2008
A general brain reading device to access the visual contents of 
purely mental phenomena such as dreams and imagery
/55
Water color effect
22
Pinna et al., 2001.
/55
Water color effect
23
Gérardin et al., Neuroimage 2018
/55
Method
24
Design	Matrix
GLM
Activation	for	all	
visual	 conditions
Most	activated	voxels	in	
each	delineated	area
Differential	time	course	for	the	selected	
voxels	in	each	ROI	for	each	condition	
p<0.05	uncor.
A
Edge-Dependent	
onsets
Control	onsets
Surface-Dependent		
onsets
C Dynamic	Causal	Modeling+	Bayesian	Model	SelectionClassification
Six-fold	
cross
validation
B
Support
Vector
Machine
/55
Results
25
I II III
V1
0.45
0.5
0.55
0.6
0.65
0.7
0.75
P
re
di
ct
io
n 
A
cc
ur
ac
y
*
*
I II III
V2
0.45
0.5
0.55
0.6
0.65
0.7
0.75
I II III
V3v
0.45
0.5
0.55
0.6
0.65
0.7
0.75
P
re
di
ct
io
n 
A
cc
ur
ac
y
*
*
I II III
hV4
0.45
0.5
0.55
0.6
0.65
0.7
0.75 *
*
I II III
LO
0.45
0.5
0.55
0.6
0.65
0.7
0.75
*
*
I II III
V3d
0.45
0.5
0.55
0.6
0.65
0.7
0.75
P
re
di
ct
io
n 
A
cc
ur
ac
y
I II III
V3A
0.45
0.5
0.55
0.6
0.65
0.7
0.75 *
*
I II III
V7
0.45
0.5
0.55
0.6
0.65
0.7
0.75
I II III
V3B/KO
0.45
0.5
0.55
0.6
0.65
0.7
0.75 *
*
I II III
hMT+/V5
0.45
0.5
0.55
0.6
0.65
0.7
0.75
I: Edge-dependent vs Control
II: Surface-dependent vs Control
III: Surface-dependent vs Edge-dependent
Retinotopic areas
Ventral visual areas
Dorsal visual areas
vs
vs
vs
nsnsnsnsnsnsns
Multi-Voxel Pattern Analysis (MVPA) from fMRI data.
/55
Conclusion
26
– Filling-in is best classified and best correlate with appearance by 
dorsal areas V3A & V3B/KO
– Uniform chromaticity by ventral areas hV4 & LO
– Feedback modulation from V3A to V1 and LO for filling-in
– Feedback from LO modulating V1 and V3A for uniform chromaticity
LO
V1
V3A
LO
V1
V3A
/55
Color decoding
27
/55
Neural Networks
28
Artificial neurons (bio-inspirés)
x: observations, w: weights, y: output
f: models basis or activation functions 
/5529
Multi-layer network
Lillicrap Nat Comm 2016
W
WT
/55
Convolutional Neural Network
30
Hierarchical extraction of information
Features of filters learnt
Multi-scale representation
[Fukushima « Neocognitron » 1980 Biol Cybern]  
/5531
V1-V2
V4
IT
FFA
PPA
[Dojat Lavoisier ed. 2017]
ü Receptive field
ü Hierarchical model
ü Max pooling
CNN as a model of the visual system …
[Yamins and Dicarlo Nat Neuro 2016]
/55
Hierarchical models for neural responses 
prediction
32
[Yamins et al 2014 PNAS]
/55
Hierarchical models for neural responses 
prediction
33
[Yamins et al 2014 PNAS]
Representation dissimilarity matrices
/55
Hierarchical models for functional responses 
prediction
34
[Eickenberg Neuroimage 2017]]
OverFeat
6 convolutional layers
3 fully connected
Data:
Kay et al Nature 2008
/55
Hierarchical models for functional 
responses prediction
35
[Eickenberg Neuroimage 2017]
[Güclü and van Gerven J Neuro 2015; Cadena et al 2017 bioRxiv; 
Cichy et al 2017 Scient Rep; Greene et al PLOS 2018; 
Seeliger et al . NeuroIm 2018; Wen et al Scient Rep 2018]
Data:
Naselaris et al Neuron
2009
/55
Hierarchical models for functional 
responses prediction
36
[Eickenberg Neuroimage 2017]
/55
Seen/imagined object arbitrary categories
37
[Horikawa and Kamitani Nat Comm 2018]
Train: 1200 im/50 obj cat (x24)
Test: 50 im/50 obj cat (x35)
9 min each->9h15
1im/50 obj cat  (x10)
Idem test
10 min each ->3h33
Decoding accuracy
CNN1…8
HMAX1..3
GIST, SIFT+BoF
/55
Seen/imagined object arbitrary categories
38
[Horikawa and Kamitani Nat Comm 2018]
CNN1…8
HMAX1..3
GIST, SIFT+BoF
Train: 1200 im/50 obj cat (x24)
Test: 50 im/50 obj cat (x35)
9 min each->9h15
1im/50 obj cat  (x10)
Idem test
10 min each ->3h33
/55
Dynamic natural vision
39
[Wen el al CC 2017]
2.4h Training movie x 2
40 min Testing movie x10
3 subjects, 3T scanner
3.5mm3, 220x220mm2
AlexNet 8 layers
bird
face
scene
ship
96 256 384 384 256
11
5
3
3
3
4096 4096 15
227 x 227
55 x 55 27 x 27 13 x 13
13 x 13 13 x 13
pooling
poolingpooling
stride of 4
input L1 L2 L3 L4 L5 L6 L7 L8
/55
Exploring visual representation
40
2000 human 
faces
[Wen et al. Scient Rep 2018]
64 000 images
80 classes
800 categories
/55
Nobody is perfect …
41
blur
noise
[Dodge and Karam ICCV 2017]
/55
CNN as a model of the visual system …
42
• CNN driven for image recognition 
• Explains significant variance of neuroimaging data in 
ventral stream
• Models a  hierarchical representation of feedforward 
visual information processing (i.e. ventral stream)
• Supports the generation of expected cortical 
activation
• Supports decoding & then semantic categorisation
• Validates the existing models
BUT still incomplete …
/5543
[Lecun et al 2010]
ü Receptive field
ü Hierarchical model
ü Max pooling
ü Layer2layer conn.
ü Magnitude factor
ü Color
ü Metamer
ü Attention
ü Local vs Global
ü Perspective effects
ü Learning
ü Illusion
ü Eye mvt
CNN as a model of the visual system …
[Nguyen et al CVPR 2015]
[Dali 1941] [Bacon 1972]
/55
Illusions - Hallucinations
44
[Ffytche 2007 Dial Clin Neurosc] DeepDream.com
/5545
CNN a model for … auditory cortical responses
For ≠ added
noises
[Kell et al Neuron 2018]
/55
AI for catching human errors
46
Makari et al BMJ 2016
Medical error-the third leading cause of 
death in the US
/55
New tools …
47
MS lesion
http://clickme.ai/
/55
Art History
48
Elgammal et al. 2018 arxiv 1801.07729
How characteristics of style are identified?
How the patterns evolve?
76921 paintings
Train(85%)
Val (9.5%)
Test (5.5%)
/55
e-Nosology
49
MS
Stroke. 
TBI
ALZ
PD
How characteristics of pathology are identified?
How the patterns evolve?
« to discover
fundamental patterns and trends not 
necessarily apparent to the individual
human eye »
/55
New models for neuroscience
50
from Mike Hawrylycz
How from the activity of millions of neurons distributed in several brain areas emerge
simple percepts and how they are linked to emotion, motivation or action?
Big Data Analytics
Ontology Based Data Access
/55
[Silver et al Nature 2016]
Mixing Hype AI (Supervised L+ Renforcement L) + Old fashion AI (Tree search)
Future
Learning
ConsciousPerception
Memory
Attention
/55
Consciousness model
52
Global 
workspace 
theory 
[Baars 1988, 
97, 2002]
Soar: 
[Newell 1983, 92 ]
Global ignition
[Dehaene 2003]
[Dehaene and Changeux Neuron 2011]
/55
The Unreasonable Effectiveness of Data
53
• Classifiers based on millions of specific 
features perform better than elaborate models 
that try to discover general rules.
[Halevy et al IEEE Intell Syst 2009]
/55
But …
54
/55
But …
55
/55


Machine Learning & Neurosciences

https://hal.archives-ouvertes.fr/hal-02165616/file/ML%26NS_MD2018.pdf

Machine Learning & Neurosciences

Abstract

Similar works

Full text

Available Versions

Archive Ouverte en Sciences de l'Information et de la Communication