Gates, Monica

Kourtzi, Zoe

Perez Pozuelo, Ignacio

Wang, Rui

Welchman, Andrew

Apollo (Cambridge)

Learning predictive temporal structure under uncertaintyRui Wang1, Monica Gates2, Ignacio Perez-Pozuelon2, Kaijia Sun3, Wenxu Wang3, Yuan Shen4, Peter Tino4, Andrew Welchman2 & Zoe Kourtzi23 School of Systems Science, Beijing Normal University, CN 4 School of Computer Science, University of Birmingham, UKPoster 085.181 Institute of Psychology, Chinese Academy of Sciences, CN 2 Department of Psychology, University of Cambridge, UKINTRODUCTIONExperience is thought to facilitate our ability to extract structures from streams of events. We have shown that extracting complex temporal regularities relates to individual decision strategy (matching vs maximization) (Wang et al., 2017). Here we test whether this ability for learning predictive structures is maintained under uncertainty and whether it generalizes.TaskSequencesSequence (8-14 items) Cue ResponsePredict which symbol follows the sequence?METHODStimeA BCDTargetContextLevel-1AB0.80.20.8CD0.80.20.20.20.8A B C DRESULTSWe trained participants with sequences of symbols determined by first-order Markov models (i.e. level-1), where the symbol at time i is determined probabilistically by the immediately preceding symbol. 10 15 20 25Block0  0.20.40.60.81.0  Peformance index500.10.20.30.40.5 PrePostNormalized performance indexExperiment 2.1: Probability uncertainty We decreased the discriminability of symbol probabilities (i.e. changing the previous marginal probabilities from 80% vs. 20% to 60% vs.40%).10 15 20 25Block0  0.20.40.60.81.0  Peformance index5-0.4-0.20.20.40.60.8Strategy indexNo jitterJitter0Conditional probability matrix00.10.20.30.40.5 PrePostNormalized performance index10 15 20 25Block0  0.20.40.60.81.0  Peformance index5 20% vs 80%40% vs 60%TargetContextLevel-1AB0.60.40.6CD0.60.40.40.40.6A B C D10 15 20 25Block0  0.20.40.60.81.0  Peformance index5 10 15 20 25Block0  0.20.40.60.81.0  Peformance index5 10 15 20 25Block0  0.20.40.60.81.0  Peformance index500.10.20.30.40.5 PrePostNormalized performance indexUncorrTrial None-0.4-0.20.20.40.60.8Strategy index0UncorrTrial None-0.2 0 0.2 0.4 0.6-0.200.20.40.6-0.2 0 0.2 0.4 0.6-0.400.20.40.60.8-0.2PI improvemnt - New symbolsTrial-by-trialStrategy indexPI improvement -Trained Modelling feedback learningtime0.1s0.5s0.1s0.3s0.1s0.4s0.1s0.4s0.1s0.4s0.1s Probability of occurrence was important for structure learning, and decreasing the probability contrast of the items compromised performance in the prediction task.Experiment 2.2: Uncertainty in stimulus presentation rate Increasing uncertainty in stimulus presentation rate  by temporal jittering did not impair observers’ performance, but led observers to adopt a strategy closer to probability maximization.Experiment 2.3: Feedback uncertainty Three groups of observers were provided with trial-by-trial feedback based on whichever symbol was correctly predicted by pre-defined sequences, no feedback and uncorrelated feedback, respectively. Feedback played an important role in predictive learning: Trial-by-trial feedback yielded a larger improvement of performance than no feedback, and encouraged participants to adopt a strategy closer to maximization. Providing uncorrelated feedback resulted in limited improvement.None Uncorrelated00.10.20.30.40.5Normalized performance indexPrePostNew  We manipulated ‘uncertainty’ in the sequences in three respects: probability of symbol occurrence, stimulus presentation rate and feedback, and have demonstrated how learning changes under uncertainty.   Our results suggest that adopting maximization reduces uncertainty when learning in variable environments. Further, maximization facilitates predictive learning and generalization.SUMMARYLearner (N=3)Weak learner (N=12)Learner (N=13)Weak learner (N=5)Learner (N=17)Weak learner (N=1)Learner (N=12)Weak learner (N=3)Learner (N=8)Weak learner (N=7)Random guessLearner (N=13)Weak learner (N=5)Experiment 1: BaselineExperiment 2: Learning under uncertaintyExperiment 3: Does learning transfer to novel symbols? We first replicated the previous findings of learning temporal statistics, that is observers succeed in extracting regularities and making predictions over multiple training sessions. Behavioural analysisPerformance index (PI): We assessed human responses by quantifying how closely the probability distribution of participant responses matches the presented symbols.Presp:   Participants’ response distribution for target i per contextPpres:   Presented distribution for target i per contextPI (Presp,Ppres) =  min ( Presp(i), Ppres(i) )iStrategy choice: We computed a strategy index that indicates each observer’s preference for responding using probability matching (i.e. match the exact sequence statistics) vs. maximization (i.e. predict the most probable outcome for each context). After training, observers were tested with a new sequence of stimuli comprising four different symbols. It was shown that predictive learning of temporal statistics transferred fully to distinct new stimuli.UntrainedTrained Correlating individual strategies with learning and transfer performance showed that observers who  adopted the maximization strategy showed improved performance and higher learning transfer.PI improvement -Trained Sequence viewingREFERENCESWang, R., Shen, Y., Tino, P., Welchman A. E. & Kourtzi Z. (2017). Learning predictive statistics from temporal sequences: dynamics and strategies. Journal of Vision, 17(12):1, 1-16 FeedbackDecision making10 15 20 25Block00.20.40.60.81.0Peformance indexTrialNoneUncorr5I. II.Observers extract statistics (level-0 and level-1) from  blocks of sequences (chunk) during sequence viewing To understand how predictive learning of statistics evolves under various uncertainty conditions (e.g. feedback) , we created a model which integrates chunk learning and reinforcement learning processes.Extenal learning is reinforced by feedback which is given for each human choicefitnessCorrect responsesIncorrect responsesv - intensity of feedbackk - noiseTo update the protocol and make a choiceObservers update the models through the two learning processes and make the most probable choicesModel simulationfor each symbol ,chunks of a certain memory length n,Internal learning is weighted by the probability of occurrence of each symbol (level-0) or adjacent pairs (level-1) over chunks

Learning predictive temporal structure under uncertainty

https://www.repository.cam.ac.uk/bitstream/1810/288427/1/sfn2018_Markov_uncertainty_v4.pdf

Learning predictive temporal structure under uncertainty

Abstract

Similar works

Full text

Available Versions

Apollo (Cambridge)