PCA visualization of LSTM hidden layer activity of an example 7-hidden-unit network (#7).

Abstract

Shown is a 9-timestep time course of the 2-back task, running from stimulus n to delay 3:2. Column 1 and 2: timestep labels and example input vectors. Column 3: Time course of dimensionality-reduced LSTM hidden layer activity. Each dot in the figures indicates the unit activity from a single trial. Column 4: Same as Column 3 but now each color corresponds to one of the six stimulus types, indicating stimulus n’s identity, and the black dashed lines illustrate the “schematic” stimulus coding axis. Blue and red squares highlight the two delay timesteps used to identify UMI and PMI dPCs, respectively. Column 5: Same as Column 4 except that the colors now correspond to stimulus n’s status for the n-to-n + 2 comparison that occurs at timestep n + 2 (green: match trials, blue: nonmatch trials). Black dashed line at timestep n + 2 (yellow square) illustrates the decision-coding axis. As can be seen in Column 5, the stimulus coding axis rotates counterclockwise (in the image plane) over time such that it becomes “perpendicular” to the decision axis at timestep n + 1 and aligns with it at timestep n + 2. Percent variance explained: PC1–72.2%, PC2–15.7%.</p