Nanoscale imaging of clinical specimens using pathology-optimized expansion microscopy

Expansion microscopy (ExM), a method for improving the resolution of light microscopy by physically expanding the specimen, has not been applied to clinical tissue samples. Here we report a clinically optimized form of ExM that supports nanoscale imaging of human tissue specimens that have been fixed with formalin, embedded in paraffin, stained with hematoxylin and eosin (H&E), and/or fresh frozen. The method, which we call expansion pathology (ExPath), converts clinical samples into an ExM-compatible state, then applies an ExM protocol with protein anchoring and mechanical homogenization steps optimized for clinical samples. ExPath enables ~70 nm resolution imaging of diverse biomolecules in intact tissues using conventional diffraction-limited microscopes, and standard antibody and fluorescent DNA in situ hybridization reagents. We use ExPath for optical diagnosis of kidney minimal-change disease, which previously required electron microscopy (EM), and demonstrate high-fidelity computational discrimination between early breast neoplastic lesions that to date have challenged human judgment. ExPath may enable the routine use of nanoscale imaging in pathology and clinical research

Correction: Use of Frontal Lobe Hemodynamics as Reinforcement Signals to an Adaptive Controller

Decision-making ability in the frontal lobe (among other brain structures) relies on the assignment of value to states of the animal and its environment. Then higher valued states can be pursued and lower (or negative) valued states avoided. The same principle forms the basis for computational reinforcement learning controllers, which have been fruitfully applied both as models of value estimation in the brain, and as artificial controllers in their own right. This work shows how state desirability signals decoded from frontal lobe hemodynamics, as measured with near-infrared spectroscopy (NIRS), can be applied as reinforcers to an adaptable artificial learning agent in order to guide its acquisition of skills. A set of experiments carried out on an alert macaque demonstrate that both oxy- and deoxyhemoglobin concentrations in the frontal lobe show differences in response to both primarily and secondarily desirable (versus undesirable) stimuli. This difference allows a NIRS signal classifier to serve successfully as a reinforcer for an adaptive controller performing a virtual tool-retrieval task. The agent’s adaptability allows its performance to exceed the limits of the NIRS classifier decoding accuracy. We also show that decoding state desirabilities is more accurate when using relative concentrations of both oxyhemoglobin and deoxyhemoglobin, rather than either species alone

Classifier performace for different data windows and types.

<p>Mean±SEM classifier success rate (equal to the mean of the diagonal elements in the confusion matrices) across 20 experiments inluding both color conditions (n = 776 rewards; n = 683 penalties) for varying sizes of peri-event window, when using different components of the NIRS hemodynamic signal. All windows began at cue onset. Thus, the 0 window duration post event corresponds to the use of 8 seconds of data between cue onset and the outcome event. All other windows include post-outcome data.</p

Experiment and Model Summary.

<p><b>A:</b> The reward signal is derived from the subject's frontal lobe hemodynamics. The Δ[HbO] and Δ[HbD] signals recorded at times around events are classified using a support vector machine (SVM) in order to read out their prediction about the subjective desirability of the event. Any classifier is subject to some misclassification noise (green arrow with red imperfections, and vice versa) so the RL agent that uses this signal as reward information must be robust to occasional misclassifications. Gray inset: The error rates achieved by the SVM classifier in this study were added to the win/loss feedback to a model task in which the reinforcement learning agent had to select actions to be taken by a rake tool in order to achieve the goal of pulling a pellet off of the front side of a table, without knocking it off the back side. The adaptation of the action values for the most recently observed state (and thus the adaptation of the agent's control policy in subsequent visits to that state) is dictated by the reward signal. The agent learns to select the action with the highest expected return, given the current state (i.e. the locations of the pellet and rake tool). <b>B:</b> Brain MRI of the rhesus macaque used in this study. The T1-weighted MRI image (right panel) was registered to a standard atlas (left panel) to locate the DLPFC region of cortex (indicated by the crosshairs). Skull landmarks were then used to localize and place probe guides during implantation. The lower right subpanel shows a 3D reconstruction of the subject's head with dots at the locations of the NIRS probes used as sources (purple) and detectors (red).</p

Comparison between trials with and without significant facial movements.

<p>Conventions as in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0069541#pone-0069541-g004" target="_blank">Figure 4</a>. (A) Peri-event NIRS signals for trials in which no movement was identified on video. (n = 62 rewards; n = 69 penalties) (B) Peri-event NIRS signals for trials in which overt facial movements were observed; see <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0069541#s2" target="_blank">Materials and Methods</a> (n = 35 rewards; n = 24 penalties).</p

Performance of the QSARSA learner when faced with noisy reward signals.

<p>Each bar represents the results for a set of trials with feedback accuracy to the agent as indicated along the horizontal axis. Bar heights represent mean fractions of true reward outcomes (i.e. trial successes) out of 20,000 trials after convergence. Error bars are standard deviations. With increasing reward signal accuracies, the rates of reward improve and the inter-trial variance decreases.</p

Single trial classification performance on NIRS signals.

<p>(A) Confusion matrix for test set prediction performance of SVM classifier using both Δ[HbO] and Δ[HbD] on cued trials with a single color scheme. Results are totals across 15 experimental sessions. Data used is from the cue onset to 15s-post outcome. Each box contains the percentage of test set trials in the “Actual class” that were assigned the label in the “Predicted Class” by the SVM (as labeled in panel C). Absolute numbers of trials are in parentheses. Thus, the successful classifications are on the diagonal. All other panels use the same conventions. (B) Confusion matrix for the same data as in panel A, but with the class labels shuffed. (C) Confusion matrix for classification of unexpected liquid rewards (juice) versus idle baseline (sham events). Totals are across 6 sessions. (D) Confusion matrix for unexpected penalties (vinegar) versus idle baseline (sham events). Totals are across 4 sessions.</p

Hemodynamic responses to uncued rewards and penalties.

<p>(A) Mean±SEM amount of liquids consumed when both were presented ad libitum simultaneously for 20 minutes in the animal's home enclosure on 3 days. No vinegar was consumed on any day. (B) Mean±SEM Peri-event changes in Δ[HbO], Δ[HbD], and Δ[HbTot] relative to baseline for unexpected delivery of 0.5mL of pleasant liquids (pomegranate juice or water) or unpleasant liquid (vinegar). Events delivered at pseudo-random intervals (min 40s). Asterisks indicate times at which the responses in pleasant and unpleasant trials were significantly different (Welchs t-test, p<0.05). (n = 121 rewards; n = 88 penalties).</p

Hemodynamic responses to cued rewards and penalties.

<p>Mean±SEM Peri-event changes in Δ[HbO], Δ[HbD], and Δ[HbTot] relative to baseline for cued delivery of 0.5mL of reward liquid (pomegranate juice) or enforcement of a penalty time-out period (10s of presentation of a stationary red disc). Asterisks indicate times at which the rewarded and penalized trials were significantly different (Welch's t-test, p<0.05). (A) NIRS signals around cue and outcome presentation for blue cues predicting rewards and red cues predicting penalties (n = 658 rewards; n = 588 penalties). (B) NIRS signals around cue and outcome presentation with the color significance reversed: blue cues predict penalties and red cues predict rewards (n = 118 rewards; n = 95 penalties).</p