Automatic labeling of molecular biomarkers of immunohistochemistry images using fully convolutional networks - Fig 5

<p>Heat-maps resulting from biomarker labeling on an example IHC image (a) using WI-Net. The IHC image (size 2048×2048 pixels) belonging to a whole slide image obtained from the scanner was fed to the WI-Net (a). The outputs of the WI-Net were the heat-maps (size 2048 × 2048 pixels) marking p16 positive regions (b), Ki-67 positive regions (c), p16 and Ki-67 positive regions (d), p16 and Ki-67 negative regions (e). The four heat-maps were combined to produce an overall biomarker heat-map (f).</p

Comparison between color deconvolution approach and WI-Net approach for locating p16 and Ki-67 positive pixels in two IHC images.

<p>The IHC image on the left two columns corresponds to a high-grade cervical lesion. The IHC image on the right two columns corresponds to a normal cervical epithelium. The first row shows the RGB images. The second row shows the regions marked as p16 positive by the two methods. The third row shows the regions marked as Ki-67 positive by the two methods.</p

Distribution of nuclei in the training and test sets.

<p>Distribution of nuclei in the training and test sets.</p

The ROI selected on an IHC image (left), and evenly distributed layers in the ROI parallel to the basal layer (right).

<p>The ROI selected on an IHC image (left), and evenly distributed layers in the ROI parallel to the basal layer (right).</p

WI-Net performance evaluation of two IHC images taken from two WSIs: Based on biomarker heat-maps generated by WI-Net.

<p>The ground truth is taken as the labels obtained manually by humans.</p

A two-step approach to biomarker labeling in IHC images.

<p>Step one (top): N-Net was trained with nucleus images segmented from IHC images in the training set, where each image had one nucleus only. This network learned to classify each nucleus image according to the expressed biomarker(s) in its respective nucleus. Step two (bottom): the trained classifier (N-Net) was extended to WI-Net. This end-to-end, pixel-to-pixel network localized biomarkers of the IHC images. The input to the WI-Net is an IHC image and the output is a heat-map of the two biomarker expression.</p

The average percentage of pixels classified as Ki-67 positive (top left), p16 positive (top right), and both Ki-67 and p16 positive (bottom) in different layers of images of cervical epithelium as assessed for each pathology grade.

<p>Layer 1 is the first layer (i.e. basal layer) and layer 16 is the superficial layer.</p

Schematic representation of the architecture of the N-Net.

<p>N-Net consists of one convolution layer, followed by a max-pooling layer and two fully connected layers. The last layer is the output layer. This network takes an RGB image of a nucleus as input and generates a label as the output.</p

Schematic representation of the architecture of the WI-Net.

<p>The first two layers of WI-Net (convolution and max-pooling layers) are the same as the first two layers of N-Net illustrated in <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0190783#pone.0190783.g002" target="_blank">Fig 2</a>. The last three layers of WI-Net are convolution layers. The input to the WI-Net is a whole IHC image, and the output is a heat-map of the present biomarker(s) in the IHC image.</p

Sample of nuclei images obtained from an IHC image.

<p>ROI (red enclosure) selected in an IHC image, within which the nuclei were segmented in order to obtain nuclei images for training N-Net (left). Nuclei images expressing different proteins (right); p16 positive, Ki-67 positive, p16 and Ki-67 positive, and p16 and Ki-67 negative.</p