Patient cohort
The study consisted of 81 consecutive cases of HER2+ breast cancer and was approved in IRB protocol #Pro00032113 at Cedars-Sinai Medical Center. Patients with Stage I-III HER2+ breast cancer who underwent surgery followed by chemotherapy and Trastuzumab from January 2005 through December 2011 were identified from the Cedars-Sinai Medical Center Cancer Registry. Patients who presented with Stage IV disease, whose tumor tissue was not available for marker evaluation, who did not receive follow-up at Cedars-Sinai Medical Center, and who did not receive chemotherapy and Trastuzumab were excluded. HER2 expression levels were determined based on clinical guidelines [23].
Immunohistochemical staining of the same tissue section with CD45 and pan-CK antibodies
The antibodies were used in the sequence of CD45 ➔ Pan-Cytokeratin (Pan-CK) for staining of the same tissue section. For CD45, heat-induced epitope retrieval occurred with Na/EDTA pH 8.0 for ~30 min @ 90 °C. Tissues were blocked with animal-free protein blocking buffer (Vector Laboratories cat. # SP-5030) for 15 min. To quench the endogenous peroxide, the tissue was treated with H2O2 for 12 min. The anti-CD45 (Ventana Pre-Dilute, cat. # 790–2505) was applied for ~30 min @ 37 °C. Thereafter, the EnVision + System – HRP labeled polymer goat anti-mouse secondary antibody (Dako cat. # K400011) was used for 20 min, followed by DAB (3,3′-diaminobenzidine, Vector Laboratories cat. # SK-4100) for 8 min.
Next, slides were incubated with the denaturing buffer (citrate buffer pH 6) for 10 min @ 110 °C, to remove the CD45 antibody. Tissues were blocked with animal-free protein blocking buffer for 16 min. The Pan-CK antibody (Dako cat. # M3515) was diluted at 1:50 and incubated overnight @ 4 °C. The secondary antibody, ImPRESS anti-mouse alkaline phosphatase (Vector Laboratories, cat. # MP-5402), was incubated for 30 min. The chromogen VECTOR Red (Vector Laboratories, cat. # SK-5100) was applied for 10 min. Slides were stained with Modified Mayer’s Hematoxylin (American MasterTech Scientific, cat. # HXMMPT) for 1.5 min and cover-slipped. A micrograph of a region of interest from a slide stained with CD45 and Pan-CK antibodies is shown in Fig. 1a.
Immunohistochemical staining of the same tissue section with CD4, CD8 and CD68 antibodies
One section per patient was subjected to multiplex IHC on the DISCOVERY ULTRA automated slide-stainer (Ventana, Tucson, AZ). The antibodies were used in the sequence of CD4 ➔ CD8 ➔ CD68 for staining of the same tissue section. For CD4, the heat induced epitope retrieval occurred with Na/EDTA pH 8.0 for ~40 min at 90 °C. To quench the endogenous peroxide, the tissue was treated with H2O2 for 12 min. The anti-CD4 (Ventana Pre-Dilute, cat. # 790–4423) was applied for ~40 min at 37 °C. Thereafter, the EnVision + System – HRP labeled polymer goat anti-rabbit secondary antibody (Dako cat. # K400211) was used for 32 min, followed by DAB for 8 min. To quench the remaining active HRP, slides were treated with H2O2 again for 24 min.
The heat-induced epitope retrieval for the next antibody, CD8, was repeated as described above. Tissues were treated with H2O2 for 12 min, followed by anti-CD8 (Ventana Pre-Dilute, cat. # 790–4460) for ~32 min at 37 °C. The EnVision + System – HRP labeled polymer goat anti-rabbit secondary antibody was incubated for 32 min. DISCOVERY Purple (Ventana, cat. # 253–4857) was applied as the chromogen to visualize CD8-antibody binding for 24 min.
Next, slides were incubated with the denaturing buffer (citrate buffer pH 6) for 10 min at 110 °C, to remove the CD8 antibody. Tissues were blocked with animal-free protein blocking buffer for 15 min. The CD68 antibody (Dako cat. # M0876) was diluted at 1:750 and incubated overnight at 4 °C. The secondary antibody, ImPRESS anti-mouse alkaline phosphatase, was incubated for 30 min. The chromogen VECTOR Red was applied for 10 min. Slides were counterstained with modified Mayer’s Hematoxylin for 1.5 min and cover-slipped. A micrograph of a region of interest from a slide stained with CD4, CD8 and CD68 antibodies is shown in Fig. 1e.
Image analysis pipeline
We developed an image analysis workflow (Fig. 2) that employs one slide stained with Pan-CK and CD45 and a slide stained with TIL antibodies (CD8, CD4 and CD68) to quantify distinct immune cell types in regional areas of tumor (Fig. 2a). The workflow involves transferring of tumor mask from the Pan-CK slide to the slide with TIL makers by image registration. The performance of the mask transfer is evaluated against manual ground truth tumor delineations on the slide with TIL markers, and by counting of TILs in regional outlines of the tumor. This pipeline is also used to correlate counts of the TIL constituents inside and outside of the tumor (Fig. 2b).
Imaging of slides and identifying high-density cancer areas
Slides stained with Pan-CK were imaged using a high-resolution whole-slide RGB scanner Aperio AT Turbo with a 20× objective (Leica Biosystems, Vista, CA). To detect regions of interest (ROI) with a high-density of cancer cells, the color-deconvoluted whole slides of Pan-CK images were thresholded to provide an epithelial mask. Only areas of invasive breast carcinoma were analyzed, and image tiles of ductal carcinoma in-situ and normal glands were excluded from the analysis according to the guidelines provided by the International TILs Working Groups [7, 8]. Tumor nuclei detected under a Pan-CK mask were counted to obtain local cell density maps (counts/mm2). The counting was carried out on square image tiles that covered the entire tissue area of the slide. The cell density map was visualized in 3-D to identify ROIs of high tumor cell burden for subsequent analyses (Additional file 1: Fig. S1). Next, coordinates of the ROIs were transferred from the 3-D map to the slide stained with the triplex-IHC. These ROIs was then imaged on the multispectral tissue imaging platform Vectra-II (Perkin-Elmer, Waltham MA). The Vectra-II is equipped with a scientific-grade charge-coupled device monochromatic camera and a 20× objective. The exposure time was 4 ms, and a 1 × 1 pixel binning was used for image acquisition (Nuance, Perkin-Elmer). The spectral image cubes encompassed the cancer areas. Cubes were acquired in the spectral range between 420 nm and 700 nm with 5 nm resolution yielding 31 wavelength specific images per cube. Each image in the cube had 1040 × 1392 pixels (pixel size = 0.5 μm × 0.5 μm) in horizontal and vertical direction. A flat-field correction was applied to prepare the cube for digital unmixing of chromophore colors.
Color deconvolution and spectral unmixing
Two approaches were used for color separation of images scanned with Aperio or Vectra-II slide scanning platforms (Fig. 2). For digital images obtained with the Aperio slide scanner, color deconvolution, a method to transform RGB color tissue images into individual images depicting the concentration of each chromophore, was applied to separate the brown (CD45) and red (Pan-CK) colors from blue (hematoxylin). Briefly, the absorbance values of chromophore mixtures are decomposed into absorbance values of single chromophores. RGB images of the mixture of chromophores in the stained tissue and RGB spectra of single chromophores are provided as respective input data. The fingerprints that specify the RGB components of single chromophores were respectively set for DAB = [0.268, 0.570, 0.776], FastRed = [0.214, 0.851, 0.478], and hematoxylin = [0.490, 0.769, 0.410] in the code color deconvolution ImageJ plugin [24] that was applied to isolate monochromatic images of FastRed (Pan-CK) for further analysis and identification of cancer areas. Antibodies visualized after color deconvoluted images are shown in Fig. 1b-d.
In the second approach, DAB, FastRed, V-purple and hematoxylin images were spectrally unmixed for cancer area image co-registration and cell quantification using the inForm® software of the Vectra-II. Briefly, spectral cubes of cancer areas were digitally unmixed to obtain monochromatic images representing optical density of the chromogens (Fig. 1f-h). Spectral fingerprints specific for individual chromogens were used to determine the proportion of the spectrum from each chromogen in the spectrum of mixed chromogens [25]. The spectral fingerprints were collected from separate slides that were stained with single primary antibodies and corresponding chromogens.
Cancer cell mask
Spectrally unmixed Pan-CK images were exported from the Vectra-II and imported by into an image processing tool that we developed to obtain the cancer cell mask. The Pan-CK image was thresholded using an automated histogram thresholding procedure [26, 27]. The resulting binary image was post-processed. Small objects were removed by applying morphological image opening [28]. Morphological closing was subsequently applied to smoothen the cancer cell mask boundaries. The final mask was exported for the image co-registration procedure.
Image co-registration
We implemented the affine image co-registration procedure [29] in order to transfer the cancer cell mask of the cancer area to the image of the immune infiltrate. The hematoxylin images from both slides were used in the co-registration procedure to calculate the co-registration transformation matrix [29]. The matrix contained a set of parameters to direct the transfer and alignment of the images. After co-registration, the contour of the transferred mask was manually adjusted to resolve discrepancies in tumor architecture between the two slides.
Nuclear segmentation and classification of immune cells
Unmixed images were also imported to inForm 2.0 tissue image analysis software (Perkin-Elmer, Waltham MA) to detect and classify subtypes of immune cells. First, cell nuclei were detected based on the local concentration of hematoxylin and delineated by a nuclear segmentation algorithm (Fig. 2b). The circular nuclear outlines were subsequently expanded by a constant number of pixels which corresponded to the length of half of the mean radius of an immune cell nucleus. The space between the two concentric circles was defined as the cytoplasmic mask. This mask was overlaid respectively onto unmixed images of CD4 (brown), CD8 (purple) and CD68 (red) triple stains to classify these cells for enumeration. The average pixel intensity in the cytoplasmic mask was used to establish a threshold [26]. Cells positive for individual stains were counted and the number of negative cells under the co-registered cancer mask was used as the reference measurement (Fig. 2b).
Regional outlines of tumor
After transferring the tumor mask, we established 3 regions to enumerate the immune infiltrate: 1) the intra-tumoral region as defined by the manually corrected mask, 2) the tumor border region – a ring along the tumor edge measuring 7 pixels (3.5 μm), and 3) the extra-tumoral region – a second ring to the tumor border region measuring 50 pixels (35 μm) (Fig. 2b).
Cancer mask transfer evaluation, data analysis and visualization
Two parameters were measured to assess the performance of cancer mask transfer between slides: a) the overlap ratio (Ov) that represents the normalized number of pixels that are both under the transferred mask and under the ground truth cancer area (delineated by a pathologist) and defined as follows:
$$ Ov=\kern0.5em {\#}_{pixels}\left( CMSK\cap GT\right)/{\#}_{pixels} GT $$
(1)
where: CMSK is the co-registered binary cancer mask, and GT is the binary manual ground truth of cancer mask. ∩ is the logical intersection of the involved masks, and #
pixels
is the pixel count. Ov reaches 1 for the perfect concordance (overlap) of CMSK and GT, and 0 if the masks do not overlap at all.
b) The tumor cell count error (TCe) that was measured as the relative difference between the number of tumor cells under the transferred cancer mask and the number of cells under the ground truth cancer mask, which was manually delineated by the pathologist. TCe is defined as follows:
$$ TCe=\kern0.5em \left|{\#}_{cells} CMSK-{\#}_{cells} GT\right|/{\#}_{cells} GT $$
(2)
where: the #
cells
is the tumor cell count under the respective masks, and | | is the absolute value operator.
TCe reaches 0 in case the number of cells under the respective masks are the same, or 1 in case there are no tumor cells under the transferred mask.
Intra-tumoral, tumor border and extra-tumoral regions were demarcated in 358 ROIs from 81 cases. First, tumor cells were counted in the intra-tumoral region. Cells positive for CD4, CD8, CD68 and CD45 were counted in intra-, border- and extra-tumoral regions. Bland-Altman plots and one-sided t-test were used to investigate differences between immune cell counts underneath the manually annotated versus transferred tumor masks.
In order to investigate the effect of the tumor growth pattern on the mask transfer, three tissue blocks with breast tumors that contained areas of solid and glandular growth pattern were used to obtain consecutive re-cuts from each block. 10 re-cuts from each block were stained with Pan-CK and counterstained with hematoxylin. Areas displaying solid and glandular growth patterns were identified and imaged with the multispectral instrument. The Pan-CK stain was spectrally unmixed and used to build a cancer mask for each image. The overlap Ov between masks was calculated as a function of the distance between the re-cuts and the tumor growth pattern. One-way ANOVA was used to evaluate discrepancies in Ov rates derived from solid and glandular areas.
Image data analysis, performance evaluation, 3-D cell density map visualization and definition of regional outlines of tumor were coded in Matlab programming environment (The MathWorks, Natick, MA). Our previously developed tool for ground truth tissue annotations [30] was used here for the manual cancer mask editing and for generating the ground truth by the pathologist.