Material
A training set (n = 6) and a test set (n = 12) of hormone receptor negative ductal invasive breast cancer tissue samples were selected and 2-4 µm thick paraffin sections were cut and stained using automated ER and PR detection (Ventana Benchmark Ultra) with diaminobenzidine (DAB) as chromogen. Slide scanning was performed using a whole-slide scanner at 20× magnification (Aperio ScanScope, Leica Microsystems, Wetzlar, Germany), and saved in a proprietary (svs) format using JPEG2000 compression. In preparation of knowledge-based specific feature selection, image annotation for the purpose of interdisciplinary discussion of relevant tissue structures was performed using the freely available Aperio ImageScope software (Leica Microsystems). Studies were performed on anonymized archival paraffin-embedded material. The study was approved by the local Ethics Committee.
Method
The image analysis software Developer XD2 (Definiens, Munich, Germany) was used for image analysis. A rule set was developed based on knowledge-based selection of modular image analysis algorithms and parameters, e.g. for segmentation and hierarchical classification, using the functionality of the software package that facilitates translation of recurrent image features assigned to biologically relevant structures into an executable, modular analysis workflow. The whole slide images were first down sampled to 10% from the original resolution of 0.56 microns/pixel, and classified into tissue and background. The next steps were performed only in the tissue regions. First the images were segmented on different coarseness levels; then lobules and tumor were detected. Inside the lobules, the percentage of positive nuclei (ER or PR) was determined for each lobule. The lobules were sorted into three categories according to their distance to tumor tissue and the distribution of positivity was evaluated.
Pre-processing
The whole slide images have a pixel size of 0.56 microns/pixel at original scale. As processing of the whole images in original resolution requires inacceptable computational power and processing times, and large scale tissue structures such as tumor mass and lobules appear more salient at lower resolutions, the images were down sampled to 10% of the original resolution, using nearest neighbor resampling, to a working pixel size of 5.6 microns/pixel.
In order to describe textural properties of the image, an additional image layer was calculated. The textural feature used for this is the standard deviation of each pixel to the neighbor pixels. This textural layer was derived from the blue channel. The textural layer was median filtered with a 3 × 3 window and then smoothed with an 11 × 11 Gaussian kernel.
For separation of tissue and background the image was smoothed with a 15 × 15 Gaussian kernel. A threshold was automatically computed on the smoothed image, and pixels higher than the threshold were classified as background. Small areas of background surrounded by tissue were reclassified as tissue and vice versa. All subsequent operations were only performed in the tissue regions.
Segmentation
In preparation for the classification, the images were segmented at five different levels of granularity, using the multiresolution segmentation described in [11] as implemented in the image analysis software. Multiresolution segmentation aims at creating spectrally homogeneous segments. The level of heterogeneity allowed is controlled by a scale parameter. The higher the value of the scale parameter, the more heterogeneous - and thus larger - the segments are allowed to be. Here, the segmentation started with a rather large scale parameter to produce the first and coarsest level, then the scale parameter was decreased for each subsequent step of the segmentation. The scale parameter was set to 100 for the first segmentation, and decreased in steps of 20 for the following segmentations. The resulting segmentation is hierarchical, such that a segment in a finer level can be a sub-segment of only one segment in a coarser level (Figure 1). The five different levels were created as preparation for the lobule classification, where the combined segment properties of different levels are used. As spectral information to compute the homogeneity criterion, the RGB image channels were used in combination with the computed texture channel. The texture channel was weighted twice as much as the color channels, thereby placing emphasis on the textural features that separate lobules from non-lobules.
Initial lobule classification
The classification of lobules was performed in two steps. In the first step, the initial lobule classification, lobule candidates were classified based on textural, geometric and relational features in the four coarsest segmentation levels independently. Rationales for feature selection were two-fold: (1) based on visual assessment of previously annotated lobular structures in different image channels, and (2) biological knowledge such as tubular structures, reflected by regular intensity variation in images.
As textural feature, the mean (per segment) of the standard deviation to neighbor pixels (in the following abbreviated as stddev-n) was used, which describes the local variance of intensity values. While stromal tissue usually shows a rather low stddev-n, normal lobules could be observed to have a higher stddev-n within a fairly constant range of 15-36. At the low resolution, where nuclear diameter corresponded roughly to one-two pixels, this probably reflected the lobules' regular internal structure consisting of rings of epithelial cells. This property caused the lobules to stand out in the stddev-n channel as relatively bright blobs against a darker stromal background (Figure 2).
As geometric features, the area and roundness of the segments were used, reflecting the fact that most sections of lobules were within a certain area size (range, 2000-14000 square pixels, i.e. 0.0627-0.439 mm2), and their shape appeared relatively round, or oval. Roundness was defined here as the difference between the radii of the largest enclosed ellipse and the smallest enclosing ellipse, where a smaller value corresponds to more roundness, 0 meaning a perfect ellipse (accepted range, 0-1.3).
The relational features describe the relations of a segment to its neighbor segments. Here spectral relations derived from the textural layer are examined. The difference of means in the stddev-n channel to neighbor segments and the border contrast to neighbor segments are used as relational features, reflecting the fact that the lobules are brighter than their surroundings in the stddev-n channel. Consequently, both the difference of means and the border contrast have to exceed a minimum (difference of means >= 5; border contrast >= 0.4).
All conditions for the features must be met for a segment to be classified as lobule candidate. After the initial classification, lobule candidates have been identified in the four coarsest segmentation levels.
Refined lobule classification
The initial classification, where segments were classified as lobules in each segmentation level independently, still contained many errors: segments falsely classified as lobules, missed lobules, lobule regions merged with surrounding regions, missing parts of lobules. Therefore, the initial lobule classification was refined based on the properties of sub-segments, starting with the coarsest segmentation level and working through the finer levels. Each lobule candidate was evaluated using the following rules: if the sub-segments were too different from each other, the lobule classification was removed. If all sub-segments fulfilled a subset of criteria for lobule candidates (without the conditions on minimum area and border contrast), the lobule was accepted as a whole. If a sub-segment did not fulfill the criteria, and at least half of its border corresponded to the border of the lobule (thus being more at the outside), it was removed from the lobule. The remaining segments were merged, and if the merged segment fulfilled the full set of criteria for lobules (as in the initial classification) it was accepted as a lobule. In this way, the lobule classifications were consolidated and carried through the levels to the fourth level. After classification refinement, this level contained the final lobule classification and was used in the subsequent steps. As a last step, adjacent segments which fulfilled the subset of lobule criteria were merged and reclassified as lobule if the merged segment fulfilled all lobule criteria. For evaluation of the detection quality, a pathologist manually labeled detected lobules as true positive or false positive detections, and missed lobules as false negatives, re-classifying incorrectly detected structures within the user interface of the software package.
Tumor classification
Choosing cases with hormone receptor negative tumor for initial proof of feasibility, tumor areas were classified based on adding segments to seed segments, assuming a tumor with coherent tumor mass. In order to find seed segments in the down sampled images, the median of the mean stddev-n values was determined from all tissue segments which were not classified as lobules. Segments whose mean stddev-n values were very close to the median (± 0.1) were selected as seed segments, as in our samples these had a high probability to be located inside the tumor. After detecting the seed segments, neighbor segments were added iteratively on two conditions: their mean stddev-n should be close to that of the already detected tumor region (difference of at most 2) and the shared border to the tumor region should be at least 20% of the total segment border. In a post-processing step, holes inside the tumor region were closed and isolated tumor regions smaller than 1/3 of the largest tumor area eliminated, based on the fact that these usually were false detections, especially occurring at the tissue borders where the tissue was folded.
Nuclei detection in lobules
In order to detect positive (brown DAB signal) and negative (blue hematoxylin signal) nuclei and calculate the percentage of positive nuclei inside the lobules, each lobule was processed separately on the original resolution of 0.56 µm/pixel. First, brown DAB positive and blue DAB negative stain channels were separated using an implementation of a color deconvolution algorithm which uses a hue-saturation-density model for stain recognition [12]. Nuclei were detected in both channels separately. For the nuclei detection, preliminary nuclear regions were determined pixel-based where the intensity value in the respective stain channel is higher than an empirically determined threshold (0.2) as well as higher than that of the other stain channel. A watershed algorithm within the nuclear regions, applied to the respective stain channels, was then used to separate adjacent nuclei. The separated regions formed by the watershed algorithm needed to have a minimum area to be accepted as nuclei. Holes in detected nuclear objects were filled. Since some nuclei can remain merged in the watershed segmentation due to insufficient spectral border differences, in the next step merged nuclei were separated based on geometry: they were cut at significant dents. Significant dents are dents where the angle between the two tangents starting at the corner pixel is more than 30 degrees. In post-processing, the borders of nuclei were smoothed, and detected nuclei whose area was smaller than the minimum nuclear area were removed.
Evaluation of nuclear positivity in lobules
The nuclear positivity for ER and PR in the lobular epithelium was evaluated in relation to their distance to the tumor. For this purpose, Lobules were divided into three groups based on their distance to the tumor: adjacent (<0.5 mm), intermediate (0.5-2 mm) and distant (>2 mm). Distances were measured from the center of the lobule to the nearest tumor border.
The specific functionalities of the Image Miner® software (Definiens) were used to display the results in table format, to visualize potential trends observed for ER/PR positivity per lobule in relation to the lobules' distance from the tumor in graphics, and for linking individual regions of interest (in this case, individually analyzed lobules) with corresponding data points in tables or graphical displays. The latter functionality was used to facilitate a fast review of individually analyzed regions of interest in the context of comprehensively visualized data.