# Out-of-sample extension of diffusion maps in a computer aided diagnosis system. Application to breast cancer virtual slide images

- Belhomme Philippe
^{1}Email author, - Oger Myriam
^{2}, - Michels Jean-Jacques
^{2}and - Plancoulaine Benoît
^{1}

**8(Suppl 1)**:S9

https://doi.org/10.1186/1746-1596-8-S1-S9

© Philippe et al; licensee BioMed Central Ltd. 2013

**Published: **30 September 2013

## Keywords

## Introduction

While the pathologist population tends to dramatically drop, the number of pathological cases to be examined increases sharply, mainly due to early screening campaigns; developing automated systems would thus be useful to help pathologists in their daily work. As Virtual Microscopy (VM) is more and more introduced in pathology departments [1] where it holds immense potential despite the large amounts of data to be managed, its combination with image processing techniques can allow to find objective criteria for differential diagnosis or to quantify prognostic markers. Thus, many works try to develop computer-aided diagnosis systems (CADS) based on image retrieval and classification [2, 3]. The first step consists in building a knowledge database involving many features extracted from a set of well-known images; it is an 'off-line' procedure conducted once. These features are represented by vectors of non-linear data acting as a signature for the original images. In a second step, signatures are obtained from new unknown images to analyze and compared with the database; it is an 'on-line' procedure. Because of tumor heterogeneity, it is essential to build knowledge databases containing representative features of the multiple morphological types of lesions before considering to implement a CADS. But, as it is almost impossible for a pathologist to manually segment large virtual slide images (VSI), the usual practice consists in manually selecting some 'representative areas'. A bias is then introduced in the process as this choice is obviously subjective. It is then mandatory to find wiser solutions leading to an unbiased collection of these 'representative areas' (and later called 'patches'). In a previous work [4], we have proposed an original strategy: starting from a collection of breast cancer VSI, then taking advantage of stereological sampling methods and diffusion maps, a knowledge database is obtained from a reduced number of patches that are representative of given histological types. The sampling tools offered by stereology are well-suited in this context [5]. Systematic sampling starting from a random point with a fixed periodic interval is able to reduce the area to be analyzed, while preserving the collection of distinctive regions encountered in a tumor. However, even if the working area becomes smaller, the number of selected patches can be very large and may include many redundant elements. A data reduction has then to be conducted. Among the available methods, the diffusion maps technique [6, 7] has been retained since it provides a very attractive framework for processing and visualizing huge non-linear bulk data. Diffusion maps belongs to unsupervised learning algorithms dealing with a spectral analysis of non-linear data, providing a clustering only for given training points with no straightforward extension for out-of-sample cases. The work presented here focuses on a way to get around this problem and explains how unknown VSI can be classified by considering the diffusion maps as a learning eigenfunction of a data-dependent kernel. It makes use of the Nyström formula to estimate diffusion coordinates of new data [8]. An application on histological types of breast cancer is presented with VSI of Invasive Ductal Carcinoma and Mastosis.

## Materials

VSI come from histological sections of breast tumors stained in the same laboratory according to the Hematoxylin-Eosin-Safron protocol and acquired with the same digital scanner (a ScanScope CS from Aperio Technologies). The aim being to develop a generalized CADS, it is mandatory to manage color calibration of each device used along the process, from histological staining up to image acquisition [9]. For this study, we have collected image patches from two histological types: Invasive Ductal Carcinoma (IDC) and Mastosis (Ma) with patches from the 'normal' morphology for further be able to remove non-informative patches. VSI have been acquired at X20 (0.5 µm per pixel) and stored in TIFF 6.0 file format (compression 30%). The tools are developed in Python language with the help of specialized modules (PIL: Python Imaging Library, SciPy and mathplotlib).

## Methods

### Stereology

In order to reduce the expertise workload and to obtain a reliable ground truth, a stereological test grid for point counting is over-imposed onto VSI in the ImageScope viewer [10]. The grid step has been set to 1000 x 1000 pixels (3500 points in average per image). The pathologist has then to determine which histological class is associated with the local areas centered on grid points; 30 possibilities are proposed for breast tumors. A simple mark has to be drawn on a grid point in the overlay layer whose name corresponds to his choice. Each area is then extracted at the plain resolution and stored as an uncompressed TIFF image. These areas (also called 'patches') are squares of size 400 x 400 pixels. This size has been chosen according to the representative structures encountered in breast tumors and allows to expertise only 16% of a VSI.

### Features extraction

*RGB*color components (reduced to 64 values) and from the two first components (

*H*,

*E*) of the color deconvolution specific to Hematoxylin and Eosin staining [11]. For any given component

*X*, the computed features are:

*X*,

*X*reverse sorting, cumulative_

*X*, 20%-40%-60%-80% quantiles of cumulative_

*X*, mean_

*X*, median_

*X*, mode_

*X*, Skewness_

*X*, Kurtosis_

*X*, PearsonModeSkewness_

*X*, that is a total of 13 data. Three of them are themselves histograms with 64 values but will provide a single measure after computing the distance between two signatures.With the 5 components (

*R*,

*G*,

*B*,

*H*,

*E*) 65 measures will be taken into account for a patch but 1010 values will be stored in its signature. Considering the sparse numerical range of features, the symmetric Kullback-Leibler distance has been retained for its ability to easily manage such values, while remaining fast to implement. The distance between two vectors

*p*

_{ 1 },

*p*

_{ 2 }of length

*n*is then defined by:

### Data reduction

*X*={

*x*

_{ 1 },

*x*

_{ 2 },...,

*x*

_{ n }} be a set of

*n*patches that we estimate as a fully connected graph

*G*, that means a distance function is computed for each pair {

*x*

_{ i },

*x*

_{ j }}. A

*n*x

*n*kernel

*P*is obtained from a Gaussian function whose coefficients are given by:

In fact, *p*(*x*_{
i
},*x*_{
j
}) may be viewed as the transition kernel of the Markov chain on *G*. In other words, *p*(*x*_{
i
},*x*_{
j
}) defines the transition probability for going from *x*_{
i
} to *x*_{
j
} in one time step. The eigenvectors Π_{k} of *P*, ordered by decreasing positive eigenvalues, give the practical observation space axes. It must be noticed that Π_{0} is never used since linked to eigenvalue ⌊=1 (i.e. the data set mean or trivial solution). Projection is then done along (Π_{1},Π_{2},Π_{3}) for a 3D visualization. Choosing *∑* in *w*(*x*_{
i
},*x*_{
j
}) is an empirical task which should permit a moderate decrease of the exponential; some works use the median value of all distances *D*_{
KL
}(*x*_{
i
},*x*_{
j
}) where other use the mean distance obtained from the k nearest neighbors of a subset of *X* [6].

### Out-of-sample Nyström extension

*P*∊ ℝ

^{n×n}can be computed no faster than

*O*(

*n*

^{3}); this limits SCA techniques to moderately sized problems [12]. Fortunately, the Nyström extension, originally applied for finding numerical solutions of integral equations, can be used to compute eigenvectors and eigenvalues of a sub-matrix formed by

*m*columns of

*P*randomly subsampled and then extended to the remaining

*n-m*columns [8]. Given an

*n*x

*n*matrix

*P*and an integer

*m*<

*n*. Let call

*P*

^{(}

^{ m }

^{)}the matrix formed by

*m*columns of

*P*that is the graph Laplacian of a set

*Y*⊂

*X*with |

*Y*|=

*m*.

*Y*is then a training set. The orthonormal matrix of eigenvectors

*U*

^{(}

^{ m }

^{)}and their associated eigenvalues in a diagonal matrix

*Ë*

^{(}

^{ m }

^{)}are classically obtained from

*P*

^{(}

^{ m }

^{)}by solving:

*P*

^{(}

^{ m }

^{)}

*U*

^{(}

^{ m }

^{)}=

*Ë*

^{(}

^{ m }

^{)}

*U*

^{(}

^{ m }

^{)}. This step has to be run once and then may be considered as an 'off-line' procedure. The Nyström formula allows to obtain the approximate eigenvectors of all the set

*X*by:

where *λ*_{
i
}^{(}^{
m
}^{)} and *u*_{
i
}^{(}^{
m
}^{)} are the *i*^{th} diagonal entry and *i*^{th} column of *Ë*^{(}^{
m
}^{)} and *U*^{(}^{
m
}^{)} respectively. *P*_{
N
}_{,}_{
M
} is a *n* x*m* sub-matrix of the complete graph obtained from distances *w*(*x*_{
i
},*x*_{
j
}). Its computation is an 'on-line' procedure having to be conducted for each new test set (*X\Y*). For a 3D visualization, the second to fourth columns are used (the first one being the trivial solution).

## Results and discussion

^{2}. A total number of 1857 patches, classified as Mastosis (919 Ma), Invasive Ductal Carcinoma (812 IDC) and Normal (126 Nor) have been extracted from their inner stereological test grid. At first, table 1 shows that features extraction is

*O*(

*n*) while the spectral analysis is close to

*O*(

*n*

^{ 3 }); it has to be noticed that the latter involves both eigenvectors decomposition and code for managing the CADS. Figure 1 illustrates the projection of patches with their true eigenvectors (in black) and their estimated coordinates obtained from 1000 patches (in red). The visual comparison shows that computing a classical Euclidean distance between two points should be equivalent in both cases. Figure 2 shows the same approach from only 500 patches. Besides a shift between clouds of points, a rescaling is visible but the main shape is still preserved. To confirm this assertion we have analyzed for each patch the histological type of their nearest neighbor. This has been done both with the true eigenvectors and the estimated coordinates. In our application four cases are considered: a 'Ma' patch may be associated with another 'Ma' or 'IDC' whereas a 'IDC' patch may be associated with 'IDC' or 'Ma'. When a patch is close to the 'normal' type, we consider it as non-informative. Table 2 shows that the Nyström extension allows to obtain very similar results than the true eigenvectors (row 'reference').

Computation time on a PC (dual core)

Patch number | Features extraction (in seconds) | Spectral analysis (in seconds) |
---|---|---|

250 | 46 | 17 |

500 | 98 | 69 |

1000 | 180 | 308 |

2000 | 407 | 1429 |

Histological type in the nearest neighborhood

Number of test points | Ma | IDC | ||
---|---|---|---|---|

Ma | IDC | Ma | IDC | |

500 | 70.7% | 18.0% | 15.6% | 78.8% |

1000 | 72.2% | 17.6% | 15.3% | 80.4% |

reference | 73.9% | 17.1% | 14.8% | 82.1% |

## Conclusion

This work is the second part of a CADS we aim to develop based on an original strategy starting from VS and leading to an unbiased knowledge database containing reference patches of breast tumors. The first part has been presented in [4]. We have shown that combining stereological sampling and data reduction based on diffusion maps offers an interesting general framework. The results illustrated here are a proof of concept of the second part that is to classify new unknown patches. About 400 high resolution VS are now available in our lab; the benign and malignant breast tumors are classified into 30 histological types and subtypes. We plan to project some reference patches extracted from these 30 classes in the same 3D space, in order to build clusters, and then to classify a new unknown VS previously split in patches. But the spectral decomposition is very CPU intensive and managing for example 30 000 patches at a time (1 000 per histological type) would rapidly become impossible to compute. The Nyström extension seems to provide a good approximation of eigenvectors which then allow to reduce this computational burden.

## Authors’ Affiliations

## References

- Weinstein RS, Graham AR, Richter LC, Barker GP, Krupinski EA, Lopez AM, Erps KA, Bhattacharyya AK, Yagi Y, Gilbertson JR: Overview of telepathology, virtual microscopy, and whole slide imaging: prospects for the future. Hum Pathol. 2009, 40 (8): 1057-69. 10.1016/j.humpath.2009.04.006.View ArticlePubMedGoogle Scholar
- Kayser K, Gortler J, Bogovac M, Bogovac A, Goldmann T, Vollmer E, Kayser G: AI (artificial intelligence) in histopathology-from image analysis to automated diagnosis. Folia Histochem Cytobiol. 2009, 47 (3): 355-61.PubMedGoogle Scholar
- Oger M, Belhomme P, Klossa J, Michels JJ, Elmoataz A: Automated region of interest retrieval and classification using spectral analysis. Diagnostic Pathology. 2008, 3 (Suppl 1): S17-10.1186/1746-1596-3-S1-S17.PubMed CentralView ArticlePubMedGoogle Scholar
- Belhomme P, Oger M, Michels JJ, Plancoulaine B, Herlin P: Towards a computer aided diagnosis system dedicated to virtual microscopy based on stereology sampling and diffusion maps. Diagnostic Pathology. 2011, 6 (Suppl 1): S3-10.1186/1746-1596-6-S1-S3.PubMed CentralView ArticlePubMedGoogle Scholar
- Baddeley A, Jensen EB: Stereology for Statisticians. 2005, Chapman & Hall/CRCGoogle Scholar
- Belkin M, Niyogi P: Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation. 2003, 15: 1373-1396. 10.1162/089976603321780317.View ArticleGoogle Scholar
- Coifman RR, Lafon S, Lee AB, Maggioni M, Nadler B, Warner F, Zucker S: Geometric diffusions as a tool for harmonics analysis and structure definition of data: Diffusion maps. Proceedings of the National Academy of Sciences. 2005, 102 (21): 7426-7431. 10.1073/pnas.0500334102.View ArticleGoogle Scholar
- Williams C, Seeger M: Using the Nyström method to speed up kernel machines. Advances in Neural Information Processing Systems. 2001, 13: 682-688.Google Scholar
- Kayser K, Borkenfeld S, Gortler J, Kayser G: Image standardization in tissue–based diagnosis. Diagnostic Pathology. 2010, 5 (Suppl 1): S13-10.1186/1746-1596-5-S1-S13.View ArticleGoogle Scholar
- Herlin P: Computer-Assisted Stereology for Pathology Applications. Science Webinar series. 2009, [http://www.aperio.com]Google Scholar
- Ruifrok AC, Johnston DA: Quantification of histochemical staining by color deconvolution. Anal Quant Cytol Histol. 2001, 23: 291-299.PubMedGoogle Scholar
- Homrighausen D, McDonald DJ: Spectral approxima-tions in machine learning. 2011, ArXiv:1107.4340Google Scholar

## Copyright

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.