Volume 8 Supplement 1

## Proceedings of the 11th European Congress on Telepathology and 5th International Congress on Virtual Microscopy

# Cell nuclei extraction from breast cancer histopathologyimages using colour, texture, scale and shape information

- Antoine Veillard
^{1, 2}Email author, - Maria S Kulikova
^{1}and - Daniel Racoceanu
^{1, 2}

**8(Suppl 1)**:S5

**DOI: **10.1186/1746-1596-8-S1-S5

© Veillard et al; licensee BioMed Central Ltd. 2013

**Published: **30 September 2013

## Background

Standard cancer diagnosis and prognosis procedures such as the Nottingham Grading System for breast cancer incorporate a criterion based on cell morphology known as cytonuclear atypia. Therefore, algorithms able to precisely extract the cell nuclei are a requirement in computer-aided diagnosis applications.

## Materials and methods

We propose a method based on the creation of a new image modality consisting in a grayscale map where the value of each pixel indicates its probability of belonging to a cell nuclei. This probability map is calculated from texture and scale information in addition to simple pixel color intensities. The resulting modality has a strong object-background contrast and evens out the irregularities within the nuclei or the background. The actual extraction is performed using an AC model with a nuclei shape prior included to deal with overlapping nuclei.

### Feature model

First, a color deconvolution [1] is applied in order to separate the immunohistochemical stains from which 3 grayscale images are produced: a haematoxilin image, an eosin image and a third residual component orthogonal in RGB space. Next, local features based on Laws’ texture measures [2] are computed for each pixel of the 3 obtained images. 5 different 1-dimensional convolution kernels (L5 = (*1*, *4*, *6*, *4*, *1*), *E*_{
5
} = (*–1*, *–2*, *0*, *2*, *1*), *W*_{
5
} = (*–1*, *2*, *0*, *–2*, *1*), *S*_{
5
} = (*–1*, *0*, *2*, *0*, *–1*) and *R*_{
5
} = (*1*, *–4*, *6*, *–4*, *1*)) are used to compute 25 different 5 × 5 kernels by convolving avertical 1-dimensional kernel with a horizontal one. The 5 × 5 kernels are applied at every pixel to extract 25 features which are then combined into 15 rotationally invariant features after normalizing by the output of the *L*_{
5
}^{
T
} × *L*_{
5
} kernel and smoothing with a Gaussian kernel of standard deviation *σ* = *1.5* pixels.

The same process is repeated at 4 different scales after locally re-sampling the image using Lanczos-3 sinc kernels. Re-sampled images are locally computed around each pixel to allow the computation of the 15 texture features for the same pixel at different scales. Local texture features are computed at 1:1, 1:2, 1:4 and 1:8 scales for every pixel.

### Probability map

*x*is used to compute the probability

*p*

_{ n }(

*x*) of each pixel to belong to a cell nuclei. Let

*μ*

_{ n }(resp.

*μ*

_{ b }) be the mean of the feature vectors for the pixels belonging to the nuclei (resp. to the background). A class dependent LDA is performed in order to find two directions in the feature space,

*w*

_{ n }and

*w*

_{ b }, such that the projection of the classes on these directions has a maximum inter-class scatter over within-class scatter ratio. The estimated class probability associated with the feature vector x is then calculated from the linear scores

*l*

_{ n }= (

*x – μ*

_{ n })

*· w*

_{ n }and

*l*

_{ b }= (

*x – μ*

_{ b })

*· w*

_{ b }using the softmax function:

The resulting probability map exhibits strong contrast between the objects and the background. Moreover, nuclei and background appear more homogeneous than in the original image. A post processing step is also applied to fill small holes still remaining in nuclei (larger holes are not removed to prevent the unintended deletion of interstices between different nuclei).

### AC model including shape prior

The actual extraction of cell nuclei is performed from the probability map with an AC model with shape prior information. The total energy *E*(*γ*) associated to a contour γ is a weighted sum of an image term *E*_{
i
}(*γ*) and a shape term *E*_{
s
}(*γ*). The latter is itself the weighted sum of a smoothing term *E*_{
sm
}(*γ*) and a shape prior term *E*_{
sp
}(*γ*).

The shape prior term
allows to control the perturbations *δr*(*t*) of a contour around a circle at different frequencies *k* of the Fourier components by adjusting the coefficients *f*_{
k
}. Detailed formulas and explanations for this and the other energy terms can be found in the work of Kulikova *et al.* [3]. The shape prior information allows to properly extract overlapping nuclei according to their expected shape without arbitrarily discarding the overlapping parts.

The detection of nuclei is performed by a marked point process model the details about which the interested reader can find in [4]. An empirical study in [5] shows that this particular combination of MPP and AC over-performs other state-of-the-art methods for nuclei detection and extraction.

## Results and discussion

The training set used for the LDA consists of 6 1024×1024 images where the nuclei have been manually delineated by a pathologist. Object and background parameters used in the AC model are also calculated from the training set. Weight parameters for the energy terms in the AC model are adjusted with a grid search. Images used for training are distinct from the images used for validation.

## Authors’ Affiliations

## References

- Ruifrok AC, Johnston DA: Quantification of histochemical staining by color deconvolution. Analytical and Quantitative Cytology and Histology. 2001, 23: 291-299.PubMed
- Laws K: Textured image segmentation. PhD thesis. 1980, University of Southern California
- Kulikova M, Jermyn I, Descombes X, Zhizhina E, Zerubia J: A marked point process model with strong prior shape information for extraction of multiple, arbitrarily-shaped objects. Proc. Signal-Image Technology and Internet-Based Systems. 2009
- Descombes X, Minlos R, Zhizhina E: Object extractionusing a stochastic birth-and-death dynamics in continuum. J. Math. Imaging Vis. 2009, 33 (3): 347-359. 10.1007/s10851-008-0117-y.View Article
- Kulikova MS, Veillard A, Roux L, Racoceanu D: Nuclei extraction from histopathological images using a marked point process approach. Proc. SPIE Medical Imaging, San Diego, California, USA. 2012
- Dalle J, Li H, Huang CH, Leow W, Racoceanu D, Putti TC: Nuclear pleomorphism scoring by selective cell nuclei detection. Proc. Workshop on Applications of Computer Vision. 2009

## Copyright

This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.