Model for detecting metastatic deposits in lymph nodes of colorectal carcinoma on digital/ non-WSI images

Abstract

Introduction

Colorectal cancer (CRC) constitutes around 10% of global cancer diagnoses and death due to cancer. Treatment involves the surgical resection of the tumor and regional lymph nodes. Assessment of multiple lymph node demands meticulous examination by skilled pathologists, which can be arduous, prompting consideration for an artificial intelligence (AI)-supported workflow due to the growing number of slides to be examined, demanding heightened precision and the global shortage of pathologists.

Method

This was a retrospective cross-sectional study including digital images of glass slides containing sections of positive and negative lymph nodes obtained from radical resection of primary CRC. Lymph nodes from 165 previously diagnosed cases were selected from Agha Khan University Hospital, from Jan 2021 to Jan 2022. The images were prepared at 10X and uploaded into an open source software, Q path and deep learning model Ensemble was applied for the identification of tumor deposits in lymph node.

Results

Out of the 87 positive lymph nodes detected by AI, 73(84%) were true positive and 14(16%) were false positive. The total number of negative lymph nodes detected by AI was 78. Out of these, 69(88.5%) were true negative and 9 (11.5%) were false negative. The sensitivity was 89% and specificity 83.1%. The odds ratio was 40 with a confidence interval of 16.26–98.3. P-value was < 0.05 (< 0.0001).

Conclusion

Though it was a small study but its results were really appreciating and we encourage more such studies with big sample data in future.

Introduction

Colorectal cancer (CRC) constitutes around 10% of global cancer diagnoses and death due to cancer. In men, it is the third most prevalent cancer and the second most common cancer in women. It is estimated that by 2035, new CRC cases will reach around 2.5 million worldwide, mainly due to changes in lifestyle and dietary habits [1, 2]. The most effective CRC treatment is the surgical resection of the tumor and the regional lymph nodes. The Tumor-Node-Metastasis (TNM) staging system, as defined by the American Joint Committee on Cancer, plays a vital role in predicting prognosis and shaping treatment decisions for CRC patients. N-staging requires the evaluation of metastases in the regional lymph nodes removed during CRC resection [3]. This is crucial for prognosis as the patients having early-stage colon cancer require additional surgery while adjuvant chemotherapy is given in the advanced stage of the disease [4]. Assessment of multiple lymph node demands meticulous examination by skilled pathologists. A minimum of 12 lymph nodes removed during CRC resection should be examined as recommended by the American Joint Committee on Cancer (AJCC). However, the lymph nodes actually removed can exceed up to four times this number [3, 5].

Routine pathology labs face an immense diagnostic workload due to the increased occurrence of CRC [5]. In addition, the global shortage of pathologists with the time consuming process of diagnosis, results in diagnostic delays therefore impacting the optimal healthcare of patients [6, 7]. In most cases, examining lymph node metastasis histologically in CRC patients can be arduous, prompting consideration for an artificial intelligence (AI)-supported workflow due to the growing number of slides to be examined, demanding heightened precision and effort [8]. Lately, artificial intelligence has made a significant progress in the medical field [9]. Computer-assisted image analysis in histopathology assessments has demonstrated its ability to efficiently extract quantitative features with accuracy and consistency thereby aiding the decision-making processes, ensuring diagnostic uniformity with the aim to alleviate pathologists’ workload and expedite diagnostics [10, 11].

Deep Learning (DL) algorithms not only have the capacity to aid in diagnoses, but can also forecast clinically significant molecular traits, recognize the prognostically linked histological characteristics and their correlation with metastasis and evaluate distinct elements within the tumor microenvironment [5, 8, 12]. Unfortunately, these advancements are not equally shared particularly in the developing part of the world which contributes more than two third of the world population and contains the bulk of world diseases [13].

Some studies have explored the use of deep learning models on simple digital images rather than whole slide images [11, 14]. These digital images, being smaller in size, can be easily transmitted without the need for high-tech computers or cloud services typically required for whole slide images. The outcomes from these studies were promising, indicating that computational pathology using local data can be utilized in resource-limited settings, bypassing the necessity for scanners. Studies of this nature have inherent limitations, being time-consuming and tedious, often unable to encompass the entirety of the picture at once [11]. However, these studies serve as valuable proof-of-concept endeavors, especially in developing countries, paving the way for potential full-scale implementation of digital pathology in the future.

In this study, an existing deep learning model was applied to digital images to identify metastatic deposits in lymph nodes of previously diagnosed cases of colorectal carcinoma. The aim was to compare the results obtained by the deep learning model with those determined by pathologists, assessing the agreement between the two methods. We also wanted to check the performance of deep learning model which was test and trained on whole slide images and on a different set of population to our population and on simple digital images. This kind of deep learning model can help the pathologist in future using real time data on whole slide images especially in developing world which is currently facing the shortage of pathologists against ever increasing load of tumor cases with every passing year [7].

Methodology

This was a retrospective cross-sectional study which included digital images of glass slides containing sections of lymph nodes obtained from radical resection of primary CRC. 165 images of previously diagnosed cases of colorectal cancer were selected from Agha Khan University Hospital, from Jan 2021 to Jan 2022 after the approval of ethics review committee No. 2024-9618-28116 of Agha Khan University. Slides with over fixation and poor staining were excluded. The images were prepared at 10X through a camera connected to Nikon microscope. Both positive (Fig. 1) and negative lymph nodes regions (Fig. 2) were photographed. Both positive and negative lymph node images were included. A consultant pathologist took the images from region of interest based on the area where morphology and quality of image was most clear. Both positive and negative tumor regions in a lymph node were photographed. (Figures 1 and 2) The images were then uploaded into an open source software, Q path, by a computational pathologist. Deep learning model Ensemble was applied for the identification of tumor deposits in lymph node images (Fig. 3).Images were classified as positive or negative by the software. The results were assessed for concordance by an independent pathologist who compared the results generated by deep learning software with the manual findings. Diagnostic Test for Accuracy was applied to find out the sensitivity and specificity.

Fig. 1

Positive lymph node involved by metastatic adenocarcinoma

Fig. 2

Lymph node not involved by adenocarcinoma (negative lymph node)

Fig. 3

AI generated results on Q Path highlighted by deep learning model Ensemble

Results

In our study, out of 165, AI was able to detect 87 (52.7%) positive lymph nodes (Table 1). Out of 87, 73 (84%) were true positive and 14 (16%) were false positive. The pathologist detected 82 (49.7%) positive lymph nodes. The total number of negative lymph nodes detected by AI was 78 (47.3%). Out of these, 69 (88.5%) were true negative and 9 (11.5%) were false negative. The pathologist detected 83 (50.3%) negative lymph nodes. The sensitivity was 89% and specificity 83.1% (Table 2). The odds ratio was 40 with a confidence interval of 16.26–98.3 (Table 3) in this study. The p-value was highly significant and it was < 0.05 (< 0.0001). The area under receiver operating curve was 0.86 with 95% confidence interval of 0.81–0.91.

Table 1 Pathologist vs. Artificial Intelligence (AI)

Table 2 Sensitivity, specificity and area under receiver operating curve (ROC)

Table 3 Odds ratio between pathologist and artificial intelligence

Discussion

A meticulous microscopic examination of tissue biopsies is the gold standard for diagnosis but it is a time-consuming procedure. Deep Learning (DL) algorithms are revolutionizing histopathology by facilitating quantitative analysis of histological features. By comprehensive evaluation of the tumor microenvironment, they hold the potential to enhance patient stratification for targeted therapies [15]. Computer-assisted image analysis in histopathology has been shown to provide efficient, accurate, and consistent quantitative feature extraction. This supports decision-making and ensures diagnostic consistency [16].

Despite the strides made by pathology laboratories in optimizing their analytical processes and automating procedures to minimize errors and to ensure high sensitivity and specificity, diagnostic inconsistencies still persist. This is due to different laboratory protocols and inter-observer variability among pathologists [17]. Numerous studies have indicated that by using deep learning models as supplementary diagnoses, the diagnostic performances either statistically significantly improved or accelerated the diagnostic process [18,19,20]. Deep learning algorithms can offer crucial diagnostic insights in cases of discrepancies, enhancing clinical practice diagnoses. Pathologists can collaboratively review and analyze scanned histological images from various locations concurrently. Deep learning can also provide dependable predictions for colorectal histopathological images [17, 20].

In our study, the sensitivity was 89% and specificity was 83.1%. Another study has reported a sensitivity of 95.8% in the detection of lymph node metastases in breast cancer using an AI algorithm for the assistance of pathologist [21]. In our study, the pathologist serving as the gold standard for detecting metastatic lymph nodes in colon cancer, identified 82 (49.7%) positive lymph nodes and 83 (50.3%) negative lymph nodes out of 165 nodes examined. The AI system, in comparison, detected 87 (52.7%) positive lymph nodes and 78 (47.3%) negative lymph nodes. These results indicate that while the pathologist remains the benchmark for diagnostic accuracy, AI can complement their expertise by acting as an effective supplementary tool, potentially reducing the risk of false negatives and ensuring that fewer metastatic lymph nodes are overlooked. The negative lymph nodes detected by the pathologist and AI indicate a comparable specificity. Consequently, incorporating AI into the diagnostic process can enhance the overall detection accuracy, providing a valuable second opinion that supports pathologists in delivering more precise and comprehensive evaluations of lymph node status in colon cancer patients.

In our study, the sensitivity was 89% and specificity was 83.1%. A study by Amjad et al. comprising 4225 slides including 3 internal and 1 external validation cohorts showed high sensitivity and specificity when compared with the pathologist [5]. The area under receiver operating curve (ROC) was 0.86 with 95% confidence interval of 0.81–0.91 in this study. An AUC of 0.86 suggests that the AI has a high level of accuracy in differentiating between metastatic (positive) and non-metastatic (negative) lymph nodes. The 95% confidence interval of 0.81 to 0.91 provides a range within which the true AUC is expected to fall 95% of the time, indicating the precision of the AUC estimate. The narrow range reflects a high degree of confidence in the AI’s performance. These findings underscore the AI’s potential as a reliable tool in supporting pathologists, offering a high probability of correctly identifying the presence or absence of metastatic lymph nodes in colon cancer patients. Similarly, in a study by Wu S et al., the lymph node metastases diagnostic model maintained an AUC of 0·943 (95% CI 0·918–0·969) in breast cancer images and 0·922 (0·884–0·960) in prostate cancer images [22]. Our finding is in accordance with another study by Tan L et al. in which the area under the ROC curve was 0.97 with 95% confidence interval. They used multi-instance learning and had an accuracy of 95.3% while predicting lymph node metastasis in colorectal cancer [23].

In our study, the odds ratio was 40 with a confidence interval of 16.26–98.30 and p value was highly significant (< 0.0001) suggesting that the findings are unlikely due to chance or random variation. In 2018, a study investigated the impact of deep learning assistance provided to the pathologist in the review of metastatic lymph nodes in breast cancer [24]. It revealed a notable improvement in accuracy when compared to either the algorithm or the pathologist working independently. Utilizing the algorithm significantly increased the sensitivity (91% versus 83%) in detecting micro metastases in breast cancer. This highlights the valuable support that deep learning algorithms can provide in medical diagnostics and decision-making processes.

Conclusion

Though it was a small study but its results were really appreciating and encouraging. We used an algorithm which was made on whole slide images, tested and trained also on different population. But we used it on our population and also on digital images rather than whole slide images. This study was limited by the absence of whole slide scanner. More studies with bigger sample size are suggested.