Skip to main content

Diagnostic accuracy of oral cancer cytology in a pilot study

Abstract

Background

Recently, cytology has been applied to the diagnosis of oral lesions. We aimed to explore the diagnostic accuracy of oral cytology based on the histological diagnosis.

Methods

Histological diagnoses of 327 cases were classified as Negative, Borderline lesion –, Borderline lesion +, oral intraepithelial neoplasia/carcinoma in situ (OIN/CIS), or Positive. Cytological diagnoses were classified as NILM (negative for intraepithelial lesion or malignancy), LSIL (low-grade squamous intraepithelial lesion), HSIL (high-grade squamous intraepithelial lesion), or SCC (squamous cell carcinoma). The cytology slides were evaluated by 10 raters and the results were compared with the histology results.

Results

In 142 cases that were histologically negative, the number of NILM, LSIL, HSIL, and SCC and other malignancy was 77 (54.2%), 47 (34.3%), 8 (5.6%), and 10 (7.0%), respectively. Among 32 cases of Borderline lesion –, the number of NILM, LSIL, HSIL, and SCC and other malignancy was 11 (34.3%), 11 (34.3%), 9 (28.1%), and 1 (3.1%), respectively. Also, in 4 cases of Borderline lesion +, the number of NILM, LSIL, HSIL, and SCC and other malignancy was 2 (50.0%), 0 (0.0%), 0 (0.0%), 2 (50.0%), respectively. Among 12 cases of OIN/CIS, the number of NILM, LSIL, HSIL, and SCC and other malignancy was 1 (8.3%), 2 (16.7%), 4 (33.3%), and 5 cases (41.7%), respectively. Among 137 cases with a histological diagnosis of Positive, the number of NILM, LSIL, HSIL, and SCC and other malignancy was 7 (5.1%), 22 (16.1%), 19 (13.9%), and 89 (65.0%), respectively. Sensitivity, specificity, and positive predictive and negative predictive values were 93.5, 50.6, 62.4, and 89.8%, respectively, when the cytological diagnosis of Negative was assumed to be NILM; they were 77.8, 83.9, 81.0 and 81.1%, respectively, if the cytological diagnosis of Negative was assumed to be NILM and LSIL. The number of false-positive and false-negative diagnosis affected cases with LSIL and HSIL may indicate the difficulty in the cytological diagnosis of borderline lesions. While the negative predictive value was relatively high (89.8%) when cytological Negative was assumed to be NILM only.

Conclusion

Histopathological examination should be recommended in cases with cytological diagnoses of LSIL, HSIL, and SCC.

Background

Exfoliative cytology is a reliable tool for assessing malignant change in various organs [1]. Cytology has been also applied to the diagnosis of oral lesions [24]. Accurate cytological diagnosis of oral lesions, especially in distinguishing benign lesions from malignant ones, is essential for treatment as well as for clinical and epidemiological research including the study of prognosis [5].

With respect to cytological diagnosis, the classification of cervicovaginal smears into five classes was initially proposed by Papanicolaou, who formulated a series of guidelines for smear interpretation [6]. This system was generally well received, although the significance of the classes was often modified to meet the requirements of laboratories in consultation with clinicians [7]. In December 1988, a committee of experts who convened under the auspices of the National Cancer Institute (USA) in Bethesda, Maryland, proposed a diagnostic system for the interpretation of cervicovaginal smears. The resulting Bethesda System (modified in 2001) was officially accepted by the federal authorities in the United States [8]. Recently, this system has been used in various fields such as thyroid, renal, and female genital cytology [9].

Fuller et al. pointed out that although oral cytology seems to have a higher diagnostic value, further study is needed to assess this [10]. However, the accuracy of oral cytology is unknown. Similar to other fields, diagnosis in oral cytology has been based on the Papanicolaou classification, not by Bethesda System [8, 9]. However, there had been no established diagnostic guideline in Japan. The Japanese Society of Clinical Cytology (JSCC) then organized a working group for oral cytology to confirm the accuracy of oral cytology according to Bethesda System [8, 9] in 2013, and established a diagnostic guideline committee for oral cytology in 2013.

This study investigated the clinical applicability of oral cytology following Bethesda System [8, 9] before the publication of diagnostic guideline by JSCC [11] by exploring the diagnostic accuracy of oral cytology based on histological diagnosis as the gold standard.

Methods

Samples

Patients diagnosed with oral disease were examined in this study. All cytology samples were prepared from benign or malignant oral lesions for biopsy and/or surgically resection. The samples were prepared by smearing collected cells on glass slides, which were then immersed in 95% ethanol, fixed, and stained with Papanicolaou staining. The samples derived from the patient population had results of oral cytology. Data from five Japanese institutions were included (Department of Oral and Maxillofacial Surgery, Shimane University Faculty of Medicine; Department of Oral Pathology, School/Graduate School of Dentistry Osaka University; Second Department of Oral and Maxillofacial Surgery, Osaka Dental University; Department of Health Promotion, Division of Oral Pathology, Kyushu Dental University and Department of Oral Pathology, Division of Oral Pathogenesis and Disease Control, Asahi University School of Dentistry). The samples (including data) were retrospectively collected in clinical practice between October 2007 and November 2013, and cytological and histological tests were performed simultaneously.

This study was planned and performed following STARD checklist for reporting of studies of diagnostic accuracy (http://www.stard-statement.org) (data not shown).

Procedure of cytological diagnosis

All the raters who passed the board examination for cytology of JSCC reviewed the samples. The slides were evaluated independently by at least two raters, and a representative cytology result of each case was determined by a majority vote.

Cytological diagnoses were performed based on Bethesda System [8, 9], and were classified into NILM (negative for intraepithelial lesion or malignancy), LSIL (low-grade squamous intraepithelial lesion), HSIL (high-grade squamous intraepithelial lesion), SCC (squamous cell carcinoma), and IFN (indefinite for neoplasia or non-neoplasia) (Table 1).

Procedure of histological diagnosis

A histological diagnosis was provided by oral pathologists at each institution, and then the number of biopsy samples was determined at the investigator’s discretion. These histological slides underwent hematoxylin and eosin staining. Their histological findings were classified into five categories as follows. Negative was defined as non-malignant lesions including inflammatory ones; Borderline lesion – was defined as mild, mild–moderate, or moderate dysplasia; Borderline lesion + was defined as severe dysplasia; OIN/CIS was defined as oral intraepithelial neoplasia or carcinoma in situ; and “Positive” was defined as squamous cell carcinoma and other malignancies. Borderline lesion – and Borderline lesion + were prepared based on general rules for clinical and pathological studies on oral cancer [11, 12].

Accordingly, the results of representative cytology were compared with the histology results.

Statistical analysis

To compare between cytological and histological diagnosis, the histological diagnoses were classified into Negative (Negative and Borderline lesion –) and Positive (Borderline lesion +, OIN/CIS and Positive), and cytological diagnosis were also classified into Negative (NILM) and Positive (LSIL, HSIL, SCC, Other malignancy) or Negative (NILM and LSIL) and Positive (HSIL, SCC, Other malignancy). To examine the diagnostic performance by comparing their cytological diagnosis against the histological diagnosis, the sensitivity, specificity, positive predictive value, and negative predictive value were calculated, and all statistical analysis was performed using SAS version 9.3 (Cary, NC).

Results

Samples

A total of 423 samples of oral cytology with accompanying histological slides from five institutions were screened. Among the 423 samples, 96 samples (22.7%) were excluded from the study because of poor quality (e.g., scant cellularity or air-drying artifact). The remaining 327 samples were reviewed. The remaining 327 samples were reviewed by 10 raters. Among these samples, 93, 47, 82, 80, 18, 1, and 6 samples were reviewed by 2, 3, 4, 5, 6, 7, 8 raters, respectively (Table 2).

Table 1 Diagnostic guideline for oral cytology, proposed by the Diagnostic Guideline Committee for Oral Cytology of the Japanese Society of Clinical Cytology
Table 2 The numbers of reviewing samples per number of raters

Histological diagnosis

The histological diagnoses of 327 cases were classified as Negative, Borderline lesion –, Borderline lesion +, OIN/CIS, or Positive (Table 3).

Table 3 Histopathological categories of the reviewed samples

Result of cytological diagnosis compared with histological diagnosis

Table 4 shows the results for cytological diagnoses compared with histological diagnoses. In 142 cases that were histologically negative, the number of patients with NILM, LSIL, HSIL, and SCC and other malignancy was 77 (54.2%), 47 (34.3%), 8 (5.6%), and 10 (7.0%), respectively. Among the 32 cases of Borderline lesion –, the number of patients with NILM, LSIL, HSIL, and SCC and other malignancy was 11 (34.3%), 11 (34.3%), 9 (28.1%), and 1 (3.1%), respectively. Also, in the 4 cases of Borderline lesion +, the number of patients with NILM, LSIL, HSIL, and SCC and other malignancy was 2 (50.0%), 0 (0.0%), 0 (0.0%), 2 (50.0%), respectively. Among the 12 cases of OIN/CIS, the number of patients with NILM, LSIL, HSIL, and SCC and other malignancy was 1 (8.3%), 2 (16.7%), 4 (33.3%), and 5 cases (41.7%), respectively. Among 137 cases with a histological diagnosis of Positive, the number of patients with NILM, LSIL, HSIL, and SCC and other malignancy was 7 (5.1%), 22 (16.1%), 19 (13.9%), and 89 (65.0%), respectively.

Table 4 Results of cytological diagnoses compared with histopathological diagnoses

Distribution of histological diagnoses from the viewpoint of cytological diagnosis

Fig. 1 shows the distribution of histological diagnoses from the viewpoint of cytological diagnosis. Among cases with a cytological diagnosis of NILM (98 cases), 78.6, 11.2, 2.0, 1.0, and 7.1% had a histological diagnosis of Negative, Borderline lesion –, Borderline lesion +, OIN/CIS, and Positive, respectively. Among cases with a cytological diagnosis of LSIL (82 cases), 57.3, 13.4, 0.0, 2.4, and 26.8% had a histological diagnosis of Negative, Borderline lesion –, Borderline lesion +, OIN/CIS, and Positive, respectively.

Fig. 1
figure 1

Distribution of the 327 cases with a histological diagnosis in each cytological diagnostic category. The percentages are shown above the columns

On the other hand, among cases with a cytological diagnosis of HSIL (40 cases) 20.0, 22.5, 0.0, 10.0, and 47.5% had a histological diagnosis of Negative, Borderline lesion –, Borderline lesion +, OIN/CIS, and Positive, respectively. Among cases with cytological diagnoses of SCC or Other malignancy (107 cases), 9.3 , 0.9, 1.9, 4.7, and 83.2% had a histological diagnosis of Negative, Borderline lesion –, Borderline lesion +, OIN/CIS, and Positive, respectively.

Diagnostic performance of cytological diagnoses

With respect to cytological diagnostic performance, the sensitivity, specificity, positive predictive value, were 93.5, 50.6, 62.4, and 89.8% when cytological Negative was assumed to be NILM only, respectively (Table 5). Also, these measures were 77.8, 83.9, 81.0, and 81.1% when cytological Negative was assumed to be NILM and LSIL (Table 5).

Table 5 Diagnostic performance of cytological diagnoses

Discussion

The cytological collection had various methods and instruments. In this study, all samples were conventionally smeared and Papanicolaou stained, although recently liquid-based cytology has become common in the field of gynecology [13, 14]. The conventional method is still common in the field of oral and maxillofacial surgery, despite the technical error of delay in fixation of the smear sample leads to an air-drying artifact [4]. Actually, in this study, 96 slides (22.7%) were deemed inadequate for review due to scant cellularity or air-drying artifact, all of which were prepared in one institution. Also, these inadequate samples were partially due to the fact that cytology instruments do not obtain a sample from the lesion [4], because the cytological instruments varied from cotton swabs to intratooth brush in period of sample collection. To improve the quality of the cytology slides, the methods and instruments of cytological collection should be reconsidered and standardized.

As most of oral neoplasms originated in squamous cell epithelium, the squamous epithelium should be deeply reviewed for accurate histological diagnosis of oral lesions [15]. In the field of oral histopahology, there has been controversy for classification or severity of oral epithelial dysplasia (OED) into categories of Borderline lesions + or – [12]. According to a literature [16], mild epithelial dysplasia refers to alteration in limited to the basal and parabasal layers, moderate epithelial dysplasia refers to alteration in the basal to mid-portion of the spsinous layer, and severe epithelial dysplasia refers to alteration that affect more than half of the thickness of the epithelium.

Because most dyslpastic changes occur in the basal and parabasal layers [17], to obtain a sample from the deepest layer of lesion is important to make an accurate cytological diagnosis of oral lesions derived from squamous cells. Further, because most cases of oral SCC are the differentiated type and are different from cervical SCC, most of the superficial cells of these lesions are not malignant and could be a reason of a false-negative diagnosis. An another reason for these “misdiagnoses” can be that the nuclear morphology of oral epithelial cells is easily affected by malignant transformation as well as inflammation and oral bacilli, so the shape of the nuclei would seem to be atypical and dysplastic, which would make accurate diagnosis in oral cytology difficult [14].

The uncertain cytological diagnosis of borderline lesions may affect our results as shown in Figure 1. However, in previous study [4, 1820], detail of diagnoses in borderline lesions was not available. In future, the detailed information for borderline lesions should be investigated regarding the oral cytological diagnoses.

As borderline lesions diagnosed as LSIL and HSIL are then indicated for surgical resection in our hospital [5, 21], we strongly recommend that cases with cytological diagnoses of LSIL, HSIL SCC and Other malignancy should belong to the category of cytological positive. In our study, the number of false-positive and false-negative diagnosis affected cases with LSIL and HSIL may indicate the difficulty in the cytological diagnosis of borderline lesions (Table 4, Fig. 1). While the negative predictive value was relatively high (89.8%, Table 5) when cytological Negative was assumed to be NILM only.

As with most white oral lesions, the color is derived from the thickened keratin layer or thickened spinous layer, which masks the vascularity (redness) of the underlying connective tissue [16]. Accurate diagnosis of such white lesions is clinically difficult [16, 17], and a precise diagnosis of dysplasia in intraepithelial lesions is difficult even in histopathologic specimens [16]. Recently, Sekine et al [21] reported that nucleus accumbens-associated protein 1 (NAC1) has the potential to be used as a biomarker for distinguishing OED from CIS/OSCC. Standardization of the diagnosis of borderline lesions such as epithelial dysplasia is needed from a cytopathological viewpoint. There are very few studies on the accuracy of oral cytology [22], so it still remains unknown whether our results are satisfactory.

In this study, the cytology slides were evaluated by 10 raters and the results were compared with the histology results. Though each rater evaluated certain samples only one time in this study, one limitation was that the intra-examiner reliability was not able to be evaluated. In future, however, we will design and perform the further study to evaluate intra-examiner reliability on cytological diagnosis for oral cancer. Furthermore, revision of diagnostic guideline by JSCC [11] should be needed, as detailed classification, such as atypical squamous cells of undetermined significance (ASC-US) and atypical squamous cells cannot exclude HSIL (ASC-H) in the field of gynecology, should be considered to achieve more accurate diagnosis for borderline lesions in oral cytology.

Conclusion

In conclusion, histopathological examination should be recommended in cases with cytological diagnoses of LSIL, HSIL, and SCCC.

Abbreviations

ASC-H:

Atypical squamous cells cannot exclude hsil

ASC-US:

Atypical squamous cells of undetermined significance

CIN:

Cervical intraepithelial neoplasia

CIS:

Carcinoma in situ

HSIL:

High-grade squamous intraepithelial lesion

IFN:

Indefinite for neoplasia or non-neoplasia

JSCC:

The Japanese society of clinical cytology

LSIL:

Low-grade squamous intraepithelial lesion

NAC1:

Nucleus accumbens-associated protein 1

NILM:

Negative for intraepithelial lesion or malignancy

OED:

Oral epithelial dysplasia

OIN:

Oral intraepithelial neoplasia

SCC:

Squamous cell carcinoma

References

  1. Coss LG, Melamed MR. Diagnostic cytology. Its origins and principles. In: Koss LG, Melamed MR, editors. Koss’s diagnostic cytology and its histopathologic bases. 5th ed. Philadelphia: Lippincott Williams & Wilkins; 2006. p. 3–20.

  2. Sousa MC, Alves MGO, Souza LA, Brandão AAH, Almeida J, Cabral LAG. Correlation of clinical, cytological and histological findings in oral squamous cell carcinomas. Oncol Lett. 2014;8:799–802.

    PubMed  PubMed Central  Google Scholar 

  3. Segura IG, Secchi D, Carrica A, Barello R, Arbelo D, Burgos A, Brunotto M, Zarate AM. Exfoliative cytology as a tool for monitoring pre-malignant and malignant lesions based on combined stained and morphometry techniques. J Oral Pathol Med. 2015;44:178–84.

    Article  Google Scholar 

  4. Gupta S, Shah JS, Parikh S, Limbdiwala P, Goel S. Clinical correlative study on early detection of oral cancer and precancerous lesions by modified oral brush biopsy and cytology followed by histopathology. J Cancer Res Ther. 2014;10:232–8.

    Article  PubMed  Google Scholar 

  5. Karino M, Nakatani E, Hideshima K, Nariai Y, Tsunematsu K, Ohira K, Kanno T, Asahina I, Kagimura T, Sekine J. Applicability of preoperative nuclear morphometry to evaluating risk for cervical lymph node metastasis in oral squamous cell carcinoma. PLoS One. 2014;9:e116452.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Papanicolaou GN, Traut HF. The diagnostic value of vaginal smears in carcinoma of the uterus. Am J Obstet Gynecol. 1941;42:193–206.

    Article  Google Scholar 

  7. Suen K. Guideline of the papanicolaou society of cytopathology for the examination of fine-needle aspiration specimens from the thyroid nodules. The papanicolaou society of cytopathology task force on standards of practice. Diagn Cytopathol. 1996;15:84–9.

    Article  Google Scholar 

  8. Solomon D, Davey D, Kurman R, et al. The 2001 Bethesda System: terminology for reporting results of cervical cytology. JAMA. 2002;287:2114–9.

    Article  PubMed  Google Scholar 

  9. Sakamoto A. Reporting system of thyroid cytology in Japan. J Jpn Soc Clin Cytol. 2014;53:337–41.

    Article  Google Scholar 

  10. Fuller C, Camilon R, Nguyen S, Jennings J, Day T, Gillespie MB. Adjunctive diagnostic techniques for oral lesions of unknown malignant potential: systematic review with meta-analysis. Head Neck. 2015;37:755–62.

    Article  PubMed  Google Scholar 

  11. Naito Z. JSCC Atlas and Guidelines for Cytopathological Diagnosis 5. Kanehara Shuppan Co. Ltd, Tokyo. 2015;p. 20-7.

  12. Japanese Society for Oral Tumors. General rules for clinical and pathological studies on oral cancer. 1st ed. Tokyo: Kanehara Shuppan Co. Ltd; 2010.

    Google Scholar 

  13. Karimi-Zarchi M, Peighmbari F, Karimi N, Rohi M, Chiti Z. A comparison of 3 ways of conventional pap smear, liquid-based cytology and colposcopy vs cervical biopsy for early diagnosis of premalignant lesions or cervical cancer in women with abnormal conventional Pap test. Int J Biomed Sci. 2013;9:205–10.

    PubMed  PubMed Central  Google Scholar 

  14. Afrogheh AH, Pelser A, Hille J, Schubert P. Head and neck pathology: SY13-1 liquid-based trans-epithelial flexible brush cytology of high-grade laryngeal mucosal lesions. Pathol. 46 Suppl. 2014;2:S19.

  15. Fu YS, Wenig BM, Abemayor E, Wenig BL. Head and neck pathology with clinical correlations. Philadelphia: Churchill Livingstone; 2001. p. 3–37.

    Google Scholar 

  16. Nevill BW, Damm DD, Allen CM, Bouquot JE. Abnormalities of teeth. In: Oral & maxillofacial pathology. 2nd ed. Philadelphia: W.B. Saunders; 2001. p. 315–87.

    Google Scholar 

  17. Cawson RA, Binnie WH, Barrett AW, Wright JM. Chronic white lesions and premalignant lesions. In: Oral disease. 3rd ed. London: Mosby; 2001. p. 14.1–14.24.

    Google Scholar 

  18. Mehrotra R, Singh MK, Pandya S, Singh M. The use of an oral brush biopsy without computer-assisted analysis in the evaluation of oral lesions: a study of 94 patients. Oral Surg Oral Med Oral Pathol Oral Radiol Endod. 2008;106:246–53.

    Article  PubMed  Google Scholar 

  19. Kaugars GE, Silverman Jr S, Ray A, Page DG, Abbey LM, Burns JC. The use of exfoliative cytology for the early diagnosis of oral cancers: is there a role for it in education and private practice? J Cancer Educ. 1998;13:85–9.

    CAS  PubMed  Google Scholar 

  20. Sciubba JJ. Improving detection of precancerous and cancerous oral lesions. Computer-assisted analysis of the oral brush biopsy. U.S. Collaborative OralCDx Study Group. J Am Dent Assoc. 1999;130:1445–57.

    Article  CAS  PubMed  Google Scholar 

  21. Sekine J, Nakatani E, Ohira K, Kanno T, Nariai Y, Kagimura T, Urano T. Nucleus accumbens-associated protein 1 expression has potential as a marker for distinguishing oral epithelial dysplasia and squamous cell carcinoma. PLoS One. 2015;10:e0131752.

    Article  PubMed  PubMed Central  Google Scholar 

  22. Scheifele C, Schmidt-Westhausen AM, Dietrich T, Reichart PA. The sensitivity and specificity of the OralCDx technique: evaluation of 103 cases. Oral Oncol. 2004;40:824–8.

    Article  PubMed  Google Scholar 

Download references

Acknowledgments

The authors thank all the raters in this study. We also thank all members of the Oral Cytology Working Group of the Japanese Society of Clinical Cytology for their cooperation and supplying the cytological slides and histological data. The Translational Research Informatics Center (TRI) was not involved in the process of data collection and analyzed the provided dataset that the principal investigator collected. The principal investigator guarantees the quality of the data.

Funding

This study was funded by the Japanese Society of Clinical Cytology (Head: President Hiroshi Sasaki). First author has received a donation of 3,000,000 Japanese yen from the Japanese Society of Clinical Cytology for this research project. The funder had no role in the study design.

Availability of data and materials

Please contact author for data requests.

Authors’ contributions

JS participated in cytological diagnosis and drafted the manuscript. KH and TI participated in the design of the study and cytological diagnosis. EN participated in the design of the study and performed the statistical analysis. JS and HS conceived of the study, and participated in its design and coordination and helped to draft the manuscript. All authors read and approved the final manuscript.

Authors’ information

None.

Competing interests

The authors declare that they have no competing interests.

Consent for publication

Not applicable.

Ethics approval and consent to participate

All procedures performed in this study were approved by the ethical committee at Shimane University (Approval no. 1270; March 29, 2013). It was not necessary to obtain approval from the ethical committee of the authors’ institution, because the study used the hospital database of Shimane University and those of four other hospitals employing unlinked anonymity. The unlinked anonymity of the patients was finally ensured by the president of Shimane University Faculty of Medicine. For this retrospective study formal informed consent is not required.

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Joji Sekine.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Sekine, J., Nakatani, E., Hideshima, K. et al. Diagnostic accuracy of oral cancer cytology in a pilot study. Diagn Pathol 12, 27 (2017). https://doi.org/10.1186/s13000-017-0618-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13000-017-0618-3

Keywords