A comparability study of 5 commercial KRAS tests
Diagnostic Pathology volume 5, Article number: 23 (2010)
Activating mutations in the KRAS gene occur frequently in human tumors, including colorectal carcinomas; most mutations occur in codons 12 and 13. Mutations in KRAS have been associated with poor response to anti-epidermal growth factor receptor antibodies. Therefore, an accurate and readily available analysis of KRAS mutational status is needed. The aim of this study was to evaluate concordance between KRAS assays performed by 6 different laboratories.
Forty formalin-fixed paraffin-embedded colorectal cancer tumor samples were obtained. Sample sections were submitted for KRAS mutation analysis to 5 independent commercial laboratories (Agencourt, Gentris, Genzyme, HistoGeneX, and Invitek) and to the Amgen DNA Sequencing Laboratory for direct polymerase chain reaction sequencing. The assay used by Invitek is no longer commercially available and has been replaced by an alternative technique. Results from the commercial services were compared with those from Amgen direct sequencing by κ statistics.
KRAS mutations were observed in codon 12 and/or 13 in 20 of 40 (50%) samples in Amgen direct sequencing assays. Results from HistoGeneX (κ = 0.95), Genzyme (κ = 0.94), and Agencourt (κ = 0.94) were in almost perfect agreement with these results, and the results from Gentris were in substantial agreement with the results from Amgen (κ = 0.75). The Invitek allele-specific assay demonstrated slight agreement (κ = 0.13).
This study provides data on the comparability of KRAS mutational analyses. The results suggest that most (but not all) commercial services provide analysis that is accurate and comparable with direct sequencing.
Inhibitors of epidermal growth factor receptor (EGFR), including the monoclonal antibodies panitumumab and cetuximab, have recently emerged as treatment options for metastatic colorectal cancer (mCRC) [1, 2]. Mutations in KRAS have been associated with poor responses to both cetuximab and panitumumab in patients with CRC . The aim of this study was to evaluate comparability among KRAS assays performed by 6 different laboratories in order to identify a vendor and assay that would be used to determine the clinical utility of KRAS in our pivotal panitumumab trial in mCRC .
Tissue Samples and DNA Isolation
Formalin-fixed paraffin-embedded (FFPE) human CRC samples (N = 40) were obtained from the following procurement service providers: Asterand plc (Detroit, MI), Ardais Corp (Lexington, MA), and the National Disease Research Interchange (Philadelphia, PA). Fourteen (35%) samples were from men, 20 (50%) were from women, and 6 (15%) were unassigned (Table 1). The median age was 67 years (range, 35-94 y); age data were not available for 7 patients. The samples were primary resections from colon adenocarcinoma (n = 36), rectum adenocarcinoma (n = 3), and rectum carcinoma (n = 1) with a range of poorly to well-differentiated tumors of different stages with variable tumor, normal, stromal, and necrotic content. The aim was to select samples that were representative of samples expected in clinical trials. All study procedures were conducted in accordance with the Declaration of Helsinki.
Mutational analysis of KRAS sequences was performed by the Amgen DNA Sequencing Laboratory and 5 independent laboratories that provide diagnostic services for academic/clinical research laboratories and/or clinical trials. Each laboratory was provided with 10-μm tissue sections from all 40 samples in 2007 except Agencourt. Agencourt was provided with extracted genomic DNA from the remaining 35 specimens in 2009. DNA was extracted using the QIAamp FFPE Tissue kit (Qiagen Inc, Carlsbad, CA) according to the manufacturer's instructions, with the addition of a 16-hour proteinase K lysis step.
Direct Sequencing of KRAS by the Amgen DNA Sequencing Laboratory
Exon 2 of KRAS was amplified from isolated genomic DNA using the Roche Expand Long Template PCR System (Roche Applied Science, Indianapolis, IN). The forward primer sequence was 5'-AAGGTACTGGTGGAGTATTTG-3', and the reverse was 5'-GTACTCATGAAAATGGTCAGAG-3', resulting in a predicted amplicon length of 295 bp. Cycling conditions were as follows: 93°C, 3 minutes; 40-46 cycles at 93°C, 15 seconds; 62°C, 30 seconds; 72°C, 30 seconds; and 72°C, 4 minutes. Polymerase chain reaction (PCR) products were directly sequenced in triplicate (3730xl DNA Analyzers; Applied Biosystems). Sequences were analyzed using Sequencher® software (Gene Codes Corporation, Ann Arbor, MI). Results were reported when all 3 parallel PCR products generated 4 acceptable sequences, resulting in a total of 12 sequences for each sample.
Commercial KRAS Mutation Analysis
Five commercial services were contracted to analyze KRAS mutational status. HistoGeneX (Antwerp, Belgium) used the DxS K-RAS Mutation Test Kit (DxS Ltd, Manchester, UK) that interrogates the 7 most common somatic mutations of codons 12 and 13 (G12A, G12D, G12R, G12V, G12C, G12S, and G13D) using allele-specific PCR amplification with an amplification-refractory mutation system technology in combination with Scorpion® detection to measure amplification products by real-time PCR [5, 6]. The DxS K-RAS Mutation Test Kit is capable of detecting approximately 1% of mutated KRAS from background wild-type DNA.
Gentris Clinical Genetics, Inc. (Morrisville, NC, USA) assessed KRAS mutational status by direct DNA sequencing of codons 12 and 13. Gentris used a pathology service (Integrated Laboratory Systems, Inc.) to enrich for tumor tissue by removing normal tissue from the slides and then extracting genomic DNA from the remaining material. Isolated genomic DNA was then amplified with primers that spanned codons 12 and 13 of KRAS, and the resulting PCR products were subjected to cycle sequencing in reverse reactions.
Genzyme Corporation (Cambridge, MA) used an allele-specific primer extension technique and reported results for any mutation of codons 12 and 13.
Invitek GmbH (Berlin, Germany) used their Invisorb® Spin Tissue Mini Kit for DNA extraction and evaluated KRAS mutational status using their Invigene® K-ras Genotyping Kit, which employs allele-specific oligonucleotide hybridization. Results were reported for 6 possible mutations at each of codons 12 and 13.
Agencourt Bioscience Corporation conducted picotiter plate pyrosequencing using the Roche 454 system described previously [7–9]. Briefly, unique barcode-containing primers were used to amplify KRAS exon 2. All the uniquely tagged samples were pooled at equimolar concentrations and adaptors were ligated to allow bead binding. Optimized mixing conditions were then used to ensure that each bead-bound DNA molecule was within an oil droplet that contained PCR reagents. Emulsion PCR was performed using primers that included biotin to enable selective capture of those beads where successful amplification took place. The emulsion was subsequently broken, the biotin-containing beads captured, and each bead loaded by centrifugation into 1 well of a picotiter plate. Traditional pyrosequencing was performed, yielding a library of sequences linked to unique barcodes. Results were reported as the sequences derived from flowgrams, together with the 10-bp sequences employed for each sample to allow deconvolution of the data at Amgen. The number of mutant and wild-type sequences for each sample was reported.
KRAS genotype was classified as either wild type or mutant. Techniques were considered to be in agreement if both identified wild type or a mutant, even if the allele reported was different. Agreement was assessed by κ statistics  using SAS/STAT® software version 9.1 (SAS Institute, Cary, NC).
Results of Direct Sequencing by the Amgen DNA Sequencing Laboratory
Twenty (50%) patients had KRAS exon 2 mutations in sequencing performed by Amgen: G13D (n = 10), G12D (n = 5), G12V (n = 5), G12A (n = 2), and G12S (n = 1) (Table 2). Three of the G13D mutations occurred in combination with a codon 12 mutation.
Comparability of KRAS Mutational Analysis
Variation was observed in the comparability of direct sequencing by Amgen and the results of commercial analyses as shown by the tabulation of the reported results (Table 2) and the comparison of these results using the κ statistic (Table 3). The results of allele-specific testing by HistoGeneX using the DxS kit (κ = 0.95; 95% CI, 0.83—1.00) and by Genzyme (κ = 0.94; 95% CI, 0.85—1.00) closely matched the results of direct sequencing by Amgen, differing on 2/40 samples and 1/35 samples, respectively. The results of picotiter plate sequencing by Agencourt were also almost in perfect agreement with the results of direct sequencing by Amgen (κ = 0.94; 95% CI 0.52—0.98; 32 results reported from 35 samples). Results from the 2 techniques differed in only 1 sample when making a determination of KRAS wild-type versus mutant sequence. Amgen direct sequencing identified sample 9 as wild type, whereas Agencourt picotiter plate sequencing detected a G12V mutation. The results of direct sequencing by Gentris were in substantial agreement with the results of direct sequencing by Amgen, with disagreement on 4 samples (κ = 0.75; 95% CI, 0.52—0.98; 32 results reported from 40 samples). The results of mutational analysis by allele-specific oligonucleotide hybridization by Invitek (as conducted in 2007) were in slight agreement with the results of direct sequencing by Amgen (κ = 0.13; 95% CI -0.15—0.42; 27 results reported from 40 samples). KRAS analysis by Invitek identified only 3 wild-type samples, 2 of which were in agreement with direct sequencing by Amgen.
The quantitative nature of the Agencourt picotiter plate sequencing assay, which generates counts of mutant and wild-type sequences, provided additional data that, in many instances, allowed clarification of the results from other laboratories (Table 2). For example, 3 of the 5 other laboratories detected the G12S mutation in sample 7, whereas picotiter plate sequencing identified the mutant allele in 6% of the DNA. In sample 21, direct sequencing by Amgen identified a G13D mutation, HistoGeneX detected a G12D mutation, and Invitek and Genzyme detected both G13D and G12D mutants. Picotiter plate sequencing identified G13D and G12D mutations in 26% and 12% of the DNA, respectively. In sample 9, picotiter plate sequencing identified a G12V mutation in 41% of sample DNA, whereas the other assays identified a wild-type genotype. It is unclear whether this is due to tumor heterogeneity or other factors.
The results of this study suggest that analyses of KRAS mutational status by most commercial services are accurate and are generally in close agreement with results of direct sequencing by Amgen. The results from Agencourt, HistoGeneX/DxS, and Genzyme were in almost perfect agreement with Amgen sequencing data and the results from Gentris were in substantial agreement with the Amgen analysis, whereas the results from Invitek were in slight agreement. It should be noted that Invitek has released a new assay since these comparisons were performed in 2007. The allele-specific oligonucleotide hybridization assay used in this study is no longer commercially available and has been replaced by an alternative technique. The frequency of KRAS mutation (20/40 as assessed by the Amgen laboratory) in the tumor biopsy specimens used in this study is consistent with that previously observed in clinical studies investigating KRAS mutations in CRC [11, 12]. In 2007, these data (together with other considerations) led to the selection of the DxS assay performed by HistoGeneX for the evaluation of KRAS mutational status in samples from the panitumumab pivotal trial .
Several factors may have contributed to the lack of agreement between KRAS testing assays observed in this study. These factors may have included differences in the sensitivity and specificity of KRAS tests, differences in sample processing and DNA enrichment techniques between laboratories, and variation in tumor content between tumor samples and within tumor samples. Although direct sequencing is typically considered the gold standard for mutational analysis, emulsion PCR followed by picotiter plate sequencing may offer several advantages . Because of the high sensitivity of picotiter plate sequencing, accurate detection of very low abundance mutations in a tumor sample is feasible . Indeed, this highlights an important issue regarding sample selection for concordance studies of somatic mutation analysis in solid tumor specimens. Ideally, samples for concordance studies would comprise serial sections from a single homogeneous tumor block, allowing both preanalytical and analytical variables to be measured as a composite from a single sample. However, serial sections from a single block distributed amongst analytical platforms is not ideal given the likelihood of variable ratios of tumor:normal:stromal:necrotic tissue among sections and the potential for tumor heterogeneity (as suggested by results for samples 8 and 21, which contained 2 different mutations that might vary in ratio through the block). We propose that concordance testing should be a 2-step process using separate sets of samples. The first would determine the ability to successfully extract PCR-amplifiable DNA (by employing sections from a range of tumor blocks with variable tumor:normal:stromal:necrotic tissue ratios) with a second set of samples (comprising aliquots from DNA extracts with predetermined mutant:wild-type somatic mutation ratios) to determine the ability to correctly report mutations. Some of these samples should be close to the lower level of sensitivity expected of such assays (eg., 1%).
KRAS mutation assays may provide physicians with guidance when making decisions on the use of anti-EGFR therapy in the treatment of metastatic CRC . However, the lack of concordance among results of some assay techniques emphasizes the importance of a rigorous quality assurance program to ensure that results are of high quality [15–17], potentially using the sample types described above.
Whitehall et al recently published a study comparing 6 KRAS mutation assays in the analysis of primary colorectal cancer samples . As in this study, most techniques assessed were found to be in good agreement, with some notable exceptions. The key strength of our study was that laboratories (with the exception of Agencourt) analyzed KRAS from tumor sections on glass slides rather than from DNA, and the results were consequently a closer mimic of clinical testing procedures. A recent study by Weichert et al compared 4 KRAS mutation assays of primary colorectal cancer samples and metastases using DNA extracted from tumor sections on glass slides and similarly determined that the assay techniques were generally in good agreement .
The results of this study suggest that most (but not all) commercial services provide analysis that is accurate and comparable with direct sequencing. However, the lack of concordance among results of some assay techniques, suggest that rigorous quality assurance programs will be necessary to ensure consistent and accurate results.
Jonker DJ, O'Callaghan CJ, Karapetis CS, Zalcberg JR, Tu D, Au HJ, Berry SR, Krahn M, Price T, Simes RJ, Tebbutt NC, van Hazel G, Wierzbicki R, Langer C, Moore MJ: Cetuximab for the treatment of colorectal cancer. N Engl J Med. 2007, 357 (20): 2040-2048. 10.1056/NEJMoa071834.
Van Cutsem E, Peeters M, Siena S, Humblet Y, Hendlisz A, Neyns B, Canon JL, Van Laethem JL, Maurel J, Richardson G, Wolf M, Amado RG: Open-label phase III trial of panitumumab plus best supportive care compared with best supportive care alone in patients with chemotherapy-refractory metastatic colorectal cancer. J Clin Oncol. 2007, 25 (13): 1658-1664. 10.1200/JCO.2006.08.1620.
Jimeno A, Messersmith WA, Hirsch FR, Franklin WA, Eckhardt SG: KRAS mutations and sensitivity to epidermal growth factor receptor inhibitors in colorectal cancer: practical application of patient selection. J Clin Oncol. 2009, 27 (7): 1130-1136. 10.1200/JCO.2008.19.8168.
Amado RG, Wolf M, Peeters M, Van Cutsem E, Siena S, Freeman DJ, Juan T, Sikorski R, Suggs S, Radinsky R, Patterson SD, Chang DD: Wild-type KRAS is required for panitumumab efficacy in patients with metastatic colorectal cancer. J Clin Oncol. 2008, 26 (10): 1626-1634. 10.1200/JCO.2007.14.7116.
Clayton SJ, Scott FM, Walker J, Callaghan K, Haque K, Liloglou T, Xinarianos G, Shawcross S, Ceuppens P, Field JK, Fox JC: K-ras point mutation detection in lung cancer: comparison of two approaches to somatic mutation detection using ARMS allele-specific amplification. Clin Chem. 2000, 46 (12): 1929-1938.
Whitcombe D, Theaker J, Guy SP, Brown T, Little S: Detection of PCR products using self-probing amplicons and fluorescence. Nat Biotechnol. 1999, 17 (8): 804-807. 10.1038/11751.
Thomas RK, Nickerson E, Simons JF, Janne PA, Tengs T, Yuza Y, Garraway LA, LaFramboise T, Lee JC, Shah K, O'Neill K, Sasaki H, Lindeman N, Wong KK, Borras AM, Gutmann EJ, Dragnev KH, DeBiasi R, Chen TH, Glatt KA, Greulich H, Desany B, Lubeski CK, Brockman W, Alvarez P, Hutchison SK, Leamon JH, Ronan MT, Turenchalk GS, Egholm M: Sensitive mutation detection in heterogeneous cancer specimens by massively parallel picoliter reactor sequencing. Nat Med. 2006, 12 (7): 852-855. 10.1038/nm1437.
Parameswaran P, Jalili R, Tao L, Shokralla S, Gharizadeh B, Ronaghi M, Fire AZ: A pyrosequencing-tailored nucleotide barcode design unveils opportunities for large-scale sample multiplexing. Nucleic Acids Res. 2007, 35 (19): e130-10.1093/nar/gkm760.
Margulies M, Egholm M, Altman WE, Attiya S, Bader JS, Bemben LA, Berka J, Braverman MS, Chen YJ, Chen Z, Dewell SB, Du L, Fierro JM, Gomes XV, Godwin BC, He W, Helgesen S, Ho CH, Irzyk GP, Jando SC, Alenquer ML, Jarvie TP, Jirage KB, Kim JB, Knight JR, Lanza JR, Leamon JH, Lefkowitz SM, Lei M, Li J: Genome sequencing in microfabricated high-density picolitre reactors. Nature. 2005, 437 (7057): 376-380.
Viera AJ, Garrett JM: Understanding interobserver agreement: the kappa statistic. Fam Med. 2005, 37 (5): 360-363.
Andreyev HJ, Norman AR, Cunningham D, Oates J, Dix BR, Iacopetta BJ, Young J, Walsh T, Ward R, Hawkins N, Beranek M, Jandik P, Benamouzig R, Jullian E, Laurent-Puig P, Olschwang S, Muller O, Hoffmann I, Rabes HM, Zietz C, Troungos C, Valavanis C, Yuen ST, Ho JW, Croke CT, O'Donoghue DP, Giaretti W, Rapallo A, Russo A, Bazan V: Kirsten ras mutations in patients with colorectal cancer: the 'RASCAL II' study. Br J Cancer. 2001, 85 (5): 692-696. 10.1054/bjoc.2001.1964.
Andreyev HJ, Norman AR, Cunningham D, Oates JR, Clarke PA: Kirsten ras mutations in patients with colorectal cancer: the multicenter "RASCAL" study. J Natl Cancer Inst. 1998, 90 (9): 675-684. 10.1093/jnci/90.9.675.
Voelkerding KV, Dames SA, Durtschi JD: Next-generation sequencing: from basic research to diagnostics. Clin Chem. 2009, 55 (4): 641-658. 10.1373/clinchem.2008.112789.
Allegra CJ, Jessup JM, Somerfield MR, Hamilton SR, Hammond EH, Hayes DF, McAllister PK, Morton RF, Schilsky RL: American Society of Clinical Oncology provisional clinical opinion: testing for KRAS gene mutations in patients with metastatic colorectal carcinoma to predict response to anti-epidermal growth factor receptor monoclonal antibody therapy. J Clin Oncol. 2009, 27 (12): 2091-2096. 10.1200/JCO.2009.21.9170.
van Krieken JH, Jung A, Kirchner T, Carneiro F, Seruca R, Bosman FT, Quirke P, Flejou JF, Plato Hansen T, de Hertogh G, Jares P, Langner C, Hoefler G, Ligtenberg M, Tiniakos D, Tejpar S, Bevilacqua G, Ensari A: KRAS mutation testing for predicting response to anti-EGFR therapy for colorectal carcinoma: proposal for an European quality assurance program. Virchows Arch. 2008, 453 (5): 417-431. 10.1007/s00428-008-0665-y.
Perspectives on Emerging Technology (POET) Report: KRAS Mutation Testing for Colorectal Cancer (CRC). 2009, Northfield, IL: College of American Pathologists
Wang HL, Lopategui J, Amin MB, Patterson SD: KRAS mutation testing in human cancers: The pathologist's role in the era of personalized medicine. Adv Anat Pathol. 2010, 17 (1): 23-32.
Whitehall V, Tran K, Umapathy A, Grieu F, Hewitt C, Evans T-J, Ismail T, Li WQ, Collins P, Ravetto P, Leggett B, Salto-Tellez M, Soong R, Fox S, Scott RJ, Dobrovic A, Iacopetta B: A multicenter blinded study to evaluate KRAS mutation testing methodologies in the clinical setting. J Mol Diagn. 2009, 11 (6): 543-552. 10.2353/jmoldx.2009.090057.
Weichert W, Schewe C, Lehmann A, Sers C, Denkert C, Budczies J, Stenzinger A, Joos H, Landt O, Heiser V, Rocken C, Dietel M: KRAS genotyping of paraffin-embedded colorectal cancer tissue in routine diagnostics: comparison of methods and impact of histology. J Mol Diagn. 2010, 12 (1): 35-42. 10.2353/jmoldx.2010.090079.
This manuscript is dedicated to the memory of Dr. Ildiko Sarosi who worked tirelessly in the pursuit of excellence in pathology to enhance the drug discovery and development process. The authors would like to thank Nancy Shen Chow and Jennifer Hawkins for experimental work, Ali Hassan, PhD (Complete Healthcare Communications, Inc.), whose work was funded by Amgen Inc., for assistance in the preparation of this manuscript. Financial support for this study was provided by Amgen Inc.
KO, TJ, SS, MW, IS, DJF, TG, WB, AB, AP, and SDP are employees and shareholders in Amgen Inc.
KO, TJ, SS, IS, DJF, and SDP contributed to the conception and design of the study; IS, TG, and WB provided study material; TJ, SS, TG, WB, AB, and AP provided study materials; TJ, MW, and AP analyzed and interpreted the data; KO and SDP wrote the manuscript. All authors read and approved the final manuscript.
About this article
Cite this article
Oliner, K., Juan, T., Suggs, S. et al. A comparability study of 5 commercial KRAS tests. Diagn Pathol 5, 23 (2010). https://doi.org/10.1186/1746-1596-5-23