A comparability study of 5 commercial KRAS tests

Background Activating mutations in the KRAS gene occur frequently in human tumors, including colorectal carcinomas; most mutations occur in codons 12 and 13. Mutations in KRAS have been associated with poor response to anti-epidermal growth factor receptor antibodies. Therefore, an accurate and readily available analysis of KRAS mutational status is needed. The aim of this study was to evaluate concordance between KRAS assays performed by 6 different laboratories. Methods Forty formalin-fixed paraffin-embedded colorectal cancer tumor samples were obtained. Sample sections were submitted for KRAS mutation analysis to 5 independent commercial laboratories (Agencourt, Gentris, Genzyme, HistoGeneX, and Invitek) and to the Amgen DNA Sequencing Laboratory for direct polymerase chain reaction sequencing. The assay used by Invitek is no longer commercially available and has been replaced by an alternative technique. Results from the commercial services were compared with those from Amgen direct sequencing by κ statistics. Results KRAS mutations were observed in codon 12 and/or 13 in 20 of 40 (50%) samples in Amgen direct sequencing assays. Results from HistoGeneX (κ = 0.95), Genzyme (κ = 0.94), and Agencourt (κ = 0.94) were in almost perfect agreement with these results, and the results from Gentris were in substantial agreement with the results from Amgen (κ = 0.75). The Invitek allele-specific assay demonstrated slight agreement (κ = 0.13). Conclusions This study provides data on the comparability of KRAS mutational analyses. The results suggest that most (but not all) commercial services provide analysis that is accurate and comparable with direct sequencing.


Background
Inhibitors of epidermal growth factor receptor (EGFR), including the monoclonal antibodies panitumumab and cetuximab, have recently emerged as treatment options for metastatic colorectal cancer (mCRC) [1,2]. Mutations in KRAS have been associated with poor responses to both cetuximab and panitumumab in patients with CRC [3]. The aim of this study was to evaluate comparability among KRAS assays performed by 6 different laboratories in order to identify a vendor and assay that would be used to determine the clinical utility of KRAS in our pivotal panitumumab trial in mCRC [4].

Tissue Samples and DNA Isolation
Formalin-fixed paraffin-embedded (FFPE) human CRC samples (N = 40) were obtained from the following pro-curement service providers: Asterand plc (Detroit, MI), Ardais Corp (Lexington, MA), and the National Disease Research Interchange (Philadelphia, PA). Fourteen (35%) samples were from men, 20 (50%) were from women, and 6 (15%) were unassigned (Table 1). The median age was 67 years (range, 35-94 y); age data were not available for 7 patients. The samples were primary resections from colon adenocarcinoma (n = 36), rectum adenocarcinoma (n = 3), and rectum carcinoma (n = 1) with a range of poorly to well-differentiated tumors of different stages with variable tumor, normal, stromal, and necrotic content. The aim was to select samples that were representative of samples expected in clinical trials. All study procedures were conducted in accordance with the Declaration of Helsinki.
Mutational analysis of KRAS sequences was performed by the Amgen DNA Sequencing Laboratory and 5 independent laboratories that provide diagnostic services for academic/clinical research laboratories and/or clinical trials. Each laboratory was provided with 10-μm tissue sections from all 40 samples in 2007 except Agencourt. Agencourt was provided with extracted genomic DNA from the remaining 35 specimens in 2009. DNA was extracted using the QIAamp FFPE Tissue kit (Qiagen Inc, Carlsbad, CA) according to the manufacturer's instructions, with the addition of a 16-hour proteinase K lysis step.

Direct Sequencing of KRAS by the Amgen DNA Sequencing Laboratory
Exon 2 of KRAS was amplified from isolated genomic DNA using the Roche Expand Long Template PCR System (Roche Applied Science, Indianapolis, IN). The forward primer sequence was 5'-AAGGTACTGGTGGAG TATTTG-3', and the reverse was 5'-GTACTCAT-GAAAATGGTCAGAG-3', resulting in a predicted amplicon length of 295 bp. Cycling conditions were as follows: 93°C, 3 minutes; 40-46 cycles at 93°C, 15 seconds; 62°C, 30 seconds; 72°C, 30 seconds; and 72°C, 4 minutes. Polymerase chain reaction (PCR) products were directly sequenced in triplicate (3730xl DNA Analyzers; Applied Biosystems). Sequences were analyzed using Sequencher ® software (Gene Codes Corporation, Ann Arbor, MI). Results were reported when all 3 parallel PCR products generated 4 acceptable sequences, resulting in a total of 12 sequences for each sample.

Commercial KRAS Mutation Analysis
Five commercial services were contracted to analyze KRAS mutational status. HistoGeneX (Antwerp, Belgium) used the DxS K-RAS Mutation Test Kit (DxS Ltd, Manchester, UK) that interrogates the 7 most common somatic mutations of codons 12 and 13 (G12A, G12D, G12R, G12V, G12C, G12S, and G13D) using allele-specific PCR amplification with an amplification-refractory mutation system technology in combination with Scor-pion ® detection to measure amplification products by real-time PCR [5,6]. The DxS K-RAS Mutation Test Kit is capable of detecting approximately 1% of mutated KRAS from background wild-type DNA.
Gentris Clinical Genetics, Inc. (Morrisville, NC, USA) assessed KRAS mutational status by direct DNA sequencing of codons 12 and 13. Gentris used a pathology service (Integrated Laboratory Systems, Inc.) to enrich for tumor tissue by removing normal tissue from the slides and then extracting genomic DNA from the remaining material. Isolated genomic DNA was then amplified with primers that spanned codons 12 and 13 of KRAS, and the resulting PCR products were subjected to cycle sequencing in reverse reactions.
Genzyme Corporation (Cambridge, MA) used an allele-specific primer extension technique and reported results for any mutation of codons 12 and 13.
Invitek GmbH (Berlin, Germany) used their Invisorb ® Spin Tissue Mini Kit for DNA extraction and evaluated KRAS mutational status using their Invigene ® K-ras Genotyping Kit, which employs allele-specific oligonucleotide hybridization. Results were reported for 6 possible mutations at each of codons 12 and 13.
Agencourt Bioscience Corporation conducted picotiter plate pyrosequencing using the Roche 454 system described previously [7][8][9]. Briefly, unique barcode-containing primers were used to amplify KRAS exon 2. All the uniquely tagged samples were pooled at equimolar concentrations and adaptors were ligated to allow bead binding. Optimized mixing conditions were then used to ensure that each bead-bound DNA molecule was within an oil droplet that contained PCR reagents. Emulsion PCR was performed using primers that included biotin to enable selective capture of those beads where successful amplification took place. The emulsion was subsequently broken, the biotin-containing beads captured, and each bead loaded by centrifugation into 1 well of a picotiter plate. Traditional pyrosequencing was performed, yielding a library of sequences linked to unique barcodes. Results were reported as the sequences derived from flowgrams, together with the 10-bp sequences employed for each sample to allow deconvolution of the data at Amgen. The number of mutant and wild-type sequences for each sample was reported.

Statistical Analysis
KRAS genotype was classified as either wild type or mutant. Techniques were considered to be in agreement if both identified wild type or a mutant, even if the allele reported was different. Agreement was assessed by κ statistics [10] using SAS/STAT ® software version 9.1 (SAS Institute, Cary, NC).

Comparability of KRAS Mutational Analysis
Variation was observed in the comparability of direct sequencing by Amgen and the results of commercial analyses as shown by the tabulation of the reported results (Table 2) and the comparison of these results using the κ statistic ( The quantitative nature of the Agencourt picotiter plate sequencing assay, which generates counts of mutant and wild-type sequences, provided additional data that, in many instances, allowed clarification of the results from other laboratories (Table 2). For example, 3 of the 5 other laboratories detected the G12S mutation in sample 7, whereas picotiter plate sequencing identified the mutant allele in 6% of the DNA. In sample 21, direct sequencing by Amgen identified a G13D mutation, HistoGeneX detected a G12D mutation, and Invitek and Genzyme detected both G13D and G12D mutants. Picotiter plate sequencing identified G13D and G12D mutations in 26% and 12% of the DNA, respectively. In sample 9, picotiter plate sequencing identified a G12V mutation in 41% of sample DNA, whereas the other assays identified a wildtype genotype. It is unclear whether this is due to tumor heterogeneity or other factors.

Discussion
The results of this study suggest that analyses of KRAS mutational status by most commercial services are accurate and are generally in close agreement with results of direct sequencing by Amgen. The results from Agencourt, HistoGeneX/DxS, and Genzyme were in almost perfect agreement with Amgen sequencing data and the results from Gentris were in substantial agreement with the Amgen analysis, whereas the results from Invitek were in slight agreement. It should be noted that Invitek has released a new assay since these comparisons were performed in 2007. The allele-specific oligonucleotide hybridization assay used in this study is no longer commercially available and has been replaced by an alternative technique. The frequency of KRAS mutation (20/40 as assessed by the Amgen laboratory) in the tumor biopsy specimens used in this study is consistent with that previously observed in clinical studies investigating KRAS mutations in CRC [11,12]. In 2007, these data (together with other considerations) led to the selection of the DxS assay performed by HistoGeneX for the evaluation of KRAS mutational status in samples from the panitumumab pivotal trial [4].
Several factors may have contributed to the lack of agreement between KRAS testing assays observed in this study. These factors may have included differences in the sensitivity and specificity of KRAS tests, differences in sample processing and DNA enrichment techniques between laboratories, and variation in tumor content between tumor samples and within tumor samples. Although direct sequencing is typically considered the gold standard for mutational analysis, emulsion PCR followed by picotiter plate sequencing may offer several advantages [13]. Because of the high sensitivity of picotiter plate sequencing, accurate detection of very low abundance mutations in a tumor sample is feasible [7]. Indeed, this highlights an important issue regarding sample selection for concordance studies of somatic mutation analysis in solid tumor specimens. Ideally, samples for concordance studies would comprise serial sections from a single homogeneous tumor block, allowing both preanalytical and analytical variables to be measured as a composite from a single sample. However, serial sections from a single block distributed amongst analytical platforms is not ideal given the likelihood of variable ratios of tumor:normal:stromal:necrotic tissue among sections and the potential for tumor heterogeneity (as suggested by results for samples 8 and 21, which contained 2 different mutations that might vary in ratio through the block). We propose that concordance testing should be a 2-step process using separate sets of samples. The first would determine the ability to successfully extract PCR-amplifiable DNA (by employing sections from a range of tumor blocks with variable tumor:normal:stromal:necrotic tissue ratios) with a second set of samples (comprising aliquots from DNA extracts with predetermined mutant:wild-type somatic mutation ratios) to determine the ability to correctly report mutations. Some of these samples should be close to the lower level of sensitivity expected of such assays (eg., 1%). KRAS mutation assays may provide physicians with guidance when making decisions on the use of anti-EGFR therapy in the treatment of metastatic CRC [14]. However, the lack of concordance among results of some assay techniques emphasizes the importance of a rigorous quality assurance program to ensure that results are of high quality [15][16][17], potentially using the sample types described above.
Whitehall et al recently published a study comparing 6 KRAS mutation assays in the analysis of primary colorectal cancer samples [18]. As in this study, most techniques assessed were found to be in good agreement, with some notable exceptions. The key strength of our study was that laboratories (with the exception of Agencourt) analyzed KRAS from tumor sections on glass slides rather than from DNA, and the results were consequently a closer mimic of clinical testing procedures. A recent study by Weichert et al compared 4 KRAS mutation assays of primary colorectal cancer samples and metastases using DNA extracted from tumor sections on glass slides and similarly determined that the assay techniques were generally in good agreement [19].

Conclusions
The results of this study suggest that most (but not all) commercial services provide analysis that is accurate and **The single number in parentheses refers to the total sequence reads for that sample when all were WT, and for MT, the fraction reflects the number of sequence reads for a given mutation over the total number of sequence reads for that sample, followed by the numerical fraction of that ratio. comparable with direct sequencing. However, the lack of concordance among results of some assay techniques, suggest that rigorous quality assurance programs will be necessary to ensure consistent and accurate results.