Teachers’ impact on dental students’ exam scores in teaching pathology of the oral cavity using WSI

We have developed and evaluated a user-friendly on-line interactive teaching and examination system for Pathology of the Oral Cavity. Abandoning the use of conventional training microscopes and glass slides since 2005, we decided to rely on virtual microscopy to facilitate learning of oral cavity pathology to dental students. Because of a high number of new students enrolling every academic year, students work in groups supervised by different teaching assistants with heterogeneous teaching experience. The aim of this study was determining whether in this context the teacher has still an impact on students’ exam scores.

Teachers' impact on dental students' exam scores in teaching pathology of the oral cavity using WSI Janusz Szymas 1* , Mikael Lundin 2 , Johan Lundin 2 From 11th European Congress on Telepathology and 5th International Congress on Virtual Microscopy Venice, Italy. 6-9 June 2012 Background There is an overall increasing tendency at medical universities around the world to digitize microscope histopathological slides from teaching collections for web-based studies [1]. For the last seven academic years (2005 -2012), we have developed and evaluated a user-friendly on-line interactive teaching and examination system to facilitate student access to Pathology of the Oral Cavity course material, including histological laboratory specimens. It made microscope laboratory studies in oral cavity pathology more efficient and teaching resources have become available for interactive use from anywhere at any time, independent of class schedules. The WebMicroscope allows students to independently explore entire histological slides. Students control the content and the rhythm of delivery of the content. Digital slides produced using whole slide imaging (WSI) can be visualized at any magnification and moved in the x-y axis, which perfectly imitates using a traditional microscope and a glass slide [2].
Examination which applies WSI technology on-line requires a sophisticated test management software. In the past, we used an examination system that was restricted to using standard static images. This was technically less challenging and followed a routine which was similar to that of conventional paper-based examinations [3,4]. WSI is more demanding as it requires the use of innovative image serving and viewing software and, because of the enormous size of the images, a storage facility capable of storing hundreds of gigabytes. However, it reflects much better the usual laboratory activities [5]. Using WSI allows asking questions about the overall diagnosis without specifically highlighting the key diagnostic features as it must be done in the field of view of a static image. Even if a question refers to a specific field of view, the student can zoom out and see that field in the overall context of the slide. This contextual information easily available in virtual microscopy is essential in teaching students both how to find the key diagnostic fields within a whole slide and to perceive how a diagnostic field fits into the surrounding non-diagnostic slide context. Our first seven-year experience in developing and using an examination system which applies WSI indicates that configuring an on-line practical exam in Pathology of the Oral Cavity requires flexibility over the question types that can be asked.
Our practical examination in oral pathology is scheduled at the end of the 3rd year dental curriculum of students in the Department of Clinical Pathology at the University of Medical Sciences in Poznan, Poland. Because the number of new students enrolling every academic year is high, students work in groups which are supervised by different teaching assistants with heterogeneous teaching experience. The aim of this study was identifying if in such a context the teacher has still an impact on the students' exam scores.

Material and methods
Each practical exam consisted of 50 multiple-choice questions per student. Every question was displayed together with the adequate whole slide image and students had to provide an answer based on the interpretation of this slide. The scope of whole slide images were broad, designed to include all slides used during the laboratory practicals in the Pathology of the Oral Cavity. Questions and whole slide images were presented in a different order to every student. For the examination in the academic years 2008 -2012, we used 50% of randomly rotated whole slide images to prevent "fossilization" of the students. Students were shown how to use the software at the beginning of the practical exam and were given unlimited time to complete the exam. There was no time limit per slide either. Students could also view slides or answer questions in any order. All sessions of the practical exam using WSI took place in a secure location, the computer laboratory at the Department of Pathology, and were proctored all the time by teaching assistants. All examinations were graded using the same scale. Participants were all dental students enrolled to the Pathology of the Oral Cavity course between 2005 and 2012.Students were also required to complete a short anonymous survey at the time of the exam.

Server architecture
The WebMicroscope was used in the current examination system to host, manage, and deliver WSI-based digital slides to the students during the practical examination. In this context, a dedicated Web viewer interacts with an image server, which serve out appropriate regions of the slide image. In this way, students can view whole slides in real time without the need to download entire images. For the practical examination in the Pathology of the Oral Cavity, multiple image servers operated at least in tandem (Poznan, Helsinki) allowing to process multiple requests for images per second.

Client module software
The client module is responsible for presenting data to the student. It is the only part of the system visible to the user and consists mainly of an independent application which communicates with the other modules of the system through well-defined interfaces. The messages of the front-end virtual-slide viewer were written based on a native, downloadable browser plug-in, or java script based dynamic HTML. In the latter case, it does not require any specific downloads to the client work station.

Statistical analysis
The results from all students were collected and analysed automatically by the Examination module and the Statistic module of the examination system software. Already at the end of the examination session, the administrator could download results for single participants, groups of students, or the whole cohort. In parallel the software automatically compiled the data on chosen answers to multiple-choice questions and performed statistical analysis on them. Here we provide descriptive statistics and t-test analysis using the analysis dataset from Pathology of the Oral Cavity examinations.

Results
Students were surveyed regarding their preferences for the use of virtual microscopy in the curriculum based on their first exposure to this technology. The survey results indicate that an overwhelming majority of students found the WebMicroscope system relatively easy to use and helpful. Their answers to the question about the usefulness of the WSI-based examination system (rating from 0 to 10) were constant throughout the seven years of this study. The lowest mean value (8.4 ± 1.4) was recorded in 2006 and the highest (9.1 ± 1.7) in 2009 and later, after some improvements.
In the first examination, in 2006, a cohort of 8 students scored with a 98% ± 2.7%. Similar result was obtained one year later with a cohort of 84 students (97,2% ± 4.7%). At this time, 50 whole slide images were used in 2006 and 64 whole slide images in 2007 during laboratory practices and during the practical examination. With a growing number of whole slide images used in the subsequent years (81 whole slide images used in 2008 and 125 in 2011), the rate of correct students' answers to the exam questions was only a little bit lower -circa 94% (Table 1). In the last four academic years (2008 -2012), we used 50% of randomly rotated whole slide images. The rotation did not influence the performance of the students.
When analyzing the students' scores by student groups supervised by different teaching assistants, we found that the mean exam results were different between the groups. There were some student groups which scored high and some groups with lower grades ( Table 2). Differences between these student groups were statistically significant (P-value < 0.050) in all academic years except for 2009-10. Unfortunately, the rotation of teaching assistants in our Department is high but during the period of this study we could identify at least three teachers whose groups have permanently scored high or the highest during the exam. We have also noted a couple of unsuccessful teaching assistants.

Discussion
Testing large numbers of students in the exam context with many virtual slides is a challenging operation with many potential bottle necks. To overcome some of them, whole slide images were stored on an independent server. Serving these slides as part of the practical examination is bandwidth and server intense, particularly if the image latency is to be kept to a minimum with many students accessing the images simultaneously. A successful examination system requires multiple image servers which are carefully load balanced which implies that, when one server gets busy due to user activity, another server takes over and provides the services that are being requested.
For this practical examination in the Pathology of the Oral Cavity, multiple image servers operated at least in tandem (Poznan, Helsinki) allowing processing multiple requests for images per second. The fine focus feature, the small overview slide [6] and the possibility to use the WebMicroscope to view the whole slide images were all perceived favorable, and most students did not notice any delays during examination. More than 90% of responders found the WSI-based laboratory practicals and the practical exam useful and helpful to improve their understanding of Pathology of the Oral Cavity. At practical examinations, the students gave from 94% to 98% correct answers. So there is clear evidence of learning benefits from using WSI. Despite widely available self-study possibilities for the students, we were able to demonstrate differences between student groups. High and low scoring in student groups was associated with particular teaching assistants.

Conclusions
Despite all technical problems, the WebMicroscope system is a promising tool for practical student examination using WSI during laboratory practicals in the Pathology of the Oral Cavity. With all the people involved working hard to solve these technical problems and preparing every year exam, we are confident that testing using WSI is going to play an essential role in the future of the university education and examinations. Nevertheless, despite widely available self-study possibilities, good teachers still create a substantial value. Exam scores are helpful in identifying such teachers. This study also shows evidence that existing measures are informative about the teachers' impact on student scores.
Abbreviations used WSI: whole slide imaging