And check each student against the rubric even though I am quite familiar with the different levels. It is easy to drift in scoring. And even though the IELTS rubric has ten levels from zero to highly proficient, I have found that 95% of my students are within two levels. I have had to subdivide each of two levels into four levels, making eight subdivided levels, and after years of experience I’m pretty accurate at scoring within those subdivided levels.
2. Record all interviews.
Before I used to use a cassette tape recorder but now I use my computer and the free program Audacity. I spot check my scoring later. After the test and without looking at the score I just gave a student, I rescore their interview and then I check against the score I gave to see if the scores are the same. There are some students that I taught for many years. I can go back and actually hear the progress term by term, year by year. This also allows an objective non-subjective comparison. Although much of oral testing is subjective in scoring, an actual comparison between one year and another can clearly show progress or lack of progress to anyone. Additionally this provides an audit trail. If I am challenged on a score by a student or a school administrator or the education ministry (which I never have) we can go back over the recording to see how the student actually did. No one has to take my word for it.
3. Compare scores but after testing.
When these are students that I have tested before at the end of the previous term, I make it a point when I am testing them to not check their old score and make sure that I have no idea what their old score was. Although I write each score into an Excel spreadsheet, I don’t want the spreadsheet to even tell me what the current class average is when I’m scoring. I want to score each student very individually without any influence from their old scores or even average class scores. After I am finished scoring all of the students, I will add their old score into my Excel file and the file will show me the differences between old score and new score. If the scoring shows a curious difference (ie: lower score this time than last time or a jump in improvement) I can go back to my recordings this time and last time to see what is happening. Sometimes a students’ performance in a test varies. Sometimes my scoring may be off. If I find a lot of jagged scores then I will check more of the recordings or even all of them to rescore. (I haven’t had to do this in a long time.) I also score the students’ speaking in our day to day lessons, whether it be a role play performed for the class or me eavesdropping on their pair-work activity or even them chatting with me after class about a book they read. These are all speaking samples. The exam counts the most in their overall score but these other speaking samples are a small part of their score. These day by day scores are entered into my Excel spreadsheet and the spreadsheet alerts me if there is a big variance from one score I gave and another. The average of these day by day scores are also matched to the students’ final oral exam scores to see if there is an inordinate amount of variance.
[Photo: The set-up. Two students ready to be tested. Computer ready with Audacity for recording and Excel displaying the oral band descriptors and for marking their scores. Photo by Dave]