Tag Archives: Ofqual

GCSE Fiasco and Ofqual’s Second Report

First off, let me confess that I have not read in detail the whole of the 133 page Ofqual report from Thursday. I’ve scanned it. I’ve skimmed it. I’ve read reports about and summaries of it. I’ve been dipping in and out of it and I think I pretty much know what it says.

But it’s always possible I could have missed something or misinterpreted what is there. Please correct me if I’ve made mistakes.

As I understand it, in very broad summary, the report claims that the June grade boundaries were correct; that there was evidence of overmarking by teachers which led to the need to raise grade boundaries.

Let’s make a few observations which are entirely my own and personal.

1) The system was too complex for teachers.

These changes brought added complexity for schools as well. They had many more decisions to take: where previously they merely had to decide whether to enter each student for the foundation or higher tier, now they also had to decide whether to enter each student for English or for English Language and English Literature; when each student should sit units and in what order, and when and whether each student should re-sit any unit or units.

Now, I don’t know about other teachers or Heads of Department but how incredibly patronising is that? We can’t make decision about which pathway student should take? Or which tier to enter them at? I’m sorry, Glenys, but we are all professionals; we are all graduates for goodness’ sake! Of course we can make those decisions. What frustrates and baffles us is the changes that are being made to the specifications mid-stream!

2) The system – coupled with pressure to achieve A*-C in English – encouraged teachers to track progress.

As unit results accumulate for any one student, schools are able increasingly to predict the student’s likely final result, and aim for it. This makes them especially susceptible to pressures, as teachers strive for the best possible outcomes for their students and school.

What is wrong with striving for the best possible outcomes? Is that not what teaching is all about? This is in no way a scientific or researched response but I know a lot of teachers both in and out of my department and school and I know that each and every one of them strives for the best possible outcome for the kids. And it is definitely for the kids. Not SLT or the school or the governors… We did not come into teaching for any other reason.

To me, it beggars believe that a genuine desire to strive for success can be listed here as a danger inherent in the system.

3) Predictions

Schools evidently had a high degree of confidence in their predictions.

How – on what basis – can the provision of a prediction indicate the confidence schools had in them?

4) Overmarking

Ofqual stated that

From the records of awarding meetings, it is clear that examiners saw considerable evidence of over-marking by teachers.

Okay. I can accept that. It is a new specification. The mark scheme is very subjective. Without the years of experience which we had on the old specification, I can believe that there may have been some over-marking. And probably some under-marking.

Both Ofqual and exam boards accept this when the report states that:

New qualifications do lead to more variation in school-level results. Edexcel, in its report to us, notes that “it is not an unusual phenomenon for grade outcomes to fluctuate, sometimes quite markedly, from year to year, when a new specification is introduced”. OCR also found that there was “slightly more variability in English than in other subjects in year of specification change, but not considerably more”.

But this is what the process of moderation is designed to combat. Isn’t it?

For any non-teacher reading this – yes I still optimistically assume I am being read! – the marking process goes like this. The student writes their essay, teachers mark it with a numerical mark – and usually give an indication of the likely grade; within the Department a process of moderation takes place where each teacher’s marking is swapped around and double or triple checked. The exam boards, around May time, will then request a sample of Controlled Assessment folders and they then check the accuracy a sub-set of two thirds of the sample that is selected.

If a centre’s marks for the sub-set are within tolerance – which is set at +/- 6% – then the centre’s marks are accepted. If the centre’s sub-set marks are outside tolerance the whole sample is moderated and referred back to the exam board where marks can be altered or then accepted.

That means there are four levels of checking marking: at teacher, Department, external moderator and exam board levels.

If examiners found “considerable evidence of over-marking,” the process of moderation should have been alerted to it and altered the marks of those centres which displayed over-marking. Ofqual reported that “Moderation by exam boards did not prove strong enough to identify and counter problems effectively.” Well, if there are these problems, they should have! Surely! Or what is the point of those procedures? What on earth do they counter?

The report goes on and states that “The Starting with the controlled assessment units in the November 2012 re- sits, moderation will be tightened..

5) Raising Grade Boundaries.

The report, in the same paragraph, says that “As a result, the grade boundaries needed to be higher.”

I’m sorry. How does that work. If some schools over-mark – even if many or most schools were over-marking – to be honest, even if only one school did not over-mark – it cannot be fair to raise grade boundaries which is a universal reaction to a specific problem.

I am confident that my Department did not over-mark. We were very careful. And professional. We searched souls getting our marking as accurate as we could. We combed the mark schemes. Our marks were – according our moderators – within tolerance.

And our kids have been penalised as a result.

And that stinks!

6) Teachers cheated

I accept that I have put it much more strongly there than Ofqual did but the inference is clear from this paragraph.

Many students already had a written exam grade before June 2012. Schools could then calculate ‒ basing their calculation on assumed (January) controlled assessment grade boundaries ‒ the number of marks on the controlled assessment they assumed would give students a particular grade for the whole qualification. The patterns of controlled assessment marks suggest strongly that this assumption influenced the way that many teachers taught, or the way they administered or marked controlled assessment.

So what are you saying, Glenys? That, when the exam marks came back in March, teachers just added a couple of marks to Controlled Assessments for kids who hadn’t quite got the C?

Really?

Okay so maybe I can accept that there may be that pressure in some cases – yes we are all looking at getting as many A*-C as possible, especially in English. But that suggestion is blatant cheating. I cannot believe that more than a handful of centres would capitulate to that pressure.

And, again, if there are centres who have done that you simply cannot penalise the entire cohort by raising the boundaries for everyone.

There are – there must be – there should be – procedures for centres that infringe the rules like that. So, apply those to any of these centres where there is malpractice and leave the rest of us alone!

7) Tightening moderation

Ofqual has declared that “Starting with the controlled assessment units in the November 2012 re- sits, moderation will be tightened.”

Ok. To what extent? In fairness it probably wouldn’t change anything in practice as, I aim to mark accurately rather than accurately + 6%… but I like to know the structures and processes that I work with.

8) Grade-free results

As of January 2012,

no grades, or other information about aggregate performance, will be issued for January 2013 assessments (whether exam or controlled assessment) until June 2013 assessments have also been marked. Awards for both January and June assessments will be made at the same time.

So lets put ourselves into the shoes of a Year 11 who needs a B grade to pursue English Literature at A-Level. They have been entered for January entry for the Prose From Other Cultures exam. How are they meant to decide whether to accept the January exam results or re-sit?

Yes, we can indicate that if the mark were applied to previous mark schemes it may have been given a certain grade. But that is not the same as knowing you can bank a B grade. I know that, once we reach the specification level, the unit grade has limited relevance but, even so, that is the piece of information I would want to decide whether to resit.

For those of you who want to read the full report, this is the link.

And this will take you to the BBC report of the response to the Ofqual report by head teachers.

GCSE English Fiasco… some personal thoughts and developments

There are time when as a teacher your head spins. Times when you feel as if you’re paddling frantically just to keep still. On some occasions, however hard you paddle, however frenetically your feet beat against the current, you feel yourself slowly slipping backwards and the sound of the weir or waterfall at your back just gets louder and LOUDER and more and more insistent.

And these are the rhythms and ebb and flow of the school year. We all know the feeling – unless it’s just my utter inadequacy I’m confessing here! We could look at:

• the first weeks of September with a hundred new names to learn, results to analyse, new class dynamics to negotiate, new traumas to heal between students…

• those days when, unthinkingly, all your classes are handing in long homeworks at the same time – which I know careful planning and forethought could avoid!

• those lovely weeks when parents evenings pile up on options evenings and pathways evenings and twilight Inset…

• moderation and OMRs and estimated grades form to be completed. In triplicate.

• the end of the year when reports and grades and key stage levels and target grades and end of year exam results all need to be compiled.

This year, however, the GCSE English debacle has spun my head so far I feel like Linda Blair in The Exorcist.

20120915-104318.jpg

One of the best analogies I have heard about the “situation” – I’m struggling to find some way to describe any of this without using words like fiasco, farce or blunder or, to be frank, the phrase cock up – it would be almost funny this level of incompetence if 1000s of kids’ futures weren’t at stake! – was from Radio 4. Imagine an orchard full of lovely ripe apples. Imagine you and your friends go to pick apples on Saturday but find that another group of kids broke in the night before and have already taken most of those bright juicy sweet apples. There are therefore less for you and your friends who waited and most of you are having to content yourselves with the dry shrivelled old windfall apples.

However, if we stick with the analogy, if we were in Wales – and only in Wales – the Government will refresh those windfall apples for you. They will be plumped up, re-juiced, rejuvenated, buffed and bronzed. Which can only make the wizened old things that the English kids are left with as bitter as a crab apple!

20120915-105713.jpg

Now, I applaud Leighton Andrews for his stance ordering WJEC to regrade it’s GCSEs for Welsh students. Thousands of Welsh teenagers will now get the grade they deserve. Brilliant!

But this does not help my head! Not at all!

Children with the same raw scores in every piece of Controlled Assessment and in their examination – possibly living less than a mile apart – possibly best friends – possibly even related – on opposite sides of the Welsh border – get different grades?!

Was this what was envisaged when powers were devolved to Wales? Seriously?

Predictably, our Mr Gove has decried Leighton Andrews decision. Why? Because it points the brightest most penetrating spotlight possible on his own unfair refusal to do the same? Oh no, silly me. Because it was “irresponsible”.

Anecdotally, it seems this year that a number of post-16 providers may have been very understanding and accepted students with a lower-than-expected GCSE grade. I stress that I have absolutely no evidence beyond anecdote to support this. Some research into this might be useful…

But this, in my view, will not be the crucial year for these kids. Fast forward 18 months or so. This debacle will have inevitably slid from people’s consciousness. This same cohort of kids will be applying for University or employment. Their CVs will still have a D grade in English on. Universities and prospective employers will still be looking for a C grade as evidence of sufficient literacy to access their course or to carry out their responsibilities. And whatever leeway may have been offered this year is very unlikely to be applied then.

Geoff Barton and the ASCL seem to me to be doing a grand job on two fronts:

• both Geoff Barton and Brian Lightman are keeping this in the public eye, maintaining awareness and keeping up the pressure; and

• looking at the ASCL’s website, there are some genuinely imaginative and creative proposals being put forward to remedy the injustice.

Pushing for a relaxation of the 20th of September deadline for EARs – Enquiries About Results – seems to me to be a sensible move. This is a complicated situation for Heads, Senior Leadership Teams and teachers to ponder, let alone parents. There are, according to the JCQ rules, three routes to consider:

1) a clerical recheck in case there are errors in adding up or pages in the scripts that didn’t get read. Save for exceptional individuals, this is unlikely to help anyone. I know the fiasco has dented trust in exams but I am broadly confident that they can add up. It’s what they do to those marks I’m worried about!

2) a remarking of the examination script. A number of parents have already requested this and – in my school at least – without any changes being made. There is nothing in the results to suggest that the marking was incorrect. Again, it is the manipulation of those marks to create a UMS score and a grade that’s the problem;

3) an appeal against the moderation of Controlled Assessments. Now if a school’s CA marks have been reduced – just to compound the problems with the grading – this seems sensible. It might raises the couple of marks needed to cross the grade threshold. But my school – along with most schools because, let’s face it, teachers can mark! It’s our job! We do it well! – has had no change. We are within tolerance. To appeal against moderation which has accepted our marks is to appeal against our own marking! Who knows what effect that might have on the size of sample requested in future years and the relationship we’ve spent a decade building up with our Examination Board! And it seems dishonest.

ASCL are apparently insisting that boards relax the rules about appealing CA marking. Their argument seems to me that teachers were led to believe that a certain mark represented a grade and now that the boundaries are altered our marking has been invalidated. But, I don’t mark with grades. I mark with numbers. Demonstration of a certain set of skills equates to a number. If Ofqual permit marks to be changed to reflect a fair grade by divorcing them from the skills criteria, I’ll eat my hat!

If this is the fudge that lets kids get the grade they deserve whilst allowing Gove to remain firm about regrading, then so be it. It’s a practical solution. But it’s not marking. It would be – and there is no other word for it – a fudge and indicative of the chaos Gove and Ofqual have created.

I do have one problem with ASCL’s advice. It seems that they are inviting schools to submit vast numbers of scripts for remarking to mark our outrage. I’m sorry, Brian Lightman, but I don’t want to do that. I don’t want my kids given the false hope that this would generate. I don’t want any more political games played with my kids grades. Remarking can always result in a lowering of the mark and if that happened to one kid it would pile injustice upon injustice.

By all means, inundate exam boards with long detailed submissions about this fiasco, append reams of kids good quality Controlled Assessment that has been wrongly labelled a D, copy everything to Ofqual and your MP, sign the epetition but, please, no more games with kids’ futures!

Teacher Bashing

Related to, but separate from, the GCSE English row – according to Ofqual’s report 42% of all schools nationwide have reported results in excess of 10% worse than expected in June 2012 –

20120902-094402.jpg

there is a huge culture of what could be called ‘teacher bashing‘ in the media and Government focussing primarily around the following three characters:

Secretary of State for Education, Michael Gove

20120902-092318.jpg

Chief Inspector of Schools and Head of Ofsted, Michael Wilshaw

20120902-092554.jpg

and our Prime Minister, David Cameron.

20120902-093707.jpg

What a delightful Shakespearean Three Witches these could make! “Fair is foul and foul is fair“… Which brings us back around to the GCSE C grades again!

Certainly many leading teachers’ unions – notably the NUT – have observed and decried the media and Governmental rhetoric of failure and blame that it levelled at teachers. And it genuinely is hard to think of any other profession that is so frequently and so venomously derided in our national papers! And there is a nice piece in the Guardian here calling for Gove to resist this “declinist narrative”.

So I figured it might be interesting, informative and or depressing to just record as many examples as I can find over the next few weeks as we all return to school.

We can start just today with

1st September 2012, Telegraph “Bad teachers ‘blight children’s futures’ warns Gove” is reported here

And David Cameron’s comments in the Mail on Sunday today 2nd September 2012 here. The full text of which can be read here.

20120902-092714.jpg

And Michael Wilshaw waded into the GCSE uproar with this saying that

The teaching’s often not good enough, the leadership often isn’t good enough, and the progress and the outcomes of children aren’t good enough

Click here for a video of his performance on The Andrew Marr Show.

I don’t imagine I’ll be able to record or link to every example of coverage in the media that denigrates, undermines or devalues the profession but I’ll try to keep an eye on things. Any links that you guys want to point out that I may miss, please comment and point me in the right direction!

Ofqual’s report on GCSE English Results (and some interesting links)

Somewhat late in the day on Friday 31st of August – its tardiness (due originally at 10:30 and delivered at about 4:00) prompting an awful lot of comments on Twitter about missed deadlines and handing in homework late – Ofqual reported on the GCSE English and Language grading row.

Their finding was that there was nothing wrong with the June grading and that they will stand: the January gradings were overly generous. Indeed Glenys Stacey, Chief Executive of Ofqual, in a BBC interview here describes students on January as having had a lucky break. What an appalling way of describing the hard work and effort our students put into their work!

The key findings are reported on the Ofqual News Page here

And for those with the nerve, the full report is here

These are, perhaps, the most salient parts of the report:

“People were particularly concerned about the June grade boundaries. We have found that examiners acted properly, and set the boundaries using their best professional judgement, taking into account all of the evidence available to them. The June boundaries have been properly set, and candidates’ work properly graded.

“The issue is not the June, but the January boundaries. Again, examiners used their best judgement in setting these boundaries, but they had less data and information to work with. Most candidates were not sitting at the time, they were waiting for June, and because they were new qualifications, examiners could not rely so much on direct comparisons with the past. As a result, those grade boundaries were set generously.

“We have thought carefully about what should be done, and spoken with external assessment experts about it. Our job is to maintain standards over time, so grades awarded are comparable from one year to the next.

“We have spoken to exam boards and they have been very responsive. Recognising the strength of feeling, they will be offering early resits for students who sat the June units. We will now go through our analysis and evidence with the representative groups for schools and colleges, so they can see it for themselves. We will also talk with schools, exam boards and assessment experts to see what lessons can be learnt and what can be done better in the future”

What this means is that the June grades will stand: the verdict is that they were the correct grades.

Now, it seems to me that this is not quite the independent review that the NAHT had demanded and there are going to be an awful lot of people who took exams in January who are now going to feel very undermined and undervalued, even though Ofqual have very generously said that they will not be downgrading the results from January.

Ofqual are saying that only 7% of candidates submitted in January and therefore there were too few candidates from which to properly grade and level work. Yet they still did it. It seems to me that if 7% is too few to accurately grade a qualification, it ought not to be offered.

Perhaps a mimimum uptake should have been put in place and unless sufficient numbers of students are submitted, the entry is denied. If one candidate alone had attempted to submit their GCSE in January, would they have been permitted to? Is it an inalienable right to take a qualification in circumstances where there is no way of accurately grading you?

7% of the GCSE entrants is also, however, quite a large number. One wonders as to the mechanics of this: when does the figure become sufficient? At 10% entry? At 15%? At 25%? I must confess I have not read the full report and those questions may be answered within it…

Ofqual says also that

Understandably, schools were over-reliant on the January 2012 boundaries to set expectations as there was little other information available to them.

Again, certainly for OCR this is not entirely accurate: raw mark grade boundaries are available for 3 cycles prior to June 2012 and they are all broadly consistent with the January 2012 grade boundaries. This blog post includes the grade boundaries and their changes from January 2011 to June 2012.

However, if we accept the point that there were insufficient submissions in January 2012 to accurately assess grades, there must have been even fewer in the first two cycles in 2011 when the course was only 6 and 12 months old respectively.

A big question that could be asked is: to what extent did examination boards warn centres that the January 2012 (or previous) raw mark grade boundaries could not and should not be relied upon? Personally, I know that I telephoned the exam board for clarification on grade boundaries and – in a telephone call I do specifically recall, followed up by an email link to the raw score grade boundaries – was told to refer to the Jan and June 2011 grade boundaries.

Also, are we to accept that Ofqual and the examination boards had no idea that this was likely to be a problem? Reports in the media suggest that this situation was anticipated three years before.

Huffington Post

TES

What was done to warn and alert centres? According to Ofqual’s news page

Exam boards will review the advice and guidance they give to schools about GCSE English including its structure, how grade boundaries are set and how they should be used.

which to me suggests and implies that the advice previously given has been minimal and / or unreliable and / or misleading.

So, putting ourselves into the position of a young man who has believed for two years that he has been working at a C grade only to be rewarded with a D, what have Ofqual offered him?

1. the opportunity to withdraw an application for remarking at no cost; and

2. the chance to resit his examinations in November 2012.

If this hypothetical young man has lost a place on a sixth form course or an apprenticeship or a college placement or employment, one wonders what sort of help this two-month early retake will be if his place has already been filled by someone else.

I also wonder what the sixth form schools and colleges’ stance is: are they adhering strictly to their original offers that expected a C-grade in English? Or are they using a wider discretion to admit students who have a D-grade?

Whilst Ofqual’s report has clarified some issues, I think there are still a range of concerns and questions that still need to be addressed and I wonder how long it will be before parents and / or unions attempt to make a legal challenge.

A response from ASCL is repeated on TES here and Steven Twigg, Shadow Education Secretary here. This is The Independent’s report. And Geoff Barton blogged about the situation on his always thoughtful blog here.

It is now looking as if a Parliamentary Inquiry is likely, according to The Guardian which claims it

looks all but inevitable

that Michael Gove will be summoned to give evidence.

This is, despite the headline, a very interesting article in the second half, showing an examiner’s point of view.

Somewhat uninspiringly – though perhaps predictably – out learned Prime Minister’s response is to “brush aside” the concerns of parents, students and teachers

saying he planned further moves to reverse ‘dumbing down’ and that he refused to ‘cave in’ to teaching unions who want to ‘pretend standards are rising each year’.

He added: ‘ “All must have prizes” is not just patronising, it is cruel – and with us it is over.’

Mr Cameron said there would be ‘no more excuses for failure in schools; no more soft exams and soft discipline’.

How terribly disappointing to our Prime Minister so completely failing to see the depth of concern and anger that this has generated, failing to understand the actual issue and continuing to undermine a profession which has nothing but the best interests of children at its heart.

20120902-064114.jpg

20120902-064120.jpg