Exam Results

[aaid]

1 minute ago, duncan II said:

It doesn’t affect me at all in the slightest. But I recognise this is wrong. They have asked the teachers to make decisions. How about they respect these decisions? If not, how about they just do away with exams for this year? It is an unprecedented time. We’re in a position to have exams now. I understand this is no use for those seeking to enter university this year, but could these universities not hold entrance exams for those on the cusp of having correct grades, or who would be expected to attain the grade should there have been proper school exams? Thinking as I type and not explaining it well (!) but there are ways around this. Universities etc have to show leeway. Just bin the whole thing. Face it that due to unprecedented times, there are no school results. But gtf with results based on affluence. Especially ones arrived at by patronisingly telling teachers of less-affluent kids that their assessments of these commoners is wrong. It’s patently wrong. 

Organise either replacement exams or university exams at short notice after schooling has been paused for four months - exactly which group of students do you think are going to be disproportionately disadvantaged by that approach?

Just go with the recommendations of teachers, fine, but how do you deal with the fact that throws out of kilter selection processes for further education and employment.

Scrap a whole year of schooling and essentially send everyone back a year - seriously?

Knock down everyone across the board to get to the required threshold.  How do you deal with the situation that a school who's assessment submissions are in line with there previous exam results being disadvantaged because another school has bumped up their assessments.   

There is no fair outcome to this or at least I can't see one.

[aaid]

11 minutes ago, TDYER63 said:

My daughter is a teacher, she teaches in a school with a decent reputation though not of Jordanhill standard. 
I asked how her pupils results went. She said that overall they were good as all her pupils passed however she was extremely disappointed that just about every one of her Nat5 and Higher estimations were changed.
She  worked her butt off doing these and had prepared evidence as to why she graded a pupil out with the level they had previously been graded at. 
For the past few years  she has been marking the Higher and Nat5 official exams so knows what the standard is and the type of things that the SQA are looking for. 

She said a lot of the changes they made to the grades were within the same band, ie high B to low B, but  that some of her N5 pupils have been moved to a B when they thoroughly deserved an A .
More concerning they changed some Higher pupils from a C to a D, and that that although the SQA argued that a D is still a pass, colleges etc dont see it that way. 

Her words were: 

‘ It feels like such a slap in the face to us teachers who spent hours making sure our estimations were robust. I feel like they don’t trust our professional judgement and it’s really overshadowed my overall happiness with the results. I think they’ve looked at those Higher pupils’ N5 result last year, and said “well they can’t have gone from a N5 D to a Higher C”, but it’s because the poor souls have worked their socks off this year. Plus I’ve put in more work with them to help them get there.’ 
 

I dont know what the answer is and sympathise with anyone having to make such a decision in a relatively short period of time, but surely you need to put trust in the teachers judgement. We are putting trust in every other ‘expert’ at this time.
I realise the SQA have a job to do in overseeing things but children can change and you cant just base  the results on past experience of the school. What was the point of all the work by the teachers ? Under normal conditions, the teachers who are marking exams get ‘spot checked’ to make sure there is consistency ,  why did they not just do the same with this years gradings systems? 

 My other daughter got a D in her English Higher prelim a number of years ago and ended up with an A in the actual exam because of the extra work she put in . She would never have received an A if she was doing her higher this year.  

 

You can absolutely understand any individual teacher thinking that way and it's also worth reminding people that 75% of the assessments were accepted without change.

However, you can't shy away from the fact that across the country, the assessments have been significantly in excess of previous years actual results, so there's a problem somewhere.

I don't for a second believe that a big enough number of teachers are "at it", there would be the odd one or two for sure but not to make the scale of difference there would've been had there not been moderation but clearly there was some need for some moderation to take place and an adjustment required.

I've yet to see an explanation for that delta.

[Orraloon]

It's clear that schools and teachers have overestimated what results their pupils were likely to achieve. There isn't going to be a nationwide jump of 15% in one year. It just doesn't happen. It could happen in individual schools but not overall nationwide. 

What the SQA need to explain is why different schools have been downgraded by different amounts. Are they really saying that schools and teachers in disadvantaged areas are more likely to overestimate their pupils abilities than more advantaged schools? Are they saying that teachers in better schools are better at judging the ability of their pupils than teachers in poorer schools? If they have data to support that theory, then they should publish it.

 

[aaid]

6 minutes ago, Orraloon said:

What the SQA need to explain is why different schools have been downgraded by different amounts. Are they really saying that schools and teachers in disadvantaged areas are more likely to overestimate their pupils abilities than more advantaged schools? Are they saying that teachers in better schools are better at judging the ability of their pupils than teachers in poorer schools? If they have data to support that theory, then they should publish it.

 

That's the key thing to me and I think you're right they need to publish that data although anonymised as individual schools being named doesn't help anyone.

There's a couple of explanations as to how the adjustments may have been made, I'm being simplistic, I appreciate that the actual calculations are a lot more complex - for example, I think they look at schools, subjects and then actual exams, not just the school in its entirety - than this and I'm making the numbers up the example.

Presumably they have some sort of target "national pass rate" that they're aiming for and lets say that's 69%, which is what it came out as.

They do a first run through at the assessments and the national pass rate comes out as 85%, which is way too high and so needs to be moderated.

The first pass through is looking at those schools whose predicted results have generally come close to their actual exam results.   In those cases you give the schools the benefit of the doubt that their assessments will be accurate.  That's the 75% that are just accepted.   I suspect that - historically and for reasons that are pretty obvious - the "more successful" schools are probably more accurate in their assessments compared to the actual exams than others and so I imagine that they have a fairly high number of "pass throughs" which then feeds into the lower number of downgrades in that sector.

That leaves you with the 25% you need to look at in more detail and those are the ones that will be getting a hair cut.   

If, for example, school A is predicting a pass rate of 75% but their "actual" success rate in exams is 70% then you give them a haircut of 5%, which will inevitably result in some people dropping down a band.   Similarly school B might be predicting a pass rate of 65% but their "actual" rate is 50%, so they need to get a haircut of 15% which might mean people dropping down several grades.   Of course, the level of haircut might need to be a lot more severe to get to the correct national number.

The other alternative is to go back to the people who made the assessments and get them to do them again as they're out of line with previous results - I can imagine how that went down.

Actually reminding me a lot of having to do salary planning for a department and the total bun fight that is.

[Rolling hIlls]

33 minutes ago, aaid said:

That's the key thing to me and I think you're right they need to publish that data although anonymised as individual schools being named doesn't help anyone.

There's a couple of explanations as to how the adjustments may have been made, I'm being simplistic, I appreciate that the actual calculations are a lot more complex - for example, I think they look at schools, subjects and then actual exams, not just the school in its entirety - than this and I'm making the numbers up the example.

Presumably they have some sort of target "national pass rate" that they're aiming for and lets say that's 69%, which is what it came out as.

They do a first run through at the assessments and the national pass rate comes out as 85%, which is way too high and so needs to be moderated.

The first pass through is looking at those schools whose predicted results have generally come close to their actual exam results.   In those cases you give the schools the benefit of the doubt that their assessments will be accurate.  That's the 75% that are just accepted.   I suspect that - historically and for reasons that are pretty obvious - the "more successful" schools are probably more accurate in their assessments compared to the actual exams than others and so I imagine that they have a fairly high number of "pass throughs" which then feeds into the lower number of downgrades in that sector.

That leaves you with the 25% you need to look at in more detail and those are the ones that will be getting a hair cut.   

If, for example, school A is predicting a pass rate of 75% but their "actual" success rate in exams is 70% then you give them a haircut of 5%, which will inevitably result in some people dropping down a band.   Similarly school B might be predicting a pass rate of 65% but their "actual" rate is 50%, so they need to get a haircut of 15% which might mean people dropping down several grades.   Of course, the level of haircut might need to be a lot more severe to get to the correct national number.

The other alternative is to go back to the people who made the assessments and get them to do them again as they're out of line with previous results - I can imagine how that went down.

Actually reminding me a lot of having to do salary planning for a department and the total bun fight that is.

They should have trusted the teachers.

[TDYER63]

1 hour ago, aaid said:

You can absolutely understand any individual teacher thinking that way and it's also worth reminding people that 75% of the assessments were accepted without change.

However, you can't shy away from the fact that across the country, the assessments have been significantly in excess of previous years actual results, so there's a problem somewhere.

I don't for a second believe that a big enough number of teachers are "at it", there would be the odd one or two for sure but not to make the scale of difference there would've been had there not been moderation but clearly there was some need for some moderation to take place and an adjustment required.

I've yet to see an explanation for that delta.

Yeah I agree the assessments have been high and there will be a few at it, but unless the gradings  my daughter has given her pupils were spectacularly wrong,  a 75% figure for ‘no changes’ also seems high . She said most of her gradings were changed and she is very thorough. I am not just saying that as her mother.

I think I would be pissed off if my pupils were all downgraded just to reach the desired historical level for the school. 

[Rolling hIlls]

7 minutes ago, TDYER63 said:

Yeah I agree the assessments have been high and there will be a few at it, but unless the gradings  my daughter has given her pupils were spectacularly wrong,  a 75% figure for ‘no changes’ also seems high . She said most of her gradings were changed and she is very thorough. I am not just saying that as her mother.

I think I would be pissed off if my pupils were all downgraded just to reach the desired historical level for the school. 

125000 downgraded according to reports.  Disgraceful.

[Orraloon]

2 hours ago, TDYER63 said:

She  worked her butt off doing these and had prepared evidence as to why she graded a pupil out with the level they had previously been graded at. 



 

 

So, she worked really hard to get a better grade for her pupil. Sounds like she wasn't the only one.

[TDYER63]

10 minutes ago, Orraloon said:

So, she worked really hard to get a better grade for her pupil. Sounds like she wasn't the only one.

🙂 aye very funny. 

 
 

[exile]

8 hours ago, aaid said:

Anything would be a bad look politically in this case.

As an example.  I was listening to John Beattie interviewing John Swinney yesterday.   I get that he's playing devil's advocate to an extent so don't have a problem with the line of questioning.

He started off by saying that the fact that kids from deprived areas had been marked down disproportionately, so they were being screwed.  Then he finished by saying. this is all arbitrary so you can't claim that the attainment gap is closing.

These are of course mutually exclusive.

Yes, I get that whatever the results had been, the opposition would put the boot in.

But it seems something has gone wrong somewhere, more than usual, if only in the messaging. Even the National had a sub-headline about exam results being downgraded - implying that the students had achieved something and that someone had taken that away (rather than it being the teachers' more extreme estimates being moderated).

It seems as if the deprivation issue was always going to be an issue and maybe they could have ensured that this was not disproportionately going to affect those from more deprived areas.

As I say, the problem is when particular individuals/groups suffer disproportinately, unfairly, it is catnip to the media, more so even than individual tragic hospital cases, because in this case the 'injury' is seen to have been deliberately built into the system.

[phart]

Gifted pupils at bad schools got fucked and marked down, while mediocre students at good schools got to keep their grades. It's a policy straight out of the Tory playbook. What school you go to shouldn't affect the result by a difference of 2.5.

Downgrading individually historical A students to C students (2 bands) cause of the historical results of their schools to fit an anachronistic bell curve model who shouldn't have happened.

Anyone defending the outcome when it is so patently unfair and discriminatory cause of who is defending it or is responsible for defending it is part of the larger Political problem, like Alan they'll defend their party and decry the other parties no matter the problem. Rank partisanship is the major problem in politics with folk wiling to eat shite depending on who's arse it came out of.

[aaid]

5 minutes ago, phart said:

Gifted pupils at bad schools got fucked and marked down, while mediocre students at good schools got to keep their grades. It's a policy straight out of the Tory playbook. What school you go to shouldn't affect the result by a difference of 2.5.

Downgrading individually historical A students to C students (2 bands) cause of the historical results of their schools to fit an anachronistic bell curve model who shouldn't have happened.

Anyone defending the outcome when it is so patently unfair and discriminatory cause of who is defending it or is responsible for defending it is part of the larger Political problem, like Alan they'll defend their party and decry the other parties no matter the problem. Rank partisanship is the major problem in politics with folk wiling to eat shite depending on who's arse it came out of.

So what would you have done differently then?

[thplinth]

Voted for someone else.

[phart]

If the entirety of ones defence is, tell me what you would have done differently then it isn't a defence.

Why would I waste my time typing out a holistic policy paper on the best practices for assigning grades, then once it was typed out spending hours defending it to political partisans on a message board who think if If i can't produce a better system that in some way means that the current system is absolved?

It's madness.

As i said before we can point out the absolute disaster of the Mars Climate Orbiter crash without being able to do the calculations ourselves. We can criticise the Iwelumo miss without being able to tuck it away ourselves.

The main problem is the SNP leadership don't see this 2.5 times issue as a problem so therefore it won't be fixed. A random on the TAMB not typing out the fix to it is irrelevant to this.

I've pointed this out multiple times now i'm not going round the houses on it again.

Edited August 5, 2020 by phart

[phart]

I posted a long thread about the shortcomings by an informed individual with possible solutions and no discussion of the content was had, instead it's oh I went to the same school as him. That's the level of the debate, it's laughable.

[thplinth]

5 hours ago, aaid said:

Playing the man and not the ball is a pretty common tactic when someone is losing the argument. 

Oh. My. God.

Coming from 'aaid hominem'.

[Rolling hIlls]

14 minutes ago, phart said:

If the entirety of ones defence is, tell me what you would have done differently then it isn't a defence.

Why would I waste my time typing out a holistic policy paper on the best practices for assigning grades, then once it was typed out spending hours defending it to political partisans on a message board who think if If i can't produce a better system that in some way means that the current system is absolved?

It's madness.

As i said before we can point out the absolute disaster of the Mars Climate Orbiter crash without being able to do the calculations ourselves. We can criticise the Iwelumo miss without being able to tuck it away ourselves.

The main problem is the SNP leadership don't see this 2.5 times issue as a problem so therefore it won't be fixed. A random on the TAMB not typing out the fix to it is irrelevant to this.

I've pointed this out multiple times now i'm not going round the houses on it again.

SQA should have trusted the teachers and schools in these difficult times.  Not just look at statistics from the past 3 years.   I have spoken to teachers today who are dismayed and having to answer parents why their kid was downgraded.  Absolute disgrace.

[aaid]

26 minutes ago, phart said:

I posted a long thread about the shortcomings by an informed individual with possible solutions and no discussion of the content was had, instead it's oh I went to the same school as him. That's the level of the debate, it's laughable.

Nope he was a teacher there, not a pupil.

Sorry but just posting a tweet that starts a thread with some suggestions doesn't constitute you explaining what you would do differently.

I would however take notice of him as he has a reputation that I am familiar with and people who I respect think highly of him.

 

[Kirk]

See at the end of the day, all this they should have got an A stuff is nonsense, the only way to guarantee it is to do the exam. I didnt give a fuck in school and got a few Cs, failed a couple of exams that I got As and Bs in prelims and didnt get appeals.

People perform different on the day. 

If you didnt get the grades and dont get them on appeal then you'll need to do a year of college or whatever to get what you need.

 

The system does seem a bit daft but the worlds fucked at the moment get on with it.

[Squirrelhumper]

15 hours ago, TDYER63 said:

My daughter is a teacher, she teaches in a school with a decent reputation though not of Jordanhill standard. 
I asked how her pupils results went. She said that overall they were good as all her pupils passed however she was extremely disappointed that just about every one of her Nat5 and Higher estimations were changed.
She  worked her butt off doing these and had prepared evidence as to why she graded a pupil out with the level they had previously been graded at. 
For the past few years  she has been marking the Higher and Nat5 official exams so knows what the standard is and the type of things that the SQA are looking for. 

She said a lot of the changes they made to the grades were within the same band, ie high B to low B, but  that some of her N5 pupils have been moved to a B when they thoroughly deserved an A .
More concerning they changed some Higher pupils from a C to a D, and that that although the SQA argued that a D is still a pass, colleges etc dont see it that way. 

Her words were: 

‘ It feels like such a slap in the face to us teachers who spent hours making sure our estimations were robust. I feel like they don’t trust our professional judgement and it’s really overshadowed my overall happiness with the results. I think they’ve looked at those Higher pupils’ N5 result last year, and said “well they can’t have gone from a N5 D to a Higher C”, but it’s because the poor souls have worked their socks off this year. Plus I’ve put in more work with them to help them get there.’ 
 

I dont know what the answer is and sympathise with anyone having to make such a decision in a relatively short period of time, but surely you need to put trust in the teachers judgement. We are putting trust in every other ‘expert’ at this time.
I realise the SQA have a job to do in overseeing things but children can change and you cant just base  the results on past experience of the school. What was the point of all the work by the teachers ? Under normal conditions, the teachers who are marking exams get ‘spot checked’ to make sure there is consistency ,  why did they not just do the same with this years gradings systems? 

 My other daughter got a D in her English Higher prelim a number of years ago and ended up with an A in the actual exam because of the extra work she put in . She would never have received an A if she was doing her higher this year.  

 

Pretty much what  my brother and sister have told me.

My main worry, after the kids affected of course, is the negative impact this will have in next years elections.

A lot of kids/parents from SNP heartlands who are going to massively pissed off and aggrieved just now.

[ParisInAKilt]

11 hours ago, phart said:

If the entirety of ones defence is, tell me what you would have done differently then it isn't a defence.

Why would I waste my time typing out a holistic policy paper on the best practices for assigning grades, then once it was typed out spending hours defending it to political partisans on a message board who think if If i can't produce a better system that in some way means that the current system is absolved?

It's madness.

As i said before we can point out the absolute disaster of the Mars Climate Orbiter crash without being able to do the calculations ourselves. We can criticise the Iwelumo miss without being able to tuck it away ourselves.

The main problem is the SNP leadership don't see this 2.5 times issue as a problem so therefore it won't be fixed. A random on the TAMB not typing out the fix to it is irrelevant to this.

I've pointed this out multiple times now i'm not going round the houses on it again.

That does seem the key point, there’s little to no political will to change this. 

[aaid]

I'm surprised* this tweet by Labour's deputy leader hasn't been picked up by the media.  I can imagine the furore if Kirsten Oswald said similar about the Johnson government. 

 

[TDYER63]

1 hour ago, Squirrelhumper said:

Pretty much what  my brother and sister have told me.

My main worry, after the kids affected of course, is the negative impact this will have in next years elections.

A lot of kids/parents from SNP heartlands who are going to massively pissed off and aggrieved just now.

 

The one thing I would say that is perhaps being exaggerated , and I very much could have picked this up wrong, is the issue about the Higher results in deprived areas being downgraded more.
The pass mark in deprived areas moved from 65% to 85 % which does not look right at all, much as the SG would love to see that sort of improvement. Could you imagine the media if they had not downgraded the results to realistic levels. They would be going fooking mental. 
It would be ‘Sturgeon using Covid to manipulate exam results favourably ’ . 

All we ever  hear is  how shit Education is under SNP by their critics, and now then we hearing how shit they are by downgrading results in deprived areas to more realistic levels. 
Criticise them by all means for the general issue in  the attainment gap between deprived and well off , but I dont really think they could win here. 

The problem is this  whole exam process has been flawed and in turn they have managed to  undermine teachers and disappoint a large number of pupils. There needs to be an investigation into the gap between the teachers marking and the SQA. 
There will be loads of appeals and the SQA will be overwhelmed. They will probably end up hiring teachers to help out 🙄

Politically it isnt good but if a headline from yesterdays Telegraph is anything to go by England is going to be much worse. Whataboutery I know,  but no one seems to be coming out of it well. 

 https://www.google.co.uk/amp/s/www.telegraph.co.uk/news/2020/08/05/scotlands-exam-result-fiasco-coming-england/amp/

 

[Orraloon]

4 minutes ago, TDYER63 said:

 

There will be loads of appeals and the SQA will be overwhelmed. They will probably end up hiring teachers to help out 🙄

 

 

In normal years it is mainly teachers who mark the exams. It's a reasonably lucrative wee sideline for teachers who want to put in a bit of extra work and earn some extra money. It will be those same folk who are dealing with the appeals. Only they will be dealing with the appeals during term time instead of marking papers in their holidays.

If your daughter fancies earning some extra money she should think about applying for next year. Although I think there is a decent chance we will be going through the same process again next year.

[phart]

4 minutes ago, Orraloon said:

In normal years it is mainly teachers who mark the exams. It's a reasonably lucrative wee sideline for teachers who want to put in a bit of extra work and earn some extra money. It will be those same folk who are dealing with the appeals. Only they will be dealing with the appeals during term time instead of marking papers in their holidays.

 

Lets' fact check the assertion above

Lucrative? Let's open the dictionary.

adjective

adjective: lucrative

1. producing a great deal of profit.

    

   SCOTLAND'S national exams body is under the spotlight over whether it is paying teachers less than both the minimum and living wages for specialist marking duties.

   For assessing coursework in their own time, teachers are receiving a fee that is as low as £3.56 per paper from the Scottish Qualifications Authority (SQA). Some papers take an hour to mark.

   https://www.heraldscotland.com/news/15182278.exams-board-faces-questions-over-whether-teacher-payments-meets-living-wage-level/