I did my higher education at two “institutes of national importance”. Both institutions followed what is called “relative grading”. It didn’t matter on an absolute scale how well or how badly you did. Your grade for the course would depend on how everyone else who took the course did. So for example, there was this one course at IIT Madras where I got 80/100, and got an S grade (the highest grade possible). The general performance of the class had not been great, so in that course 80 merited an S. In another course, however, 80 fetched me only a B (the third highest grade) – the general performance of the class had been much better.
While IITs and IIMs and some other autonomous institutions practice relative grading, it is not the “done thing” in most of the rest of India. Most of our board and university exams follow what is known as “absolute grading” – your grade for the course depends solely on your performance, without taking into account the performance of others. So it is theoretically possible to have a case where practically everyone in the class scores “90%”. Given that this is the prevailing system of grading in most of India, we assume that the board exams follow this principle, too.
Two or three days back, Debarghya Das, a student at Cornell set a cat among the pigeons by scraping the marks of every single student who took the ICSE or ISC exams (10th and 12th board respectively administered by the CICSE). What he noticed was that certain marks had gone missing – for example nobody scored 81, 82, 84, 85, 87, 89, 91 or 93 in any of the courses. This is just a sample of marks that have gone missing. There are several other numbers which are effectively “unattainable” in any of the courses. Das, on his account, has alleged some kind of “fraud”.
What is the first thing that comes to your mind when you see this rather jagged distribution? I wouldn’t blame you if you saw a hedgehog. But can you think of a graph that looks like that?
Three years back I bought myself a DSLR camera, after which I pretend to be an expert photographer. I even use Photoshop/Gimp to manipulate some of the images I click. And a decidedly much better photographer friend has told me that the first thing you do while editing a photo is to adjust “levels”. See this to know what you can do with levels. Basically, the concept is that some parts of the colour spectrum are unrepresented in an image, and by adjusting levels you make sure the full spectrum is used, thus improving the contrast of the image.
There is something known as the image histogram. I took a picture that I had shot and adjusted the “levels”. On the left you see what the histogram looks like after the levels. On the right, you see the histogram as it was before you adjusted the levels.
Doesn’t the histogram on the left remind you of the distribution of ICSE/ISC marks? And how did we get that histogram? By taking the histogram on the right (which is smoothed but all bunched up in one part of the distribution) and stretching it so that it falls across the entire distribution. And what happened when we did that? We got gaps, as you can see in the histogram on the left or the distribution of ICSE/ISC marks.
There is an article in The Hindu today that again explores this issue of missing marks in ICSE/ISC. In that the ICSE council, which administers these exams is quoted saying:
“In keeping with the practice followed by examination conducting bodies, a process of standardisation is applied to the results, so as to take into account the variations in difficulty level of questions over the years (which may occur despite applying various norms and yardsticks), as well as the marginal variations in evaluation of answer scripts by hundreds of examiners (inter-examiner variability), for each subject.”
Another money quote from the same article:
“The word tampering is wrong. There is moderation that happens across education boards,” explained a teacher, who has worked with ICSE schools in Hyderabad and Chennai. “After the first round of corrections, raw data is given to officials and head examiners who analyse how students have performed. They try to ensure the bell curve of the results does not look awkward. If it does, the implication is that the checking has been either too liberal or very strict.”
So there you go. The ICSE Council effectively follows relative grading. There is a certain distribution of marks that they desire, and they adjust the “levels” of the overall distribution of marks so that the desired distribution is achieved. The desired distribution of marks is something like “X% students get between 95 and 100, Y% get between 90 and 95”, and so on. Now, two students who had got the same number of marks as per the initial marking have to get the same number of marks after recalibration. So what the missing marks indicates is that there was clustering – a large number of students had ended up scoring in the same narrow range, and so after normalization, this range got expanded because of which you have gaps. Now, when certain sections of the range in the middle are expanded, some at the end have to get contracted (for example, if someone who originally got 70 is given 90, a person who originally got 90 deserves so much more). Which is why you see that at one end – 94-100 all possible marks are represented.
This still doesn’t explain one thing though – why is it that the same marks have gone missing in all subjects? It is impossible that the initial distribution of marks was identical across subjects. I have only one explanation for this – there was one overall mapping algorithm that was used across subjects, that converted marks obtained to the relative marks. This is also seen in the fact that the shape of the distribution across subjects varies widely (again refer to Das’s post).
So that explains the weird distribution of marks in the ICSE / ISC exams. But what explains the title of the post? In IITian English, “RG” is a term derived from “relative grading”. It is a rather derogatory term used to describe people who prefer to pull down others in their quest to get ahead (note that this is a consequence of relative grading). Taking some more liberties and using IITian English, you can say that the ICSE/ISC board has “RGed” students!
What could the mapping algorithm be, and why does it end up with gaps exactly at the same point?
Taking the gimp analogy, if I take two different photographs, and try curve adjustments to make them use the full range of Y, I will end up with gaps in the curve both cases, but not in the same places. Where the gaps are and how wide they are depends on how tight my initial distribution is.
Analogically, if the initial distribution of marks in (say) Hindi and Science are different, which they most likely are, an RG type transformation should not end up with gaps in exactly the same places, should it?
i address this in the last para of the post. basically all subjects have used the same mapping. For example, suppose i scored 80 in Hindi and the algorithm bumped it up to 87. If you scored 80 in Math, your score in Math would be bumped up to 87, too! Irrespective of the number of people above me in Hindi and above you in Math
Following the same system for all subjects does not make any sense. If we’re trying to normalize the marks curve based on the relative performance of students, it should never be done on an overall basis. That will hold true only if all students were equally good at all subjects, which is clearly not true. Some are better in Maths and some, in Hindi.
Also, your methodology speaks more about “Covering the entire spectrum” rather than doing RG – something, I personally feel, is not needed in ICSC/ISC given the huge number of students enrolled. The levels analogy only illustrates the aforementioned point.
Great post, Karthik. I think this does help explain many of the gaps if the normalization process was the same for all subjects. Also maybe this is weird but most of the lost numbers are odd so there could be some 2 multiple involved here.
Also I think they don’t normalize the marks below, say, 40. That would negate any effort (valid, I feel) to pass someone who got 34 and the teacher upped it by 1mark…
This did strike me at one point of time however I’d like to point out one thing.
Exactly the same marks were missing last year as well in both class 10 and class 12, for every subject.
And in fact *exactly* the same marks have been missing for fifteen years if not more.
I appeared for the ICSE in 1999 and took one look at the result displayed on the school board and this was the first thing I noticed while scanning through the results, no one else seemed to find it strange.
Also I tried to smoothen out the missing scores and still ended up with extremely lumpy graphs.
I certainly don’t expect to be able to blindly apply the central limit theorem and get a perfect Normal distribution- it might be skewed based on difficulty levels and it might be a mixture of 2 or 3 groups (one who studied, didn’t study, got it, didn’t get it). However I would expect a far more curvillinear structure in the graph. Though its likely that one group will be more dominant and the curve looks somewhat bell-ish.
Also check out the extremely spiky CBSE graphs.
http://www.thelearningpoint.net/home/examination-results-2013/exposing-cbse-and-icse