Oversimplified ranking of schools and students unfair
Former St John's Mahiakalo Primary School pupils in Kakamega are carried shoulder high after excelling in last year’s KCPE exam. PHOTO | ISAAC WALE |
“Not everything that can be counted counts and not everything that counts can be counted.” This quote by physicist Albert Einstein is laden with heavy wisdom that should be specifically relevant to the ongoing public debate about merits and demerits of national ranking of schools.
The decision by Education secretary Jacob Kaimenyi to adopt one of the recommendations of the Kilemi Mwiria-led task force report on education reforms to ban ranking of schools and candidates based on national examinations is a controversial one.
As an active stakeholder in the education sector (I chair the board of a secondary school), I am extremely pleased on two fronts. First, that a recommendation from a task force or commission is being implemented, a far cry from previous regimes where such reports would gather dust for ages, ending up wasting taxpayers’ resources.
Secondly, I am pleased that a conversation on an issue beyond our run of the mill political antics is occupying the media spaces.
Rankings are the hierarchical comparison of two or more entities based on a unified metric. For primary and secondary schools the comparison is based on the aggregate mean in national examinations while ranking of individual students is done though the aggregate scores.
This ranking serves as an accountability tool on the performance of various institutions. Parents and students can make informed decisions on which schools to choose while policy makers are able to structure recommendations to incentivise good performance.
But all this makes two major assumptions – that the measurement of academic performance is possible through a single examination and that the methodology of ranking is fair – both of which may be unfounded.
For every school, students post different results and there thus needs to be a meaningful way of summarising that data to help compare different institutions.
One way is to attempt to get a ‘central tendency’. This is a statistical measure that identifies a single value as representative of an entire distribution of data to provide an accurate description of the entire set. Mean is the most commonly used measure of central tendency.
The arithmetic mean is the simplest representation of the average, computed by adding all the values in the data set divided by the number of observations in it. For instance, if in a class of three, the total scores were 3,4 and 5, then the mean is (3+4+5)/3. This equals 4. Similarly, if the three scored, 1, 1 and 10, then still the mean is 4.
Statistically, these two classes are said to have performed the same since they both had an average of four.
The data, however, clearly shows that this representation is false. While the performance of the first cluster is average, the scores from the second cluster are highly skewed. One abnormally high score has singly shored up the mean of the second cluster and improved the mean.
This illustrates the main disadvantage of the mean as a metric of comparison of performance of schools given its insensitivities to the extreme values/outliers, especially on a small sample size.
This significance is also felt by the fact that different schools have different number of students. Public schools tend to have more students and hence a higher student to teacher ratio than their private counterparts. They are thus unlikely to compete fairly, posting higher examination scores yet this factor is never weighted into the overall mean.
Hence, the mean gives an unfair indication of the teachers and students of two lopsided schools with varying resources at their disposal. Our current system is that of comparing apples and oranges.
Further, the statistical significance of some means, especially by top schools, is normally too small to make meaningful sense. If school number one had a mean of 350 marks out of 500 and school number 10 had a mean of 347.5 marks out of 500, then the range of performance of all the 10 schools represents a statistical tie.
It doesn’t really matter if your student is in school number one or number 10; chances are that they would have scored almost the same marks holding other factors constant.
However, because most of these ranks are reputational rather than factual, the difference is sometimes blown out of proportion, partly to make for good PR and to sell newspapers.
Using the mean to measure performance is not only mathematically unfair, it is also easy to manipulate. Cases of schools undertaking drastic measures, including forcing students to repeat classes, registering weaker students in satellite examination centres and even cheating in exams to boost their mean have been rampant.
Competition
But what about ranking individual students? There are many victories that are public in nature. I am of the opinion that academic examination performance is not among them. We should learn the art of appreciating private victories.
If your son scored 430 marks in KCPE, it does not take splashing the news on a national newspaper to cement home that fact. The whole essence of sitting examinations should be to pass them.
There should be nothing remarkable about passing examinations at the basic levels of learning (primary and secondary).
Unfortunately, through the ranking system, we are at an early age trying to administer the spirit of chronic competition rather than collaboration.
Our system reminds me of the hunters who on seeing a bear charging towards them at high speed, one tells his company that he realises that he does not have to outrun the bear since outrunning his colleague will suffice.
Ranking, for the sake of it, erodes a bit of our humanity and bringing it to a halt is a step in the right direction.
