"CAT goes online," screamed all the newspapers a couple of weeks back. Though, the IIMs have not come out clearly on the 'what and how' of CAT Online, there is enough evidence for you to be prepared for an online exam. Firstly, there was no outright denial (other than some minor clarifications that came up on the IIM Kolkata site) of the plans which seem to indicate that the IIMs have let the newspapers build up the case for them. Secondly, IIMs have invited tenders for conducting the test by parties "who have the ability to put up infrastructure that can handle the exam on such a large scale". So there is enough circumstantial evidence to point to an impending shift to a CAT Online.
When we talk of CAT going online, there are a whole lot of questions that we may have in our minds.
~ What will online mean?
~ Why did the IIMs think of online CAT as an option of testing?
~ What are the models of online testing that are used world over?
~ What are the issues linked to each of these models?
~ What could this lead CAT into in the future?
~ What are the implications for students?
CAT Online could lead to a couple of interpretations because of the usage of the word Online. But it could only mean 'taking a test on the computer which serves questions from a local server'. It definitely does not mean 'testing on the web'.
The CAT in its present avatar has had an unblemished record with the exception of the CAT 2003 leak. Then why are the IIMs considering an alternative mode to the testing process? That may also be the question that the IIMs would have to face from the HRD ministry. I think the reason lies in the burgeoning number of CAT applicants each year. The numbers have moved from 90,000 aspirants to 2.3 lakh aspirants over a span of four years. It is a far cry in comparison to the CAT just a decade back when the number of CAT applicants was somewhere around 35,000. The ensuing complexities and the operational hassles of conducting the CAT across multiple centres covering close to 2.3 lakh applicants must be forcing the IIMs to look at the alternative routes for testing. The CAT committee must be consumed with the fear of security breaches to the test process. Any mishap at any one centre/ city on the day could lead to chaos in the testing process. Who knows, the "leak" of CAT 2003 may have been the genesis of this idea?
Take a look at GRE and GMAT as case studies. They used to be conducted at designated times of the year as paper-based tests, till the mid-nineties. They have moved over to online testing since then. In Asia, which happens to be their largest market, there has always been a perpetual fear of paper leaks. For instance, even today if you have to take A-GRE and USMLE you have to travel abroad to take the exam. India is blacklisted as a testing centre as there were instances of paper leak in the past. An important observation to make is that these are paper-based tests. So one of the reasons could be to preserve the sanctity of the paper.
But online testing by its nature also allows a host of other flexibilities that the admissions committees could look forward to. Like convenience of testing across geographies is far and wide, standardisation of tests for different cultural groups, conducting test at any time of the year. So it makes sense to understand the various testing models that are widely used in the assessments market.
Model I: On-the-web testing
On-the-web testing could be good mode to prepare for the final exam. In my knowledge there aren't any admission tests that are conducted on the web. For obvious reasons: It is difficult to ensure the sanctity of the exam in this remote testing process. So this is a model that would be rejected by anyone conducting objective type test. So IIMs would not even have considered this an option. But if CAT goes online, web testing could become an important tool for practice and learning for students.
Model II: Offline, on-the-computer, same test for everyone
Same test for everyone, which would logically mean simultaneous testing of all. I think this could be a precondition that the HRD ministry may put on the IIMs. But I believe the IIMs would like to avoid this option, for a couple of reasons. If they go for this, there is a greater degree of complexity that they are introducing into the system. They are now looking for computer terminals in the place of benches, looking at testing centres that have to increase 10-fold compared to the present. On top of this one computer glitch in one centre would spell big trouble for the authorities. The precedent does not sound too encouraging either. When XLRI tried conducting XAT online in January 2002, when merely 20,000 students were appearing for the test, they miscalculated the issues of conducting simultaneous test on the computers and finally had to cancel the test on the eleventh hour. Finally, they went back to conducting the test as a paper-based test. In my opinion, this is not an option at all.
But a moderated version of this is being successfully applied by BITS, Pilani, for its graduate admissions, the BITSAT. The BITSAT model has an outsourced partner who has built the testing engine, which generates tests based on a 'tagged database' supplied by the authorities. By an application of some algorithm the test is generated. In the BITSAT system the tests is conducted at certain designated centers over a span of 40 days. Students are expected to book the time slots when they would want to take the test. Students who take the tests in a particular time slot would get very similar papers or sometimes the same paper. But different time slots definitely get different papers. The advantage of going for same test in the same slot is that the number of new papers to be generated from the database of questions is fewer and hence, the quantum of the database for generating the tests would be smaller in comparison to different tests for different people. The number of students who took BITSAT last year was 90,000 and is expected to touch 1.2 lakh this year.
The challenge in this testing process is the standardisation of two different tests. This means that two completely different tests must be so generated to check the same skill set. They must be comparable as papers to be called standardised tests. If they are standardised the scores generated for two different candidates can be compared for the admission process. The burden of proof on the IIMs would be to prove the algorithm to create a standardised testing process.
Model III: Offline, on-the-computer, different tests for each person
There are broadly two types of tests that can be done.
A) Tests generated by a designated algorithm on types of questions, difficulty levels. This is very similar to the BITSAT model, only that every student gets a different test but the algorithm is similar.
B) Tests those are adaptive. These tests are dynamically generated, based on how the student is attempting the paper. GRE and GMAT have cracked this model. And the beauty of GRE and GMAT is that irrespective of how often you take the tests, it is likely that your scores are likely to be more or less similar (assuming that you were at your best in all the attempts). No wonder the validity of the scores is for three to five years. If someone can create this level of standardisation for CAT, I think it would be 'THE MODEL' that they would adopt.
My guess is that CAT (if it finally gets the approval from the ministry) would prefer III (B) or the model of BITSAT. It would have to do quite a rigorous testing to be satisfied about either of these models.
The future of CAT looks quite interesting. If they manage a good level of standardisation I think there are interesting possibilities in the future. You could take the tests as many times a year, you could take it at any time of the year, your scores could be valid beyond just an year, the weightage to CAT may go down and the weightage for the other aspects of a candidate may go up. There are definitely some 'hints' in the admission process. The number of questions in CAT has been reducing constantly. The last two years there have been 75 Qs in all. It is 45 per cent of the total number questions that came in CAT 1998!
Guess what, there are close to 75 questions in GMAT as well! And it is online.
All these in hindsight do not seem mere coincidences. CAT is definitely studying closely the other standardised tests like GRE/GMAT. So a logical conclusion is that CAT will be either a three-hour or a three-and-half-hour test when it goes online.
What would it mean to a student? Two things. First, comfort with working on and reading from the computer is a must. Second, part of your practice will shift to the computers. Though, I fundamentally do not see any big difference in the way people would learn, I do foresee more people willing to learn through the computers.
But before all these happen, I think, IIMs would have to cross the first hump of convincing the ministry and secondly, to enrol the affiliated institutes into this thinking process. But the very fact that they have initiated the process, definitely means that this move will be executed very soon and is not just in the realm of ideation.
The author is a director (R&D), Career Launcher India Ltd.