Quo Vadis Opinions

As regular readers will know I’m in favour of the new HPAT test  for entry to undergraduate medicine. When I wrote about the test last January, a few respondents disagreed and said it was simply to make sure less women got into medicine and that the quality of doctors would decrease. So given this lack of trust in the test it’s important HPAT runs smoothly and professionally.
However if I was assessing their performance to date, it would be a “could do better”.  Granted this is the first year of HPAT, but their procedures are causing significant irritation amongst students and are capable of undermining trust in this important psychometric test.

For a psychometric test to be worthwhile, two key criteria need to be measured.
Does the test actually consistently produce the same result for candidates of similar ability? (Known as reliability) For this reason all reputable tests always have strict conditions  to ensure all candidates sit the test in the similar conditions, the same instructions are always used, the room must be interruption free, time strictly allocated  etc,
In running the tests last February, quite a number of students experienced significant delays in sitting the test. For example in the rather cold RDS students had to hang around for an hour after registration before the test started.  Such students could argue they were put at a disadvantage to students who sat the test without delay and undermines the test reliability.  Now granted if you become stressed after waiting an hour, to the extent it worsens your performance, I’d query your ability to be a doctor, as you’ll experience a lot worse working for the HSE!
But the delay does provide ammunition for a legal challenge.

A second criteria is validity. Does the test actually measure what it supposed to measure? 
For example in 1949, the American army found  their psychometric test for selecting cooks actually measured reading ability rather than cooking ability. People who could cook well were not selected because they could not read the test questions. So the test was not valid.
The HPAT assesses three different types of ability.
   Logical reasoning and problem solving
   Non-verbal reasoning
   Interpersonal understanding
For the test to be valid, each of these abilities should be accurately measured.

It can be argued that the Leaving Certificate measures the first two abilities to a reasonable extent.
However the interpersonal understanding is most definitely not measured to any reliable extent in the Leaving cert and I believe this is where the HPAT adds real value to the selection process. After all interpersonal understanding from doctors is absolutely essential to good treatment. Doctors needs to understand what their patients are saying and the personal situations & lives  of their clients!
So a person considering a career as a doctor would find feedback on their interpersonal understanding very valuable as this indicates their suitability for the career.  But despite paying a fee of €95, candidates are not given this valuable information. This suggests HPAT do not have confidence in the individual test results.

However communications between the CAO & HPAT seems to be confused, so the students CAO record is able to hold the score of each individual result. So when the results were given to the CAO, they put up the individual test ability results. When the mistake was realised (Boards discussion board) it appears the individual scores were then replaced by the overall score.

Result: It appears some candidates got access to their scores, if they logged on to their CAO record quickly enough, others did not get access to their individual scores.
This is unfair.

I did take this up with the CAO, who said it was a matter for HPAT.
HPAT in fairness did reply promptly to my emails, but when I asked them to confirm no student had access to individual scores, there was a deafening silence.
Given there have been past legal challenges by disappointed would be medical students, HPAT needs to do better, because students are used to a more transparent system 
Also to acknowledge a personal interest. People like me who have invested considerable time and money in obtaining qualifications in psychometric testing to work with clients are frustrated that poor administration of this high profile test could damage the reputation of psychometric testing generally.

Update 28th August
The CAO application record now holds the individual test results.  Seems the HPAT people do listen to public comment!

Keywords
career coach
career builder
career coaching
career guidance Ireland

Tags: CAO, HPAT

This entry was posted on Monday, August 17th, 2009 at 7:49 am and is filed under CAO. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.