CaHRU's work on the MRCGP AKT examination has shown...
...the test is reliable and valid and demonstrated the processes needed to achieve this...
...female candidates performed better on women's questions, presumably due to differences in training...
...AKT is fair to doctors with dyslexia who declare their disability and are provided with adjustments to compensate for this...
WHAT THIS MEANS FOR ME...
A reliable, valid and fair test is important for the regulator, doctors and the general public
There are ethical and legal reasons for providing a licensing examination that is fair to all candidates
Public confidence is enhanced by doctors that are assessed fit to practice before they begin practice
WHERE CAN I FIND MORE INFORMATION?
Making assessment in the MRCGP fair: http://www.rcgp.org.uk/-/media/Files/GP-training-and-exams/Annual-reports/MRCGP-Fairness-Report-v0101215.ashx?la=en
CaHRU website: http://cahru.org.uk/research/qoph/reliability-validity-and-fairness-in-the-mrcgp-licensing-examination
Investigating the Reliability, Validity and Fairness of the MRCGP Licensing Examination
A study of the video assessment showed...
...were more likely to pass...
..examiners showed higher performance compared to GP trainee candidates...
...candidates who shared decision making during consultations...
...suggesting external validity
...while comparisons demonstrated that...
...consisting of 3 components:
Clinical Skills Assessment
Workplace based Assessment
Applied Knowledge Test
...is a medical licensing exam for medical doctors trained in general practice...
...and who wish to practice independently as GP's
The Membership of the Royal College of General Practitioners (MRCGP)...
Sex
...and those with learning difficulties, e.g. dyslexia
...from BME groups...
and Ethnicity
Research has consistently shown that performance in this and other licensing exams is linked to candidates'...
The Equality Act 2010 covers fairness to groups of candidates with 'protected characteristics'...
...i.e. gender, ethnicity and disability...
...which led to a judicial review of the exam in 2014...
examiners
examinees
deanery staff
and external bodies
...differences cause concern for stakeholders including...
A study of non-pretested vs pre-tested items showed...
...while a post-exam study of candidates taking the AKT showed...
...and highlighted items that were less clear
...areas of validity of the exam
...if rigorous processes for AKT question item development were in place...
...then new items were as reliable as previously tested items ...
A study comparing performance of female vs male candidates showed that...
...men were no better than women...
...at answering questions on men's health
CaHRU conducted the first study to show that...
The Community and Health Research Unit (CaHRU)
carried out research to explore why these issues occur
...while those candidates, particularly those from BME groups...
...who did not declare dyslexia until they had failed at least one AKT performed worse...
...this is likely due to a lack of diagnosis or reluctance to declare a diagnosis
...on questions on women's health...
...but...
There have been concerns about fairness to doctors...
...women scored significantly higher than men...
... doctors who declared dyslexia and were afforded adjustments...
... had similar pass rates compared to those who did not declare dyslexia...
...educational and psychological factors are associated with poor performance but can be affected by educational and training interventions
