2017 report into ACEM Alleged Exam Prejudice

Release of the College Investigation into ‘Alleged Exam Prejudice’ The Australasian College of Emergency Medicine (ACEM) published the findings of the Expert Advisory Group (EAG) report yesterday. While the process has taken some time, the terms of reference and externally commissioned reports suggest an attempt at a robust investigation. A media release summarised the findings of Dr Helen […]

via Emergency Exam E.A.G. Report — EmergencyPedia

One thought on “2017 report into ACEM Alleged Exam Prejudice

  1. A ……. WHITE LIE

    First of all would like to commend the efforts of EAG for all these months.they have worked tirelessly to prove that there was no discrimination occurred even with a difference of 80% in results between the two groups.
    We cannot blame the EAG for doing what they were supposed to do.They were hired by the ACEM and were an “INDEPENDENT GROUP” even though most of the members were from ACEM.ACEM paid for the investigation and got the results that they wanted.How can they go against ACEM? They had to make college look good in this fiasco.

    Lets go through some of the points in the report

    3.15.1 there is no direct evidence to support bias against Group B candidates. Combining this with the assumption that, ‘during the station construction process these issues have been addressed as well and the stations have been scrutinised for any such possible bias’, Professor Schuwirth concluded that the difference observed is a true difference in performance.

    No direct evidence but we will just assume that the college addressed the issues of bias – the complaint was that the college did not address these issues. The Analyst ASSumed that the college has addressed these issues adequately. So the only possible explanation was that these non whites were stupid enough to attempt the exam.

    3.19 While the psychometric analysis has demonstrated that examiner bias did not have a significantly statistically identifiable effect on the 2016.2 OSCE, individual examiner bias could not be ruled out completely. The EAG has identified a number of factors that may have had an impact on the examination outcomes. As scoring of candidates performance involves independent expert judgement, it is potentially open to subjectivity and conscious or unconscious bias. The absence of documented marking criteria for candidate performance for ‘minimum level of competence displayed’ for marking Fellowship OSCE domains such as communication, professionalism, scholarship and teaching, and leadership and management can lead to greater individual subjectivity in the evaluation of these domains and interpretation of a candidate’s performance as ‘just at standard’. Others factors may also have played a part: relatively low examiner diversity (e.g. in the 2016.2 OSCE 93% of examiners were from Group A countries of primary medical qualification and 7% were from Group B countries of primary medical qualification), sub-optimal discussion of cultural sensitivity in calibration of OSCE assessment criteria, and the fact that raters’ cognition is predominantly influenced by their own experiences, values and interests. Taken together, all these factors may have resulted in judgements giving rise to differential and potentially unfair outcomes for Group B candidates.

    Taken together all these things mean that these candidates were discriminated due to the subjective assessment of examiners rather than objective. Why are these unfair outcomes only for group B candidates and not for others? This is what the complaint called racial discrimination because you belonged to non white group and all these factors only applied to non white group. What else do you want to call this – upliftment of non white group ?

    4.8.2 Submissions received indicate there is a perception among DEMTs that candidates believe that if they are appointed as an ED Registrar, they are capable of working as a FACEM and therefore automatically able to pass the Fellowship examinations.

    The EAG does not even understand what the training process is. This perception among DEMTs is wrong as trainees become ED registrars once they pass primary exam.No one thinks they would be able to work at the level of FACEM as soon as they are appointed as ED registrar. Its only after they worked at that level for few years that you would think they are at the level of FACEM which they should be if trained properly.if they are not upto that level even after working 4 -6 years ,whose fault is it?You should look at those DEMTs who have not trained them to get to that level.

    Same mistake as they did in interim report – admission to training did not guarantee you a spot in OSCE exam, you have to pass multiple hurdles to get to the OSCE exam.

    4.8.8 Another FACEM sought feedback for a specific trainee and was told that the trainee’s performance “might have been good enough in India but was not good enough here”. Such experiences are not limited to these examples.

    This response from the Exam committee to a DEMT shows how biased they are towards these trainees.”Good enough for INDIA”. So definitely the exam committee is already thinking that IMGs from India are not good enough to work in Australia. Pretty good feedback from a specialist Exam committee. Exactly the sort of feedback DEMT was expecting.DEMt will then apply to all his Indian trainees and deem them not suitable enough.No surprise they just keep failing.

    In this context we can also remember the rant of a Senior consultant in The Australian who said “Indians are not even fit enough to be registered as Doctors”.

    This is definitely not racial discrimination ,it is just that exam committee doesn’t believe in Indian doctors to be good enough to be consultants even though they trained at the same level with other white trainees.

    4.9.2 Examiners were also concerned about the matters raised in the interim report about examiner selection criteria. The selection criteria necessarily require prospective examiners to demonstrate their capacity to examine trainees. The pool of prospective examiners is limited to those Fellows who already participate in College activities. Of note is one examiner’s concerns that if a requirement of cultural diversity within the Court of Examiners was imposed, advice would be required on how this requirement can be implemented without introducing another form of discrimination of one cultural group being preferred over another.

    Yes thats right. The culturally diverse group will create problems. Lets keep it all white and there should be no problems.

    5.8.4 The detailed analyses by candidate ethnicity show that although white candidates out-performed BME candidates, the differences were largely mirrored across the two different sets of examinations. Although the reason for the differential performance is unclear, the authors concluded that:

    “… the similarity of the effects in independent knowledge and clinical examinations in different specialty colleges suggests the differences are unlikely to result from specific features of either assessment and most likely represent true differences in ability.” (p.1).

    Yes the non whites are inherently stupid and cannot compete against white man.So his abilities are lower than a white person anyway so he should be told to stay at his level and not aspire for higher abilities. It is already proven in research that the BME candidates are inferior to whites ,hence it proves that here is no racial discrimination even when there is 80% difference in results between two groups.

    6.21 That is not to say it is not occurring at all in the ACEM Fellowship OSCE. The College’s approach to standard setting and use of the Borderline Regression Method to determine the cut-score and pass mark required for each OSCE is appropriate and consistent with international use. However, the effectiveness of this approach relies upon robust standard setting for each domain/station through effective calibration of examiners to achieve a consistent and fair understanding of how to judge when a candidate’s performance is ‘just at standard’ for that station.

    It is occuring and thats what occurred in the OSCE exam.The examiners were given free rein as to what they can do rather than keeping the exam objective.the examiners went about culling all those people who did not match their perception of color,accent,and race. Objectivity depends on definite marking system and reproducibility of results.ACEM even declined to video record the exam citing some silly reasons even after the candidates signed the confidentiality form.

    6.22 The EAG’s opinion is that the College’s management of the OSCE to date potentially enables systemic discrimination to manifest in some elements of the examination.

    On one hand EAG loudly proclaims no discrimination occurred and then makes multiple statements as to how it could have occurred but cannot explain the difference in the pass rates.

    7.4.7 The College has conceded that inadequate supervision and training of candidates in the workplace has possibly led to candidates who were not ready and/or not competent attempting the examination or continuing in the training program. The College advised that former ITAs conducted at the end of term appeared of questionable utility and ability to identify trainees who needed extra guidance and training in order to progress to a standard of practice at a specialist level.

    I cannot understand why suddenly overnight 90% of non white trainees became unsuitable .the training was same throughout for both white and non white trainees . Even exam preparation was done together by both groups and even attended same courses. But on the day of exam the non whites suddenly became stupid and unable to articulate in the exam- almost 90% of the non white cohort.
    Unless the college and DEMTs are running a seperate training programme for whites where they are taught separately as to how to become a consultant this is not possible.
    Inadequate training and supervision is now the fault of trainees.EAG and college are trying to say that the training was shit but its the trainees fault. The training was good and as far as I know most of the DEMTs work very hard to get you upto the level and will definitely tell you whats wrong with you. Most of the DEMTs didn’t have a clue why their trainees failed and the racist comments from the examination committee didn’t make things easier.(good enough for India)

    7.4.11 That there is a true difference in performance based upon the source of a candidate’s primary medical degree; that differences in their medical training can result in some candidates not being up to the clinical standard required due to non-comparable training methods and assessments.

    I agree it is a possibility in some cases, but 90% of the non white trainees not being upto the clinical standard? This puts serious concerns towards ACEM ’s suitability to be a training body.

    EAG cannot be considered an independent body as it was filled with ACEM members.ACEM investigated itself and gave a clean chit to itself.

    Why apologise if no if no discrimination was done?
    Why refund the exam fees if they were not at fault?

    EAG has done very well in what they were told to do. To clear ACEM of the racial discrimination complaint an portray them as a benevolent institution trying to help inherently stupid non white trainees.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

%d bloggers like this: