Overcoming political and organisation barriers to international practitioner collaboration on national examination standard-setting
Visiting Knowledge Exchange Fellow Dr Lena Gray (AQA) is working with Professor Jo-Anne Baird (Education, Oxford) to develop guidelines for practitioners – encouraging transparency about procedures in national examination standard setting.
A meritocratic society relies on fair examination standards. Examination boards and their staff may find themselves in highly political environments in which the organisations and individuals who work for them can be scapegoated for political failings. This produces a risk-averse setting that discourages transparency and open reflection – this is detrimental to advancing examination bodies’ understanding of theory, policy, and technologies. Further, the processes of setting national examination standards vary in different countries - and within the same country over time. There has been no international comparison between the setting of national examination standards, which makes it difficult to learn from recent research advances and from practitioners’ work.
Professor Baird and Dr Gray have already collaborated on an earlier project that identified key issues for practitioners trying to improve their standard setting practices, and generated an initial set of proposals for ways of tackling these issues. This Knowledge Exchange Fellowship, funded by the Higher Education Innovation Fund (HEIF) and ESRC IAA sought to consolidate this work by enabling the project team to explore and identify political and organisational barriers to implementing a theory of standards, and explore how individuals and organisations have overcome these barriers and improved their technologies and policies as a result. This has been conducted through a series of staged developments informed by partner discussions with Ofqual to produce a new guideline document, which will be circulated to fourteen national exam bodies around the world for feedback and further knowledge exchange. Guidelines and training materials developed during the Fellowship have already been used within AQA in order to aid reflection and drive improvement in practice. The final guideline document will be shared online, in print, and at practitioner and academic conferences in October and November 2017.
This Fellowship has built a stronger methodological, ethical, and conceptual foundation for a larger project which will aim to explore how a selection of nations define and implement their theory of examination standards, and what impacts they have upon how standards are interpreted in the wider community and in the media.
Key Project Activities
• Developing guideline document based on preliminary findings from England and Scotland from 2016.
• Engaging with 14 national examination organisations from across the world with new guideline document.
• Preparing and presenting the guidelines in print and online, dissemination at academic and exam practitioner conferences.