The role of the HCD/CAIAT Project in improving the ability of science teachers for constructing HCD test items in the Kingdom of Saudi Arabia

The objective of this research is to participate in improving the quality of education in the Kingdom of Saudi Arabia (KSA) by developing the skills of Saudi female science teachers in writing higher cognitive demand (HCD) questions of exemplary quality. It is an evaluative study that follows the descriptive method of research design by depending on a combination of both quantitative and qualitative inquiry. Therefore, various instruments for collecting data were employed. The sample size of 409 represents all of the female science teachers who work in the girls' schools in the urban area of Al Ahsa, a city in KSA. A suggested program called HCD/CAIAT is introduced and the main objective of the present evaluative research is to examine this project’s functional potential to improve the researched sample related practices. The project includes an innovative software package, the Computer Aided Item Analysis Technique (CAIAT) designed purposely for this research in the Arabic language to provide the sample teachers with the two parameters of classical item analysis that indicate the strengths or weaknesses of a test question (difficulty and discrimination). This package is introduced through a training course that also trains the teachers in skills of question construction and teaching on HCD level. The CAIAT is intended to stimulate the teachers' professional development (PD) by raising their awareness of the validity of their HCD test items and encouraging/assisting them to improve their HCD questions over time which is anticipated to help improve their instruction. This concept of utilising CAIAT for improving teachers' practices is breaking new ground and establishing a basis for further development in the field of study. The main purpose of the research is to answer the following two major questions. The first is to what extent can the HCD/CAIAT project assist female science teachers in Saudi schools to improve their ability to analyse their test questions, so as to write exemplary HCD test items and to teach at HCD level (Effectiveness dimension)? And the second is, to what extent could this be reflected in their on-going practice both for the test construction and for teaching (Adoption dimension)? The findings have indicated that the sample teachers' prior background in the researched concepts and skills (HCD and IAT) are limited. However, the effectiveness dimension findings showed that the teachers have successfully acquired all of the project's abilities/skills: knowledge of HCD concepts, skills of writing HCD instructional objectives and HCD questions, and using/utilising CAIAT successfully for assessing their test items. For the adoption dimension, the HCD/CAIAT package was successful in encouraging the teachers to adopt HCD and IAT which was a result of the successful role of the CAIAT software in stimulating the teachers' PD for learning (on their own) how to improve their assessment skills for HCD levels. Furthermore, the research has identified ten study variables, which are the teachers' background characteristics, in order to test the statistical significance of their role in the reported differences amongst the results found for the various aspects measured by the research data collection instruments. These teachers' characteristics are: educational qualification, prior training on test construction skills, prior training on IAT, key stage (intermediate/secondary), level of graduation (GPA/equivalent), years of experience in teaching, specialisation subject, prior experience in using computers, possess of a PC at home and ability to use some mainstream software packages. Statistically, the impact of these variables on the teachers' acquisition or adoption of the project's concept and skills was found very limited; which supports generalizability of the research findings. It is recommended that the Ministry of Education (MoE) at KSA adopt the HCD/CAIAT package in order to encourage all KSA female science teachers to tackle HCD levels in their instruction and assessment, which is very likely to have a positive impact on their efforts in teaching thinking and inducing creativity. Ten other recommendations were also suggested.