Bow Valley College, a renowned college in Canada, has implemented a Competency-Based Education (CBE) system, including in its Cloud Computing Post-Diploma Certificate program. Central to this system is a custom-designed Learning Management System focused on performance-based assessments. The assessment method is closely aligned with real-world professional competencies developed in collaboration with industry experts. Despite its robustness, the current model’s scalability is hampered by its labor-intensive nature, particularly as students require multiple assessments to demonstrate competency (Khan et al., 2023). Addressing this challenge, our research introduces an AI-powered assessment model using generative AI and Large Language Models, trained based on the current real-world scenarios designed to assess competencies. This study adopts a within-subjects design, enabling the comparison of each learner’s performance in both AI-assisted and existing assessments. This offers a unique perspective on the effectiveness of the AI model compared to conventional methods. The research methodology involves both quantitative and qualitative analyses. Performance metrics from AI-assisted and existing assessments will be statistically compared for each student, focusing on accuracy and consistency in the AI model (Wolf & Wolf, 2023). Importantly, qualitative feedback from interviews with students, instructors and industry experts will provide insights into their experiences with both assessment types. This iterative testing approach, informed by ongoing performance data and participants’ feedback, aims to continuously refine the AI model, enhancing its accuracy and reliability. We aim to develop an AI-assisted assessment framework that is both scalable and maintains the integrity of traditional competency-based evaluations, potentially revolutionizing the assessment process in CBE.

Authors:
Reyhaneh Bastani, Bow Valley College, Canada
Tanya McDonald, Bow Valley College, Canada

About the Presenter(s)
Dr. Reyhaneh Bastani, Manager of Applied Research in Educational Technologies at Bow Valley College, Calgary, Canada.

See this presentation on the full schedule – Sunday Schedule

Bookmark