SEED projects 2017 – Writing, Writing Everywhere

Technical writing to improve understanding of threshold engineering concepts

Dr James Lim

Drs James Lim, Vicente Gonzalez, and Raj Das (Faculty of Engineering)

Engineering students can often easily do the math and apply formulas to work. However, in order to apply the right formula to solve a specific question, and to understand what they are doing and why, our future engineers first need to comprehend the underlying concepts.

One of the best tools for teachers to make sure that their students understand such concepts is Test questions. However, the amount of time required for the teacher to evaluate answers and grade the test precludes repeating the exercise to continuously test the students’ understanding of concepts. This project aimed to use ZORRO-Q software to develop on-line feedback-rich activities, in collaboration with Stage III students, to assess and enhance Stage II students’ conceptual understanding. XORRO-Q is an automated assessment tool now integrated into CANVAS. It uniquely enables students to submit sketches and provides automated assessment and relevant feedback to students, and class-wide views for the lecturer. Xorro-Q uses key words or phrases to detect students’ understanding of concepts taught. The difficulty lies in getting the tests “sensitive” enough to detect all the correct answers, and specific enough to avoid “false positive” answers.

It took us several iterations of testing the questions and going over the corrections given by Xorro-Q in order to detect false positive and false negative answers and so improve the correction tool. The test was piloted with a small class (42 participants) as a self-paced online activity which was accessed by most of the cohort.

Overall, students were very receptive to the task and appreciated the possibility to repeat attempts at tackling the test questions. In the feedback, some students showed disappointment with the tool failing to positively identify some of their answers. At other times, they felt confused with how to interpret the questions correctly, and/or what was expected as an answer from them.

More than 12% of the class completed five or more attempts Students made 47 feedback submissions.

Further work planned on this project includes improving both the sensitivity and specificity of the automated correction of questions through the Xorro-Q web application.