SEED projects 2020 – Designing for learning

Quantext pilot study

Dr Andrew Withy

Dr Andrew Withy (Faculty of Arts), Prof Toni Bruce (Faculty of Education and Social Work), Angela Tsai (Faculty of Medical and Health Sciences), Dr Colin Whittaker, Dr Lokesh Padhye (Faculty of Engineering), and Steve Leichtweis (Centre for Learning and Research in Higher Education)

Several University of Auckland staff have taken part in a pilot study to explore the potential of Quantext, a text analysis tool for teachers. Quantext enables staff to quantify and visualise written (qualitative) data, for example from student online discussion forums, or typed answers to short-answer questions in assessments.

In busy university classes, across a wide range of subjects, teachers have few tools that help them to reflect on and identify opportunities to improve engagement and interactions with students. When instructors read and/or assess students’ written work, it is common practise for us to reflect and provide personalised feedback for students. However, workload pressures and time constraints prevent us from being able to analyse systematically, deeply and globally the themes and issues arising from the class as a whole. Furthermore, the current inability to objectively quantify the observations we make means that the conclusions drawn by instructors are tainted with our own inherent biases. With its ability to provide the means to aggregate and visualise short-form student writing, Quantext is one such tool. A group of five academic staff from four faculties who have been involved in the pilot study are keen to implement and further explore opportunities to use Quantext in their teaching. Specific applications of Quantext in the pilot study include, but are not limited to: demonstrating to students how their lived-experiences contribute to their studies in education; exploring threshold concepts in first-year courses; seeking regular feedback from students on aspects of lectures they find hard; improving assignment marking efficiency, and reliability.

A useful next step is to extend support for teachers beyond the funded pilot study, and to bring teachers from different faculties together to share their aspirations for, and experience with, using Quantext. The SEED grant helped resource discipline-specific Teaching and Research Assistants to help code, categorise and label a representative sample of student discussion data in Quantext, as a way of ‘training’ the AI to be able to label these types of discussion automatically in the future. As there are many types of discussion across the different courses, the initial labelling is also a way of establishing ‘best practice’ for our procedures for ongoing labelling efforts. This work helped to make Quantext more user-friendly, time-efficient and accessible for instructors.