Yes, it is annoying, but also fascinating, how the tech giants such as Amazon, Alibaba, Google and Microsoft know enough about us to alert us to new opportunities.  They use targeted advertising, reminders, nudges and suggestions linked to our online activity.  Well, that’s how they make their money, they are motivated to understand the individual customers.

Of course, Business Schools share that motivation – to understand our students better and to understand our academic colleagues in order to support learning and development.  But just how many of us use the wealth of data that is available on the School VLE, the attendance system, the Lecture capture database, the formative and summative assessments that are conducted online?

…and if we did know how to analyse it, what aspects would be important to us?

Quality Assurance is embedded in good Schools – and is a cornerstone of accreditations but drill down and the evidence of widespread use of data is scarce.

The EOCCS standard

EOCCS QA standard

The EOCCS criteria go on to check how student learning and teacher performance is monitored and how this data feeds back into the Quality Assurance system.

Many online courses use a version of the tried and tested end, of course, student feedback survey.  More responsive schools recognise the issues with low response rates to online surveys and prefer a qualitative approach such as focus groups.  Since peer feedback is even less regular, the views of students and the “scores” that they generate are used to advise where improvements are needed.  Institutional feedback systems and national “student satisfaction” surveys probably play a disproportionate part in driving change since the data is readily available and it is politic to be seen to listen to “consumers”.

There is less evidence to suggest if student-led sites such as https://www.ratemyprofessors.com/ are influential.  The mere existence of this resource, however, suggests that traditional feedback systems are not always enough to capture all reflections from students or have the impact that is desired.

Design Thinking

The thoughts of academics Bongiovanni and Balgabekova (2021) begin to scratch the surface of this topic.  In their recent paper, based on experience at a UK Business School, the authors outline a study that used a “mini hackathon” (my description) workshop not only to engage students but also to elicit feedback on their whole experience.  The “ideation” revealed that certain “pains” in their experience could be eased, through design thinking and that workable solutions could be found.

One example of “pain” was the lack of synergy and connection between international partner universities.  Students offered possible solutions but I am sure that curriculum designers and individual academics could offer more embedded responses.  What was also clear was that students welcomed the opportunity to reflect on their whole University experience, rather than on each module individually.  Also, feedback was given at a time when changes could impact the students themselves, rather than just future students.

Adrift?

Add to this “helicopter” viewpoint the volume of data that can be spawned by online systems:

• Virtual visits to the library
• Nature of Library visits
• Watching of pre-recorded video (when and how long…)
• Discussion Board activity
• Completion of formative tasks
• The list appears endless…

We have an opportunity for learning analytics to play a larger part in a more holistic quality assurance system.

However, this is not a “one size fits all” solution.  Each institution needs to tailor its use of data to consider the markets it serves and the purposes for which it collects, analyses and disseminates feedback.  After all, quality is in the eye of the beholder.

Join the activities of the EFMD Online Learning Community as we discuss, interrogate and “ideate” on topics like this.

REFERENCE:

Bongiovanni, I., & Balgabekova, D. (2021). Ask me if I am Engaged: A Design-led Approach to Collect Student Feedback on their University Experience. Design And Technology Education: An International Journal, 26(1), 89-117. Retrieved from https://ojs.lboro.ac.uk/DATE/article/view/2864