Understanding What Really Matters to Students
In February 2022, a Task & Finish Group of key stakeholders was established to address the issue of inconsistent layout across courses in Canvas, the virtual learning environment (VLE) at Queen’s. The Group are currently working together on a ‘Canvas Course Co-Design Project’ to develop a framework which schools can use to ensure a ‘student-first’ approach to course development in Canvas. Ultimately, all VLE courses should offer a good and consistent digital pedagogic baseline for student experience, ensuring greater consistency in the way it is used. The main aim of the project is to improve the user experience for all students in Canvas.
User Research
To better understand our business requirements and user needs, we conducted user research in March 2022 which consisted of a Top Task survey and Card Sort activity. We needed concrete and measurable information to establish:
- what students want when accessing course materials online in Canvas
- whether we are providing it
- whether students can find and use it easily
- whether we can do better in terms of making content easily discoverable and usable
Other educational institutions in the UK have successfully applied research methods to identify what students need when using a digital platform or application. We consulted with colleagues in Edinburgh who kindly shared their experiences of conducting a programme of user research in support of the Learn Foundations project (thank you to Duncan, Emma and Neil!). Their advice was clear – if you only do one thing to improve the VLE environment, make sure it’s focused on making the students top tasks easy to locate in Canvas!
Top Tasks
We began with a Top Tasks Survey, following a process designed by Gerry McGovern, the originator of Top Tasks Management, which is detailed in his article What Really Matters: Focusing on Top Tasks. Top tasks is about clearing away the fog and understanding what really matters to students. It is a quick, easy and effective way to help identify what is most important.
The first step in the process was to identify a list of tasks. A number of key stakeholders took part in a workshop to generate lots of potential tasks and ideas on a Padlet board. Following the workshop, a sample of modules from each faculty were reviewed to identify any additional tasks or complex course information not already captured. These activities combined, resulted in a list of 122 terms/tasks which were then shortlisted to refine the list of tasks, removing any duplicate entries and updating language to make sure the final list of 52 tasks was clear and unambiguous.
We then used Survey Monkey for the survey design and collection. The Top Tasks Survey, presented students with a very long list of tasks and asked them to select five items that were most important to them.
A total of 127 students completed the survey, casting 635 votes across the 52 tasks. The top task identified are ones that students expect to be able to easily access, as a minimum expectation! The top three items, which all related to lectures, took up over a quarter of all votes (162 votes).
- Lecture recordings – 9.6% of all votes
- Lecture slides – 8.8%
- Lecture notes/outlines/handouts – 7.1%
- Assignment submission/upload – Submit work for marking – 6.1%
- Assessment information and deadlines – 5.8%
- Grades – access grades for courses – 5.5%
- Class times – find out when your classes are – 5.4%
We ran the survey with staff as well as students to help identify where there may be a gap between staff perceptions and what students say they need. You can see the key findings from the survey in the slides below:
Top Tasks says to: “Focus on what really matters (the top tasks) and defocus on what matters less (the tiny tasks).” It is important to call out that focusing on student top tasks is not a capitulation for the students at the expense of what is important for the institution. It doesn’t mean that we ignore tasks related to our business requirements. It certainly doesn’t mean that the ‘Tiny Tasks’ get excluded from the VLE, it just highlights the items that may not warrant a prominent position within a VLE course.
Card Sort
The top tasks approach is an effective way of getting a clear idea of what we should be focusing on when developing courses or templates for Canvas but it is just one of many user research methods that can give a comprehensive idea of what’s working and what’s not. We agreed that the survey when combined with a card sorting activity, could help us further understand how students expect information to be categorised and structured in their Canvas courses and also the type of language and terminology used.
We chose to use ‘Optimal Workshop’ software which makes it very easy to run an online card sort with students. Unfortunately our student sample fell below the 30 responses we were aiming for, with 20/27 students fully completing the cart sort activity. What we did learn from the small sample was that those tasks that students had voted as most important to them in the survey, they also expected to find them located in the same place (within week/theme/topic course structure).
Next Steps
Gerry McGovern states in his article What Really Matters: Focusing on Top Tasks – ‘The essence of a great customer experience is to help people quickly and easily complete their tasks—but to do that, you need evidence of those tasks, not opinions.”
The output from the user research activities has helped us to identify the minimum expectations that students have when they access their courses in Canvas and where they broadly expect to find things. The output will also help shape the design of VLE Templates and guidance to support future course development.
Further Reading
📑 What Really Matters: Focusing on Top Tasks, article by Gerry McGovern, the originator of Top Tasks Management
📑 Optimising what’s important – Get to know your long neck, blog post by Neil Allison, Head of Prospective Student Web Content, University of Edinburgh
📑 Top tasks surveys have identified what really matters to students using Learn, blog post by Duncan Stephen, User Experience Manager, University of Edinburgh