Data Informed Design

Ruth Davis | Business and Law (BAL)

This recently redesigned subject within a BAL Master’s program used Moodle features including Moodle books, interactive activities and progress bars to continually refine delivery and content to improve learners’ experience.

 

SEA901 is a new 6 credit point subject in a recently redesigned Master’s program. The subject is a compulsory foundation subject that has heavy ‘research skills’ focus continually contextualised by introductory course content. The approach of contextualising the development of research skills within course content worked well, as most students taking the subject are mid-career professionals who completed their undergraduate degree quite some time ago and often in an unrelated field.

Historically, students entering our Master’s program had trouble with academic writing, referencing and discerning between good and bad sources of research evidence. This knowledge was used to shape the design of SEA901. The content from the old subject was repurposed and new activities were added to develop specific types of research skills.

How?

Meaningful changes were created based on feedback data from the previous iterations of related subjects. The subject content is now broken up into three collapsible topics. Each collapsible topic has a Moodle book which students use to access the course content. This approach allowed the ordering of content into manageable chapters and pages to improve student navigation and content comprehension.

Within each Moodle book, students work through pages containing content and embedded interactive activities. The objective of the activities is to provide opportunities and teacher guidance on specific research skills. One activity, for example, provides students with key pages of a book that they need to correctly reference. Accessibility issues were addressed by additionally providing the information in text format also.

H5P was used to facilitate the embedded activities in Moodle Books. One example includes an activity asking students to actively rank the currency, relevance, authority, accuracy and purpose (CRAAP) of provided subject-specific sources. In practice, however, the H5P activity did not provide the flexibility needed for the activity. Students provided constructive feedback on this activity through e-mails and discussion forums.

After the first run of the subject, students communicated that the correct or incorrect ruling provided in the H5P activity did not allow for the subjective nature of the CRAAP scale. Being marked incorrect on a subjective scale was cited as being confusing and demotivating for students. This feedback data was taken on-board and used to redesign the activity to benefit students’ engagement and confidence in developing source analysis skills.

Learning analytic functions in Moodle were also used in SEA901, including activity completion and the completion progress. The embedded H5P activities provided the teaching team with activity completion reports, while students viewed a subject level overview of their progress. By design, not every activity was included in the completion progress bar to ensure students were able to meaningfully review their progress, without being overwhelmed. For example, the Moodle books and the ‘Practice Quiz’ were selected, while other activities were purposefully omitted. 

The data collected from this tool helped the teaching staff view which content students had accessed, not only to inform conversations with students but to gain an understanding of student activity trends and changes that could be made to the content.

Reflection

When selecting the settings for activity completion and the progress bar, we found that students did not necessarily want to self-check the completion boxes themselves, so we limited the checkboxes to ‘tasks automatically checked’ boxes. We also often attached the exact date we wanted the activity completed by.

It is good to think about what you want to collect data on. Do you want students to attempt or view an activity? Or do you want students to receive a certain mark to satisfactorily complete the activity on the progress bar? Answering these questions will help as you set up the progress bar and activity completion.

Be strategic with what you collect data on, and what you choose to include in the progress bar so that students can self-regulate their learning without being overwhelmed at the volume of work to be done.

Overall, one key tip would be to be clear on what is worth collecting data on in your subject. Once you have identified these features should be tracked via progress bars so that students do not feel overwhelmed.

When selecting the settings for activity completion and the progress bar, we found that students did not necessarily want to self-check the completion boxes themselves, so we limited the checkboxes to ‘tasks automatically checked’ boxes. We also often attached the exact date we wanted the activity completed by.

It is good to think about what you want to collect data on. Do you want students to attempt or view an activity? Or do you want students to receive a certain mark to satisfactorily complete the activity on the progress bar? Answering these questions will help as you set up the progress bar and activity completion.

Be strategic with what you collect data on, and what you choose to include in the progress bar so that students can self-regulate their learning without being overwhelmed at the volume of work to be done.