Acting on your learning analytics data
Once you have utilised one (or many) of the learning analytics tools available in Moodle (see the articles Introduction to learning analytics in your OLE, Activity completion suite, Moodle reports, and Moodle site participant list for information), the next step is to consider how you might act on the data obtained from these tools. This article will discuss potential interventions for during your teaching session and afterwards, and highlight some important considerations in your course of action.
Why?
The Moodle online learning environment provides a number of ways to obtain analytical data on student interactions with activities and/or resources in your subject Moodle site. It is important to remember that interactions do not necessarily equal engagement. The learning analytics data can tell you what students are accessing and completing, but not whether they have cognitively engaged with content; however, with thoughtful analysis and reflection on your learning analytics data, you can use the findings to improve your students’ learning experiences, act proactively in response to potential issues with student interactions, and enhance your online teaching.
How?
Interventions during teaching session
Often we wait until after a teaching session has finished to reflect on the delivery of the subject and make changes; however, using learning analytics to monitor student progress and interactions with activities and/or resources in your subject Moodle site allows you to make responsive changes during the teaching session.
When you are deciding how to act upon your learning analytics data for your subject during the teaching session, there are some important things to consider. These are explained as follows:
The University has included a statement regarding the collection of learning analytics within Subject Outline templates, to ensure transparency with students:
“Where Learning Analytics data (such as student engagement with Moodle, access to recorded lectures, University Library usage, task marks, and use of SOLS) is available to the Subject Coordinator, this may be used to assist in analysing student engagement and to identify and recommend support to students who may be at risk of failure. If you have questions about the kinds of data the University uses, how we collect it, and how we protect your privacy in the use of this data, please refer to the Privacy page on the UOW website."
It is important that the data you obtain through Moodle with regards to student interactions is not shared beyond your teaching team, and is used in a sensitive and ethical manner during communications with students. Learning analytics data cannot give a complete picture of an individual’s learning. The personal circumstances and pedagogical context need to be taken into account when deciding the actions to take on insights generated through learning analytics.
If you have a teaching team, there should be careful consideration of how you will act on your learning analytics data to ensure consistency in your communications. Nominating a single member of the team to reach out to students (potentially the subject coordinator) is a useful approach so that students are not being bombarded with messages from multiple teaching team members and the messages can be standardised in tone.
When deciding on communication with students based on your learning analytics data, it is recommended that your first line of action should be to send out cohort-wide communication before targeting specific students. This can serve as a gentle reminder to all students about the expectations of interaction within the online learning environment. The Announcements forum in your Moodle site - a special type of forum in which your posts are emailed to the UOW student email account of every student enrolled in your subject - can be used for this communication. Students cannot unsubscribe from this feature, making it a useful channel for communicating with the entire student cohort of your subject. Although SOLSMail is the “official and primary method for the University to communicate important academic and administrative messages directly and securely with students at UOW”, according to the advice provided on the SOLSMail webpage contacting students via this channel for communicating expectations of interaction would not be appropriate. Please see the article Moodle announcements forum for more information about using the Announcements forum.
After communicating on a cohort-wide basis, you may then choose as your next line of action to contact individual students directly. It is important to act in a timely manner to help students avoid further problems theymay encounter later on in the semester. It is recommended that you communicate directly with students via their UOW student email address. Although there are mechanisms within Moodle for communicating privately with students (for example, the Moodle Messages and Dialog functionality), the use of these tools is not recommended as their method of notifying students that they have a waiting message from you may easily result in the message being missed.
When engaging with a student on an individual basis for academic matters, it is useful to consider the broader context; for example, are they struggling only in your subject, or in other subjects as well? There may also be personal issues which are impacting their academic performance. It is important to be aware of the support options available to students at UOW and communicate these to students when appropriate.
When communicating with the entire student cohort and/or individual students, the tone and content of your communication should be carefully considered.
The communication tone overall should be positive and encouraging.
When communicating to the entire cohort, general statements can be used; for example, if you have noticed through the learning analytics data that students are not completing pre-work activities on Moodle, the communication could be:
“Hi everyone, remember that <activity> is a critical part of your pre-work for this week’s tutorials to prepare you with the foundational knowledge that we will build on during the tutorial. Please ensure that you complete this pre-work activity before attending your tutorial.”
In this example, it is important to note that the reasoning as to why it is important for students to engage is included; the communication should not be just telling students to do things without explaining why.
When communicating directly with an individual student, a sensitive and positive approach should be utilised. Offers to support the students or provide information on support channels such as student support advisers are appropriate, for example:
Including statements such as ‘‘Do you need some help?’’ and ‘‘Please contact me to discuss this situation, as I would very much like to help you’’ can be helpful. Avoid starting your communications with “The data has indicated…” and also avoid language that could be interpreted as demotivating, for example ‘‘If you do not attempt this activity, you will fail…’’.
Interventions after teaching session
Once the teaching session is over, it is a great time to stop and reflect on your learning analytics data and any interventions carried out on a student cohort and/or individual student level. Remember that you have a number of sources of data you can draw on, as well as any student feedback surveys you have run yourself (using the Moodle Questionnaire tool or similar - please see the article Gathering student feedback in your OLE for more information), and any formal subject or teaching evaluations.
As well as your learning analytics data, you may wish to reflect on not only the level of interactions for your online activities but also their quality (for example, with student discussion forum posts).
This data can be used to then inform changes to your subject for its next run; however, remember to start small. There is no need to undertake a complete re-vamp of your subject in one go. Instead, consider your current workload and then pick a small area in which to work on implementing and evaluating a change.
Kolb’s (1984) experiential learning cycle is a great way to frame changes to your teaching based on your learning analytics data.
The cycle has four stages:
- Concrete experience – this is where you are delivering your subject during the teaching session and implementing any real-time interventions based on your learning analytics data (as discussed in the section above)
- Reflective observation – reflecting on your experiences and thinking about inconsistencies between your experiences and your teaching understandings and goals
- Abstract conceptualisation – learning from your experiences to generate new ideas for modifications to your online teaching to refine/meet your teaching goals
- Active experimentation – implementing your changes and then applying them next time you teach the subject and seenig what happens
Of course, this cycle then repeats itself, as you draw on your learning analytics data and experiences to implement your changes and then reflect and evaluate them in turn.
The following examples are situations that you may encounter in your teaching, and provide some advice on how the Moodle learning analytics data may be able to help you.
Intervention examples
The following examples are derived from authentic situations encountered by academics.
Context: I have set pre-work on the Moodle site for my students to complete, but feedback from my tutors indicates that they think the students haven’t been completing this pre-work.
How learning analytics can help you with this situation:
The activity completion functionality (please see the article Activity completion suite for more information) can be turned on for your Moodle site, and then can be set for each of the activities or resources that constitute your pre-work. You can then check whether students have interacted with the pre-work via the Overview of Students button in the Progress Bar block. Based on your findings, you might send out a cohort-wide message via the Announcements forum to remind students of the importance of completing the pre-work.
Context: I have included some helpful links in a separate section in my Moodle site, but I am unsure if my students are utilising them or not.
How learning analytics can help you with this situation:
You can use the Activity Report (please see the article Moodle Reports for more information) to see how views there have been for each of your links. Note that the Activity Report indicates x number of views by x number of participants, so this level of granularity does not allow you to determine how many times a particular student has viewed a link (e.g. one student might have viewed a link multiple times). If the total number of views and participants is low, however, you could assume that there are not that many students viewing the links. Adding links to resources to support students with their learning is a great idea; however, the best form of implementation is to include these links in context with the content to which they relate. So, for example, if you have provided links which might support students with completing an assessment task, you could adjust the content on your Moodle site to place those links into the same section where the assessment task information and submission point are located. It is also important to ensure links are meaningfully named and have contextual information, as perhaps students aren’t clicking the links because their purpose is not clear. Please see the article Moodle site layout for further information on how to best lay out your Moodle site, and the article Best practice for creating links for helpful information on managing links in your Moodle site.
Related information
- Introduction to learning analytics in your OLE | L&T Hub article
- Activity completion suite | L&T Hub article
- Moodle reports | L&T Hub article
- Moodle site participant list | L&T Hub article
- Moodle announcements forum | L&T Hub article
- Gathering student feedback in your OLE | L&T Hub article
- Moodle site layout | L&T Hub article
- Best practice for creating links | L&T Hub article
- Ruth Davis, Faculty of Business and Law | Showcase