Developing guidelines for responsible GenAI use in assessments

Dr Matalena Tofa

GWP 900/GWP 901

Learning, Teaching and Curriculum (LTC)

Matalena developed guidelines for Generative Artificial Intelligence (GenAI) use in assessments in GWP 900 and GWP 901, career development learning subjects for postgraduate students. She wanted her students to be able to make informed choices about whether and how they use GenAI, both in their learning and in preparation for their future work. 

The assessments in GWP 900/901 are primarily text based and do not require the use of GenAI, but Matalena wanted to create a safe space for learning in which students who choose to use GenAI could practice doing so responsibly and transparently. In this showcase, Matalena outlines and reflects on her journey from her considerations when creating the guidelines to receiving and reviewing students’ assessment submissions.

How?

We hear Matalena Tofa from Learning Teaching & Curriculum introduce herself and explain what she wanted her students to achieve when she decided to develop these guidelines.

Deciding on the guidelines

To decide on the guidelines she wanted to provide her students, Matalena went on a research journey that involved readings on:

• Generative Artificial Intelligence – Matalena explored a wide range of sources, from podcasts to journalism to scholarly articles by scientists, linguists, sociologists and more. In this research, she sought to better understand aspects like how Large Language Models and GenAI work, how this technology has been developed, and its ethical and environmental impacts. This exploration helped to inform her position on GenAI in her teaching and learning practice.
  
  Note: A curated list of these resources can be found in the Additional resources section at the bottom of this page.
• The process for completing assessments – Assessing ways in which GenAI could be used requires a shift in focus from the final product of an assessment (such as an essay or a report) to the process involved in completing the assessment (Hsiao et al., 2023). TEQSA’s Assessment reform for the age of artificial intelligence also proposes that assessments should emphasise and evidence “the process of learning over time and in context [to] support a better understanding of learners’ sense-making process, what they ultimately know and can do” (Lodge et al., 2023, p.4).
• Existing guidelines – Matalena also drew on examples of guidelines and responses by academics worldwide, particularly the ones that took into account critical AI literacies and the process for completing assessments. Gutierrez's (2023) guidelines, for instance, achieve this while drawing a distinction between low-risk forms of AI that are already embedded in everyday tools (e.g., spellcheck), and GenAI as a high-risk form of AI which requires guidance to support learning.
• The UOW Assessment and Feedback Policy

Introducing the guidelines to students

Matalena: “I introduced the guidelines to my students in a number of ways."

• Subject Outline: In the Additional Information about Assessments section, I included the principles for GenAI use for this subject – essentially, that the use had to be informed, purposeful, responsible and transparent.
  
  
• Guidelines for GenAI use: On Moodle, we have a specific section that includes a video explaining my reasoning for having these guidelines and the four principles described in the Subject Outline, but here in more detail. This section was also where I added resources students asked for during the semester, such as how to share a record of their chat-log when using different GenAI tools. This was also covered briefly in the first lecture.
  
  
• Assessment instructions and template: In the assessment instructions, I used the traffic light system, highlighting which uses were green (permitted without acknowledgement), yellow (permitted with acknowledgement), or red (not permitted).
  
  
   

  The image is a colour-coded table that outlines the permitted and not permitted uses of AI tools in assessments, divided into three sections:

  1. Use permitted without acknowledgement (Green background):

     • Low-risk AI tools, such as Microsoft spellcheck or Grammarly, can be used without acknowledgement.

  2. Use permitted with acknowledgement (Yellow background):

     • GenAI can be used in this assessment for specific processes, including:

       • Idea generation and brainstorming
       • Checking for grammatical errors, punctuation mistakes, and other language-related issues
       • Editing assistance for specific sentences or phrases (e.g., "what's another word for...?")

     • This use must be informed, purposeful, responsible, and transparent.

  3. Use not permitted (Red background):

     • The following uses of GenAI are not permitted to ensure the effectiveness and validity of your learning in this assessment:

       • Generating a first draft or generating assessment responses
       • Revisions and extensive editing, such as refining style and tone of your text (e.g., "rewrite this in an academic style") or translating your work (e.g., "rewrite this in English")
       • Research assistance, including finding information (e.g., "which websites or articles should I read on this topic?"), drafting summaries of information (e.g., "write a summary of this required reading"), or explaining terminology

     • Using a GenAI tool in these ways is considered a breach of the Academic Integrity policy.

  At the bottom of the table, there is additional text stating: "If you want to use, or have used, a GenAI tool in a way that is not identified here, please contact the subject coordinator before submitting your assessment."

  
   
  The template, adapted below, allows students the opportunity to acknowledge any use of GenAI tools that fell under the yellow category.
  
  
  Only complete this section if you have used a GenAI tool (e.g., ChatGPT) when writing this assessment.
  

  • Generative AI has been used in this assessment for the processes identified below:
    • Idea generation and brainstorming (Checkbox)
    • Checking for grammatical errors, punctuation mistakes, and other language-related issues (Checkbox)
    • Editing assistance for specific sentences of phrases ('what's another word for...?') (Checkbox)
  • I confirm that my use of the GenAI tool for this assessment is informed, purposeful, responsible, and transparent. (Checkbox)
  • I have attached a copy of, or link to, my use of the GenAI tool as an appendix to this assessment (Checkbox)
  
  
  

Matalena: “Once guidelines and expectations had been introduced and explained, it was time to give students practice opportunities in tutorials. We had an initial tutorial activity, before the first assessment which focused on evaluating text produced by GenAI. I provided a sample output that was relevant to the assessment we were working on. Alternatively, students had the option to generate their own output in class. The focus was on showcasing responsible uses of the tools in the context of this subject, and allowing students the opportunity to evaluate the output and check whether it is relevant and accurate.”   

Developing and implementing the guidelines with a broad teaching team 

While still developing the guidelines, Matalena consulted with colleagues in Learning, Teaching and Curriculum (LTC) and in Academic Quality Standards (AQS). She then held meetings with the team of lecturers involved in the subject to ensure that everyone understood and was comfortable with the position being taken on GenAI tools. She also met with her sessional teaching team and talked through the guidelines, sharing with them the resources available to the students. 

Implementing the guidelines

Matalena: “In terms of marking, the tutors used a spreadsheet to enter whether students had acknowledged GenAI use and any additional comments or concerns the tutors had around this use. I then looked quite closely at these because I am interested in how students have used these tools in their assessments. For the tutors, this was no different to how they would interact with any other suspicion of academic misconduct. They just needed to note it, as they would note plagiarism for instance, and notify me as the subject coordinator.”

"I really appreciate that students have responded to the guidelines and taken efforts to use GenAI in a transparent way – and that they trust the guidelines/process to be able to do that."
- Matalena Tofa

Reflections and future plans

Matalena reflects on her experience after seeing her students submit their first assessment using the guidelines, and shares her plans for refining them in the future.

Advice for Colleagues

References