Incorporating GenAI into assessments for a 3rd year subject

My name is Dr. Armin Alimardani. I teach at the School of Law, University of Wollongong, and I generally teach Law and Emerging Technologies, and Criminal Law, and hopefully in the future law and artificial intelligence. So one of the main objectives of law and emerging technologies in my subject is to prepare students for the future, kind of future-proof them. So I wanted them to learn some of the basics of these emerging technologies; and of course one of them is artificial intelligence.

So, I started with teaching them how they can use it, and more importantly what are the limitations where they can use it appropriately, or not. So if Generative AI is used responsibly, I think it can have many benefits for the students.

One simple one is students can provide text that is pretty lengthy, and they have to shorten the text so they can ask Generative AI to make the text a little bit shorter. That's one way. One of the things I really like is the students write their assignment and then feed it to Generative AI and ask it to generate 10 killer titles for their assignments. And how about we use it as a way to counter-argue our arguments, to figure out what are the holes in, you know, our arguments.

Incorporating Generative AI in my third-year subject, it was pretty stressful because I wasn't sure how it's going to look and what it's going to be. No one ever done that before. It was just kind of new to everybody, and honestly, I wasn't sure how I'm going to, you know, perform the whole thing. Everything came together bit by bit, but I knew for a fact I want to teach the students how they can use it, how to avoid their problems, potential problems because I know that's one of the area of concerns at the university and workplaces in general.

So for around 4 times every week we practice some Gen AI. How to generate things and where they can rely on it, how they should fact-check those things. And then for their assessments, I look at the assessment criteria, and one of the assessment criterion was creativity. And I thought, that would be a cool thing to use Generative AI and see whether students can use it for the creativity purposes.

The assessment’s format was in a way that they could use Generative AI as much as they can - but, based on what I taught them about use of Generative AI, they knew they must find relevant sources inside their claims that they got from Generative AI. So I wasn’t worried about that at all. I knew if there is no citations you didn’t do part of the assignment, you’re losing mark. You already know that.

But for the creativity part, and that was the most interesting part for me, that whether they can use Generative AI for something creative. Because the assignment was asking: “I want you to generate, to come up with something new. You have never heard of it before”. And many of them give it a try, and some of them figure out to find something creative - some of them couldn’t.

And the most beautiful thing was that at the end they conclude that: “I couldn’t use Generative AI for this purpose, so I decided to do my own investigation”. And that’s exactly what I wanted from students. Because it’s a tool. If it’s not working for the context, you got to put it away. Use some other tool. Some other tool was Google Scholar. Read some more papers and put the ideas together and figure out how it can be creative.

So to make sure that students, they have done a proper job implementing Generative AI in their assignments. First off, I allocated 10% to 15% of the assessment weight on use of Generative AI. And then I gave them the options: that, you might try it, it might work out, and you tell me that it worked out, and you might also try it, and it may not work out. That’s totally OK. That’s what I want from you to understand when it is useful and it’s not.

At the end of the assignment, I told them that they have like, around 3 slides in a PowerPoint, that they can explain how they went around and used with Generative AI, and just some examples of prompts they used, and if the prompts didn’t work out how they decided to change them. I didn’t want all the prompts because I knew it’s impossible to just copy/paste everything. But I just wanted them to provide me with a summary of how this whole process worked out, and whether I can give them any feedback in that process.

I think it’s our ethical responsibility to teach students at least the very basics of them – like how they’re supposed to use them and what are the limitations, how to be responsible about the use of Generative AI – and prepare them for the workplace.

But the question of how we are supposed to negotiate that with the students or, you know, kind of teaching them about Generative AI, that’s a very difficult one because the argument is some students may not learn the basic skills like critical thinking. The AI is doing the critical thinking for you right in the first draft for you; it’s taking, you know, some of the steps – and the question is whether students actually learn how to criticise Generative AI’s output to make them better. If they haven’t learned the very basics of it, how they can do the more difficult job of improving them.

And I’m not sure about this, but I think at this point, for me at least, it’s going to be like: I’m not going to use Generative AI in my first-year classes. I’m going to make sure they’re going to learn those basic skills, and then from second-year and to third-year, we’re going to transition and we’re going to implement more and more Generative AI. We’re going to even generate answers with Generative AI and ask them “How can you improve this?”

Because again, we believe in the near future companies, that’s how they’re going to function. They’re going to have the first draft of everything is in the contract. Is it a – I don’t know – computer program? And the future employees, our students, are supposed to make them better. They’re supposed to figure out what are the gaps with the problems and how they can improve it.