An introduction to how gen AI works and prompt engineering

Armin Alimardani: So in this brief session, I'm going to talk to you about how generative AI in general is being created and how it is being used. What's the process when you're using it and how it just spit out words. Understanding this would help you to also identify the limitations of generative AI and figuring out what are the best ways to use it in the context that you need.

The first step in creating generative AI is data. It's all about data. They take data from everywhere, from Internet. But the point is, it's not the whole Internet. It's some parts of the Internet that it has access to. For instance, in this table, you can see when it came to GPT 3, the model before GPT 3.5, that is ChatGPT. These are the sources they use to train the model.

Of course, there are so many important things that they are not in there. And if you ask any questions, either it's going to say I don't know, or it's going to give you some nonsense. So if you take 10 terabytes of text, 6000 GPUs, you run it to create generative AI, it would take at least 12 days. And it will cost you $2,000,000 to create a very small size generative AI.

So this process is expensive, it's lengthy, requires so much resources. That's one of the reasons you don't see every company or any person try to create generative AI. But the fact that generative AI and the training relies on the data that we are giving, it tells us something very important. You should not use generative AI as a knowledge database. They're more like reasoning machines. They're more like a person who understand the language.

They have some understanding of the history, some scientific stuff, but not everything. Things that are very obvious to us, they may not know them. You should always try to ask them questions that they use their reasoning and one of the best ways do that is to provide the content for them. Say here's the content, can you summarise this for me? Here's the content. Can you write it in a way a 16 year old can understand?

So that's the way to use generative AI as a reasoning machine. But you might say, I know of some generative AI's that are connected to the Internet. How about those? That's true. They're better in terms of figuring out what's relevant, what's the relevant data and how to answer your question. But just think about the process it goes and check couple of pages. We don't know exactly how many. And maybe the information you're looking for is not in those pages and it puts all the data together and summarises them and give them back to you.

And we don't know whether they misunderstood your question, whether this is actually what you're looking for, or whether they are just making things up because they couldn't figure out exactly what's the correct answer. So it still comes with some limitations. And again, you always must check the output and make sure that what you got is accurate. But I said it's a reasoning machine. It's not the best reasoning machine. It just makes stupid mistakes as well. Here is an example of a math question that it got it wrong and many people are a calculator can do that. How this huge generative AI cannot do it. There are very different systems and they don't work in the same way at all.

But the point is it can get very simple things wrong. And here is another example from a TED talk that shows it's really bad at reasoning machine even though we have provided the content. But when we see these examples, companies quickly go and try to fix these matters and resolve these kind of stupid mistakes. But what happens with training? What does it mean? We feed it data. So we have a basic algorithm. Usually it looks at the data and it tries to build opportunity value with it. The starting point is that we have this dictionary for a generative AI.

Usually they have 50,000 words. In other words, what we do is we transform each word in that dictionary into a number. So when I say a tree, it has a number, let's say 395. That's just a random number. So for generative AI, all the words in the world, they are represented in form of number. For instance, as you can see in this example, each word is transferred to a number. So we never give actually the word to the generative AI.

It gets the number and what it does with them. Let's say we have a word tree and we have water. The generative AI, based on how commonly these two words were found close to each other, creates a statistical correlation. So the statistical correlation between a tree and water is one that's the highest. But the statistical correlation between Europe and Spoon is 0 because they don't come next to each other. So that's what happens in the training. The AI learns the correlation, the statistical correlation between all these words in the dictionary that it has. And as a result, we have this condensed information in one place in a model. It's not like a pool of data, generative AI.

It's not like it looks at its pool of data and say, OK, I want to find this information. It looks very different. It's like a messy network. It's not like sentences are being written somewhere and it goes and read them. It takes the words and it puts them in the algorithm and some other words comes out at the end. So in other words, what I'm trying to say is when you ask a question from generative AI, it doesn't go and find the text and copy and paste it for you.

It generates them, generates one word after the other. So it makes a statistical analysis based on the trained model and generates the most probable word or number after each word. Here's an example. We say "each word is transferred to a... " and we just leave it blank and we want the generative AI to answer this question. Fill the blank for us. If you notice, there are these arrows on all these words. And what I'm trying to say is generative AI doesn't look at only the one or two last words and look at the whole context.

It goes back and look at the whole text and say, OK, now I understand the context. And based on the statistical correlation between all these words, it goes on to predict what's the most probable word to come after that. So it goes back to its dictionary and look at all the words in the dictionary, 1 by 1. Let's say the very first word is a gorilla. And according to the training data, gorilla doesn't come close to these four or five words that we have in this sentence. So the likelihood that gorilla fits this sentence as the next word is 4%. Then it goes to the next word in the dictionary number and the likelihood is 93% goes to the next one wireless, the likelihood is 1%. And it does it for all the 50,000 words in these dictionaries. And then it sorts them from the highest to lowest probability and we have numbers at the very top with 93% and it puts it in the blank. This is how it generates word by word. And This is why prompt engineering is important, because changing one word in that context can change all the statistical, the weights between all the words and what you would get afterwards.

Now I just want to remind you again that generative AI, it doesn't see words when it does the whole process to better represent it. You can see this image of a computer game, and this is how we see this computer game. We see a couple of characters and trees and stuff all around. And this is how an AI sees it. It sees all those numbers. So what it tells us, it tells us that technically it talks in numbers, it's math, It doesn't understand the way we define understanding and it doesn't do evaluations or reasonings the way we do.

But at the same time, it's not the best argument because if we look at our brains, which part of this interaction is a word or a number or anything that's communicating, it's just meaningless to us. We are in some kind of similar situation. But we all can agree that our brain and generative AI, they use very different functions. Now, in this example, I showed you how generative AI finds the most probable word, but the truth is it always selects one of the most probable words randomly.

I want to show you that in Practise in action, I am using one of the older models that would show us the probabilities and the input is write an essay about justice and the question is based on that, based on like the terms right essay, justice and so on. What would be the most probable or probable words afterwards? So when I click hit enter, I get this whole essay and I can click on each word and see their probability. For instance, the very first word was justice. If I click on justice it shows probability of 95% and that's the highest probability and it has selected that. But it's not always like that. I can click on another word and the probability is 2.45%. The bottom of that table. As you can see, it's not always the most probable word.

Every time it randomly selects a different word, why is it important? It answers the question that why every time you ask generative AI a question the exact same question, you get very different answers. It's because there is that randomness every time he uses that random selection and you may start a whole conversation with a different word and whatever happens, the subsequent of it can be all very different.

It also suggests that this is not the best reasoning machine because it uses randomness to select the next most probable word and you don't want to rely on this machine that randomly selects the words one after the other. It also begs the question that whether we humans are rational because there are so many studies that question humans as rational beings.

We do lots of irrational behaviours and the, my point here is that it's not OK just to criticise whether it's AI or anything else without reflecting it on ourselves. We, if we want to distinguish these two different systems, we are going to do it properly.

So in summary, I discussed how we train generative AI on data, what happens there? And that's what we call pre training or pre trained. And then I talked about how each word is being generated or predicted and that's what we call generative. And then I talked about how it can consider context. It means not only the last word or last two words, it considers all the words before and put them together to understand the context and predict the next word. And that's an algorithm called Transformers. And now if we move these 3 terms on the right side around that is pre trained generative and Transformers, we will get GPT. And these are the fundamental elements of any generative AI language model.

Armin Alimardani: Well hello, in this short video I will tell you how to use generative AI effectively and responsibly.

You may use generative AI for a variety of purposes like understanding complex concepts, generating counterarguments, summarising things, brainstorming, a lot of things. So in general, it can be a very useful tool if you use it properly. Now I'm going to use ChatGPT as an example to show you how you can use generative AI.

The first thing is, whatever you write in this box is called prompt. So if I say "hi", the prompt is "hi", so whatever goes in the box is prompt. Now imagine I asked you how is the weather today? And you say good. And what I meant to ask was that what is the highest and the lowest temperature throughout the day and what's the weather condition, whether it's windy, rainy, it's gonna be sunny.

So I didn't clarify my intention by saying how's the weather today, right? Now I engineer my prompt from how's the weather today to: "Please tell me how the weather is today, including the lowest and the highest temperatures and the weather conditions throughout the day". In other words, we have to very clear with our intention to do this, you should simply ask yourself whether I have provided enough information for the AI to make my intention clear. So that is called prompt engineering, making your intention as clear as possible by providing information.

Now let's do a more realistic example that is kind of useful for you. Let's say you have a paragraph, and you want to edit it to make sure there are no grammatical errors in it, right? Your prompt could be simply "Edit the following paragraph", and then you copy your paragraph.

But, if I want to engineer this prompt because it's just so simple, and it's not clearly showing what's my intention? How would I do that? First, I should ask myself who is the best person to edit this paragraph? An editor? Probably. You can say, "Imagine you're a skilled editor. Your task is to edit the following text." But hold on. If you give a paragraph of text to an editor, there are going to ask you what is it about, What's the context? Because they want to know, for instance, if they want to change a word and use a synonym that is better, whether it fits that context or not.

Let's imagine the paragraph that I'm going to copy. It's about defining criminal justice. I want to say this editor knows about criminal law. So this time I'm gonna change my prompt to "Imagine you're a skilled editor with a deep understanding of criminal law. Your task is to edit the following text." and I ask myself whether this prompt is clear or not, and the question that comes to my mind - what do you mean with editing? So once again I'm gonna remove this, and I'm gonna copy what I think is a better one. So instead of editing, I say: "Your task is to correct the grammatical errors and enhance the expression". So that's what I mean with editing. But one problem that may happen is that this editor, because they have a deep understanding of criminal law, it might introduce new information and if it is a task that I'm supposed to do it on my own, then one day I to treat information occasionally at the very end don't introduce additional information.

Now when I want to copy a paragraph, just to make sure the AI understands what is the text they're supposed to edit, I add 3 quotation mark and I'll copy my text. And this is not the best text to copy, I directly got it from Wikipedia on the definition of criminal justice, and then I hit enter. And, it happens a lot to the best of us that this is not what we wanted. Instead of going back and copy the whole thing and redoing it, If I hover my mouse, I can see this pencil here. If I click it, I can edit it. Let's say I just want to test, remove this and save and submit. And then you redo it. Then, I can click here and compare which prompt worked out better for me. Let's say I don't like the tone. I might say that "Rewrite the edited paragraph, make the tone more academic". It says right, so you can do lots of different things, and you can just back and forth have a conversation.

Now some people may say that when you want to start a new conversation, and you're done with this paragraph, and now you want to do something else, you might simply write, forget all the previous conversations, and then you start your new prompt, right? Wrong, don't do that. Even though it may ignore the previous conversation, you are filling up the memory of this generative AI with the information that you don't need anymore. The easiest way is up top left. Click here, and you get a new chat and start over. Now another quick example. Let's say this time you want to summarise something. What would be my prompt? First, I have to explain who is this person? What is their role? I will start with something like, "You are an expert in something", whatever is the contents of the text I don't want to summarise, and I would say whatever is the task, "Summarise the following X whatever is the content. And let's say this summary is intended for a specific purpose. You say "Summary is intended for like a specific audience". You write these specific audience to get the tone right.

Another thing is when you're summarising something, let's say is 1000 words, you wanna know how much you want the summary to be. You might say, "The summary shouldn't be more than 300 words". Generative AIs, many of them do not do that. Following this intention then say yeah, I wanted to be maximum 300 words. A work around that usually works well is to say how many paragraphs shouldn't be more than three paragraphs.

Another example again with summarising, Let's say you want to learn about Heisenberg's Uncertainty Principle and you have no idea what it is. So the trick is, you ask the AI to summarise it in a way that you can understand. What you can say is I've already written it, "You're a high school teacher. Summarise the following texts on Heisenberg's Uncertainty Principle. This summary is intended for students in year nine, therefore it must be easy to understand and engaging. The summary shouldn't be more than one paragraph long". And again I'm gonna copy something from the Wikipedia directly copy paste and then I hit enter. You can see it is talking in a very casual way, because you intended for high school students. Now you can ask, "Make the tone more professional", for instance, or more casual, or make it so simple a six-year-old can understand. So you can play around this and make any changes if you want to.

Another cool thing you can do is, let's say you want to apply for a job and there are the selection criteria and you have your own information but you don't know how to write it. You want starting point. You want the ChatGPT to write the first draft for you and then you want to revise it. So you write your prompt, whatever it is, "prompt", I'm not going to write the whole thing. Here, the trick is to write like selection criteria, so you have already explained that you're an expert in whatever it is and you want to help users or people to apply for jobs. Use the selection criteria and the applicants' information to write a cover letter that shouldn't be more than 4 paragraphs and you can again adjust it to make it as good as possible. So you write the selection criteria and then information and you copy the selection criteria here, whatever it is. And you write your own information and you will get something.

Prompting engineering is a skill. Whoever says there are specific ways to do the prompt engineering and it will always work, that's incorrect because they are constantly changing these models and there are new and different approaches to do prompt engineering and get better outputs. It's just you gotta play around and trust me, you will learn pretty quickly.