Introduction to Live Demonstrations

OK, we're going to continue now.

So I'm going to try a live demo, like Joe, although I'm doing it on OpenAI stuff, not on my own stuff, which is always risky.

And as we, I don't know how many of you are aware, but basically when you do live demos, the later you are in the day, the less likely they work, because the US The US comes online and OpenAI servers are actually struggling.

You can literally look at this for your own usage. At the end of the day, the APIs will respond slower. ChatGPT will respond slower. All of this happens because you have a big part of the US that's using the same thing.

Creating an Interactive Game with GPT

So what I'm going to try and do is create a GPT that you can feed any content to. And it basically, on the fly, creates a game out of it that helps you learn whatever that content is. I'm going to do that direct.

So how many of you have already created a GPT once? Good. That means you're all on Tad GPT Plus. And that was like a third of the room.

Get in there. I've asked this question so many times that I can literally plot the open AI usage across the world at this point.

So I'm going to create a GPT here. And what am I going to call it? GPT tutorial game creator.

Description here is going to be very simple. Create a game that teaches me the topic of any resource I feed it.

And then I'm going to cheat because I have a template that I use for these demos. So I will explain a little bit what it is.

Let me see. Can you kind of read that? I'll try and see if I can zoom in a little bit. There. That works, right?

OK, so I've got a few things that go into a GPT template. And this is literally what I do now. Weirdly, this has become my life somehow.

Designing the GPT Template

But there are two things you can start. You can see I always start a GPT with you follow the process and respect what's important. And I end it with you follow the process and you respect what's important.

Does anyone in the room know why you would start and finish it in this way? Alex does because he can cheat because he actually went on our program.

The reason is that these models, the more context you give them, the more they have a propensity to forget stuff. And the stuff that they forget is the stuff that's in the middle. So the way that I fix it is I tell it at the start and at the end that it shouldn't forget what's in the middle. Which is literally just a hack, and some of these things are just absolutely crazy that they work, but I can guarantee you they work.

And then there are a few other things, like you might find the bottom bit interesting, which is the, you might be familiar with chain of thought reasoning, so you take a deep breath before, sorry, you outline your thinking step by step before answering a query. This is one way to get much better answers out of LLMs.

There's a technical reason why this works. Does anyone know why chain of thought reasoning works? No.

OK, so chain of thought reasoning works because these models work at predicting the next word, or sorry, the next token, which is somewhere in between a few letters and a word. 1And so when it produces its first word, that becomes part of the input for the second word and the third word and the fourth word and so on. 1So if you get the model to outline its own thinking before giving you the final answer, that entire thinking process becomes part of the input for the final answer you're getting, which means you get dramatically better answers out of it.

I know it is weird to think about technology in this way, but it actually works. It reduces hallucination, improves accuracy, gets you dramatically better reasoned answers, and so on.

Most people will do this the other way around. They will ask a question and then ask, why did you give me this answer? Wrong way around, because then you get the bad answer and it is going to give you an explanation as to why that is the right answer, even though it was the bad one. So that's the hallucination problem you run into.

And then there are a few other things which are weird, but also work, which is taking a deep breath before answering gets you better answers. Giving it a tip for great answers gets you better answers.

I know I shouldn't say this, we're in a law firm here. When I do this session for lawyers, they're like, but what if they're going to claim for this one day? It's like you told them you're going to give them tips. And then the other one is, yeah, if you emotionally charge it, you basically say something bad is going to happen when you give me a bad answer, you will get better answers out of this.

There are a whole bunch of different papers on this, so you can look up these different techniques. It's not me making them up. The one I use is if you do a bad job, I will lose mine. But you can use any other kind of emotional charging. And there are certain reasons as to why these work like this.

Okay, so what I'm going to do here

Setting a Persona for GPT

is first I'm going to give it a persona. So I'm going to say, imagine you're an experienced learning or game learning designer.

I'm not sure. I know these are two different jobs, and I'm trying to put them into one different thing.

And I'm going to say the goal of this particular GPT is to produce an entertaining, engaging, and educational experience on a particular topic.

The context, very simple as well. I'm going to say you are fed a resource from which you should produce the experience.

And then I go through the process. So how would you go about something like this?

One, you can say ask, or if I didn't tell you, ask me for the resource on which you should build the experience.

Two, use your browser to navigate to the resource in question and absorb its contents. So in this case, I'm actually going to try browsing.

And so if that doesn't work, I'll revert back to a PDF. But we'll see what that does.

And then three, I'm going to say build an experience that we can play together turn by turn, which follows the, I'm going to say, guidelines. And then here, I'm going to cheat again.

Incorporating Key Learning Criteria

Actually, so in the audience, what do you think are some of the key criteria that make a great learning experience? Any things that should be taken into account as we're thinking about these types of experience? What makes for great learning?

Interactive, okay. Should be interactive.

Sorry? Sense of empathy. Sense of empathy.

Okay, it should include, just making this a little bit more modular, interactivity, sense of empathy, anything else? Feedback. Feedback, okay.

Okay. Immediate feedback loops. I'm going to keep it at that, maybe, and we'll kind of figure out where we go here. Yep, thank you.

OK, so now I'm just going to close this. And what I have as well, I forgot that I just did this. See if my copy paste function here works.

I'm actually going to save this for a second so that I will go full screen. OK, and then I'm going to zoom back in.

I'm going to say build me an experience based on the following resource. Obviously it didn't work. Okay. Well, you will see a thing here. There we go.

Okay. So I'm going to just zoom out and give you an idea of what this actually means.

So this resource is just a website, the future of M&E, how AI development is revolutionizing development, and it goes into a whole bunch of different things. Just so you see what's happening here, it's basically lining out what this resource is. So it's a resource that talks about enhancing efficiency, scalability, personalization in talent development, highlights benefits like ROI, virtual coaching, interactive simulations, gamification, also points out challenges, but really the whole thing.

I actually haven't read the resource. I just thought it was an appropriate one that we could look at. And the whole thing is that I shouldn't have to know, because what this GPT is going to do is going to crawl this resource and then build a game out of this. So we'll see what it does.

I mean, this is literally live, so I don't know what will come out of this. Let's see. OK.

Demonstration: Building a Learning Game

It now says gaming setup. You're the head of L&D at a futuristic company that's at the forefront of integrating AI into talent development. Your mission is to leverage AI to enhance the learning experiences.

Okay, so gameplay scenario challenges. You'll face a series of scenarios where you must make decisions on how to implement AI in your training program, test your knowledge on AI ethics, data privacy laws, and so on. Feedback loop. After each major decision or quiz, you'll receive immediate feedback on your performance and empathy, empowerment, first challenge.

You've decided to introduce an AI-driven personalization learning platform. How do you address employees' concerns about data privacy and ensure their information is secure? Option A, implement a strict data privacy measures and transparently communicate these to employees. Limit AI use to non-sensitive training areas to avoid potential data privacy issues. Offer an opt-out option for employees not comfortable with AI-driven learning.

Interesting, all these three are actually... I thought there must be an obvious wrong one and I was obviously going to choose that, but which one do we go for here? What is the decision? Implement the strict privacy measures and communicate them? Limit the AIs to non-sensitive things or just allow people to opt out? Sorry? Limit?

Yeah, okay, who says A? Okay, who says C? Okay, who says B? So it's mostly B with an exception of C. Interesting thing, personally, I like C only because it allows you to show what it might do. So people that want to opt out, like it basically shows you what you can do, whilst B is kind of, it's stopping you. But we're gonna do with B. Okay, let's go with B.

strategic decision that balances innovation with privacy concerns, immediate feedback. While this decision safeguards against immediate data privacy issues, it may limit the effectiveness of personalization training programs. Employees could miss out on tailored learning experiences that AI can provide, potentially affecting their engagement. Okay, so it then goes on, gives you a whole bunch of explanation. It goes on to the next challenge.

As you implement AI in non-sensitive areas, you notice varying levels of engagement among employees. How do you ensure that AI-driven tools are effectively meeting the diverse learning needs across the workforce? Option A, gather feedback rarely from employees to continuously adjust and improve the AI tool. B, implement advanced AI analytics to monitor engagement and learning outcomes, making data-driven adjustments. They use data-driven adjustments. Now everyone is skewed. It's going to be B.

Option C, create a mentorship program where employees can share feedback and learn from each other's experiences with AI tools. Who is for option A here? So gathering feedback and then improving. Okay. Who's for option B? Implementing analytics and making data-driven solutions. Okay. Very good. I know I kind of skewed that one almost by the way I was talking about it. Option C, creating mentorship program where employees can share feedback and learn from each other's feedback on these tools. Good. That seems to just about edge it in this case. I'm not going to continue with this. You can see how quickly it created this game by literally just giving it a resource.

Audience Participation: Choosing a Topic

So instead of doing this, I'm going to ask to this room, now, what topic should we get this GPT to create a game on? Anything. Any topic. What are you excited about learning?

Future of marketing. Future of marketing. OK, we're going to go and do that. the future of marketing.

So I am now going to take, I have no idea what this is, but it's an article that has some content about the future of marketing. The reason I'm doing it here based on resources, you can build this GPT and give it a PDF. Here I asked it to take a website. It can be your internal resources.

You have a PDF that explains a particular product that only you know about. How is it positioned? How do you counter position against competition? What are the unique benefits?

Game Creation Based on Audience's Choice

It can literally just create this on the fly. Now, create a game using this website. Okay, future marketing trends 2024. You are the CEO of a startup aiming to become a top player by 2024.

To achieve this, you must navigate through various marketing challenges based on the trends outlined on the website. These include integration of virtual and augmented reality. Interesting, I guess that is a trend. Sustainability and ethical marketing, AI-driven interactions, all make sense.

Gameplay mechanics. Startup selection. Choose your startup's focus area. Marketing challenges.

It's now going through. Obviously, all the things you have to think of. Impact meter. The impact meter shows the impact of your choices on brand popularity and sales.

Feedback loops. After every decision, you get a feedback loop. Okay, let's start with the first challenge related to integration of virtual and augmented reality marketing. Imagine you have a product that could greatly benefit from immersive advertising.

Your options are invest in a VR campaign, showcasing your product in a virtual world, creating an AR, so an augmented reality experience that users can access via their smartphones to see your product in their environment, or focus on traditional marketing methods due to budget constraints. I mean, realistic choices here, right? OK, so what do we go for? Option A, invest in a VR campaign showcasing the product in a virtual world.

Who would go for A? Sure. OK. Who would go for B, an augmented reality experience that can go to everyone?

Well, don't even have to ask C, because B basically got the majority of the room. OK, so let's go with option two. Also, a beautiful thing with LLMs is it doesn't matter if you make typos. It actually does understand.

Well, normally it does. This is the load factor of ChatGPT in the US. It's literally just taking longer time to answer, but it will come through. It should.

Feedback and Impact Assessment

Have you also used other GPDs, like Google, Gemini, and the like? DAVID KWONG- Yeah, so they actually don't have a GPT creator. That's one of the massive benefits right now that ChatGPT has that none of the others have. Yes, OK, so here, choosing the AR experience is a strategic move, reflecting the trend towards more interactive and personalized marketing.

Immediate feedback, budget impact moderate. I guess AR is easier to build than VR today. Popularity impact high. OK, I actually didn't think this was too good.

VR is also restrictive use. I would have thought VR has an audience that's much smaller than AR because AR is phones. It's a shame that it didn't actually point that out. But you can see, I just threw a website at it that I literally randomly picked from the internet here.

Conclusion: The Impact of Live Demos

Yeah? How much of the response that it is giving is coming from that website and how much is coming from its own model? Is it all from the website or is it... That is an extremely difficult question to answer.

It is restricting it based on the content you're giving it, but obviously the entire formulation and its understanding of what the website even means is based on the language model that's been built. So it is definitely a combination. I would even say it's probably 90% its own model, 95% its own model, and then 5% the way you're actually restricting it to only talk about that particular topic.