Introduction

Hi, my name is Mika and I'm going to start by... First of all, can you hear me? Because I decided not to use a mic. Yeah, great.

The Power of Coaching

I'm going to start by telling you just a little bit about what we do. So, I'm going to start with a question.

When someone comes to you with a challenge, what do you instinctively do? How do you instinctively help? Most of us jump into trying to give advice or solutions or really rather telling people what to do.

And that can be frequently okay if you actually are a subject matter expert, but sometimes and very often that can actually be very damaging, especially if you do that consistently and if you're in a leadership or managerial position. If you keep telling somebody what to do that means you're not really helping them grow and we're not really helping them develop their critical thinking and also they're less likely to take ownership and responsibility and accountability for whatever task comes out of that solution.

So there's a different way of helping somebody when they're coming to you with a challenge which is helping them solve their own challenge rather than solving it for them. And that sometimes means that we're sharing thoughts and experiences and so on, but it means that we're not telling them what to do, but we're actually still maintaining the sense of agency and we're helping them be responsible and accountable for whatever solution they're coming up with.

Why Coaching Matters in Leadership

And that's called coaching and it's increasingly shown to be an essential skill for leaders and managers to the extent that there's recent research that shows that employees who have a coach-like manager are eight times more likely to be highly engaged compared to those who don't. 1And the good news is that coaching is a rocket science. All of us can develop that skill.

The bad news is that it requires behavior change, which means that it's easier said than done. And that's why to be a professionally trained coach, usually you go through long training processes and you do a lot of practice and you're required to do over 100 hours of practice.

Our Coaching Solution

Now, this is where we come in. We want to make it easy. We make it easy for managers and leaders to have coaching conversations. And the way we do it is we'll take a group of participants, we'll pair them up with each other, we'll give them an experiential workshop, and then we'll onboard them onto our digital platform that acts like a scaffolding to help them have effective coaching conversation from the get-go.

Incorporating Generative AI

And so today I'm going to a bit of show and tell and show you how we've been using generative AI to make our product more effective.

So as I mentioned, we started with this experiential workshop that culminates into a cheat sheet that effectively distills the, I guess, how to have an effective coaching conversation into one page. And it has these two dimensions. And the reason why I'm going into detail is because I'll show you how we're using AI to help develop that over time.

So The first dimension is the key coaching techniques, so reflective listening, repeating back what you've heard, asking insightful open questions, and sharing thoughts in a non-directive, non-advisory manner. And the other dimension is what we call our coaching conversation arc, how to actually have a coaching conversation from A to Z in an effective way, that you start with somebody who's got a challenge and you get them by the end and they say, right, I've got some next steps and I know how to move forward.

Now we started with this generic cheat sheet and what we did next is to expand it to a concept of coaching pathway. So we looked at all the most common coaching topics, whether that is somebody who is looking at where they want to be one to three years from now or struggling to motivate their team or perhaps preparing for a difficult conversation or making a difficult decision. So we took some of the most common topics and we created what we call our coaching pathways or effectively a sequence of coaching prompts that were done by our professional coaches.

And then we created a very simple prompt to chat GPT and we said, here's 20 potential coaching pathways, now here's 10, 20 other topics, now generate them for us. And then we continuously iterated on that and created about 100 of these coaching pathways. So fairly simple kind of use of of AI.

And then what we did is we created a user friendly, I guess, interface for two people who have no prior experience with coaching to coach each other. And that's what we call our session runner screen. So effectively, you can run your session through this screen, you can have video in the app and so on, and the app will help you and take you step by step as coach and coachee through how to have that interaction.

Enhancing the Coaching Experience with AI

And then we've basically brought in AI elements where we felt that the user needed more support and where we were providing more human support. So this is, I'm going to run you through the experience of the coach and the coachee.

as they start the session, one of the most important things for a session to be effective, the coachee needs to have a clear topic that actually you know, make sense to them is going to make a clear impact to their happiness and performance and they feel confident through the session that's a good topic for them to talk. So first we get them started with a brief exercise of noting down some key reflections, you know, asking them what's keeping them up at night and that really gets them going in terms of, okay, these are more or less the topics that I'd like to talk about. But what we found is that people are not I'm not confident and this is what I want to talk about and I'm not really kind of sure is this really the most important topic for me. So what we then did was actually create a very simple bot, an AI coach assistant that helps them through a conversational based approach identify the key coaching topic.

And what that looks like, and I'll show you the prompt. So we created it, again, it was our team of internal coaches that looked at what do we do when we have a coaching conversation. And we use the, this is the interface, the OpenAI Playground interface that helps you iterate on that prompt and continuously iterate it.

So the structure that we follow is making sure that we're always telling the AI who they're acting as, so your highly experienced professional coach, what is their objective, and then we give them a task give it a task, and we split it in very small steps. And we continuously iterated. Probably at the beginning we started with five steps, and it's now about 12 steps.

And we continuously iterated on those steps as we saw where the AI is working or not working. So for example, in this case, we kept telling it, do not move on to step five until they have confirmed that you have understood correctly. Because the AI was just jumping the gun.

And interestingly, that was working as we were testing it in the playground. And then we deployed it into production. And what we realized is the AI kept jumping and just quickly, not waiting for you to say, yes, you've understood correctly, just jumping to the next step. So we were like, OK, what's happening? What's the difference between what's happening in the playground versus in production?

And what we realized is that we were testing it in the playground with ChatGPT. And then in production, for some reason, our CTO decided to go with ChatGPT for Turbo, which is the more, I guess it's a different version of GPT-4 that is designed actually for speed and scalability. And it's kind of cheaper. And then I had a chat with ChatGPT, and I was like, right, we have these differences. What's the issue? And it actually kind of gave you the situation. It said, look, that's designed for speed and for scalability and cost efficiencies. But actually, if you want to have a more meaningful, in-depth coaching conversation and so on, then you're better off using ChatGPT, using GPT-4, which was really interesting, like a very small nuance. So that's kind of one of the, as I mentioned, that's one of the ways that we help the coachee.

The other thing that we noticed was that people were having these reactive, they were identifying reactive coaching topics. And they were saying, actually, I want to think through more proactively. Where do I want to be two, three months from now or a year from now? And so we introduced the concept of a coaching focus.

The idea is actually helping you identify where do you want to be two to three months from now and then making sure that your session, session, session, you're focusing on that coaching topic. And initially what we did is we had these 15 minutes call with our customers and we were helping them identify what their coaching focus was. And then again, we understood that within 15 minutes, we can help somebody identify that coaching focus.

So the next thing that we did was to create an AI coach assistant that helped them identify their coaching focus. And again, very simple prompt. I mean, I say kind of like simple, but it's a fairly kind of like long prompt. But again, we iterated on that similar, you know, I guess structure, you are highly experienced professional coach, this is your objective, and these are some of the steps for you to follow. And I guess what I'm trying to say is that it's not rocket science, it's just iteration.

Supporting the Coach

So next thing, I guess, to show you was more how do we help the coach through this experience? So for the coach, what's really important is for them to feel, A, prepared ahead of the session, B, to feel like they've got the confidence through the session, they've got the right prompts and they've got the right feedback during the session, and C, at the end, to get more in-depth feedback on how they did so they can do better next time. So again, remember, this is somebody who's got no prior coaching experience.

So what we did here, we thought, OK, at the beginning, to get them prepared, first of all, we broke down the content from our workshop into small chunks and bite-sized lessons. And we created these short videos. And we then created, again, AI-powered quizzes to help them reflect on the content that they've watched and to do some simulations actually to practice.

Again, I'll show you using, we did that, we iterated using GPT-4, the playground. So we created a prompt that, in this case, invited them to have a simulated conversation with the AI who was acting as Gochi. And again, broke it down in very small steps.

And towards the end, we gave it the script of the video so that they are aware of what was taught in the video. In some cases, actually, the AI knows quite well what coaching is and how to provide feedback. But in other cases, we need to be very prescriptive in terms of how it responds. So that is, again, one use.

1Then next, what I mentioned is that we needed to prompt the coach to help them really have that coaching conversation with confidence.

And in some cases, we realized that we didn't need AI. It wasn't overkill.

So we just simply created very static prompts, such as in the first place, when you're starting a session, ask them, what are some of your reflections since the last time that we met? And let me recap what I've had. Then again, very static prompts, let's clarify the topic, let's set a good outcome for the session.

As we get further into the topic, then this is where we need more nuanced coaching prompts and more nuanced, I guess, ways of interacting with our coachee. And this is where we brought in that concept of coaching pathways and we used the concept of vectorizations and embeddings to match the topic and the outcome that was effectively brought to the session with one of our coaching pathways and to effectively bring in those coaching prompts that I was showing you earlier when I was showing all those coaching pathways.

And the next step for us is to actually not even recommend the coaching pathways, but actually feed all the coaching pathways to the AI and then get it to actually just recommend very nuanced coaching prompts depending on the topic and the outcome is.

Future Developments in AI-Powered Coaching

So, what I'm actually really excited, I feel like we've just scratched the surface, what I'm really excited about what's coming up within the next couple of months is that real-time feedback during the conversation, nudging the coach on to things like, you know, remember to set a good outcome or, hey, you've set a good outcome or, you know, do some reflective listening or you've got 10 minutes left from the conversation, start wrapping up and so on.

And last but not least, actually providing more in-depth more in-depth feedback on how they did in terms of the coaching conversation. So actually providing them with an overview of how effectively they coach the other person and things like looking at the quality of their questions and looking at to what extent the questions were open versus closed or to what extent they were asked curiously or in a judgmental way.

Again, how we're approaching this analysis is through a long series of a lot of prompts that we've broken down in order to help the AI both classify things and provide recommendations.

And I'm actually really excited because when we started doing this work, we were using transcription with a tool called DeepGram and then the analysis, whereas now with GPT-E4O, we're able to jump straight into doing the voice analysis. So I hope

Conclusion

this was insightful. I'm going to stop here.