Introduction

So the talk today is really about how we're going to scale productivity with humans and agents. Now, with agents we all realise that we can scale productivity in a massive way, but the human part of it is actually equally as critical. And we're going to kind of talk through that.

If I just go to the next... I'm just going to talk through it, but I'm going to just intro it and then I'm going to hand over to Avik and he's going to take you through some more details around it and then I'm gonna end it with an example that we're

Why Scaling Productivity Needs Both Humans and Agents

A Personal Starting Point: The “Warren Buffett” Agent Idea

going to show you and so how I got involved in this with with the founder essentially we were talking about agents and building agents and we were testing it out and then I said well I'd love a an agent that could invest for me like Warren Buffett so I want to copy Warren Buffett and I want to build an agent and have that agent running all the time and making these wonderful investments for

me and that's beyond my skill level but what I'm doing is is actually almost like capturing the cognitive makeup of Warren and putting it into an agent and then as we had this conversation we're like okay well what does good look for

an agent we talk about agents we talk about skills talk about all of this and we can even build agents but what's good what's a good agent and in actual fact then when you expand it a bit more you've got agents and then you've got teams of agents so what's good in an agent so that's kind of how we started with this um and then the first step what we had to do was really before we get to

the agents in an enterprise context or an organization we actually like to figure out what's what's good within our organization as in what's the level of the employees capabilities within the organization and then we can start to look at how we pair them with agents and then formulate teams organizational redesign and really get the outcomes that we're expecting to see in in the horizon so that's kind of what we're going to talk about so I'm going to pass

over to Avik he's going to take you through the detail of it and hopefully if we have time I'm going to take you through a little example of how we've scaled this productivity and as it's an ongoing project but we'll talk a bit more about at the end as well. I'll hand over to Avik.

What We’ve Learned So Far: A Structured, Measurable Approach

Hi, Avik Mukherjee. I know Rana and the CEO of Polonise since we were 14, which explains explains why he has no hair left I want to say that all we want to do really is

to show you what we've learned so far we're a new startup so there's obviously we haven't been around for ages so there's a limit to what we've learned given the time we've been in it so we'll go there I want to talk about

Defining Productivity: Speed Plus Quality

productivity before we get into it I want to say productivity isn't just about speed right it's about speed at accuracy that you define as what you define as quality so that's what I want to leave you with before we get into it

Why Capability Building Must Be Data-Driven

what we're finding is you can't you can't guesswork it you can't have opinions you need to have a data -driven approach why what I mean by that do you Do you want to click on that one?

Is that what we do is we build capabilities, or we accelerate capabilities of teams. We build capabilities, and we accelerate the capabilities of teams.

The way we do that is we do training in a simulated way, actual work. That actual work means a lot.

what we're finding is we need data to make sure that the work is as close as to the real world as possible without that you don't get the you don't get the benefits that you're looking for that's that's one of the big things that we

Training Through Realistic Simulation and Reinforcement Loops

find do you want to go to the practice makes perfect so part of that training that we do is a reinforcement loop. So some things we know from before, it's an old story and some things are new.

This is an old story. Practice makes perfect. It does and we've seen that people who do these reinforcement loop trainings over and over and over again accelerate in terms of their ability to handle that

and we measure that at the capability level so for every training that you do we have set of capabilities and we'll measure you to a benchmark to the capabilities and we've seen people exceed the benchmark as they do more and more upscale yeah so what I mean by uplift that scale is in an organization

Uplifting at Scale to Remove Bottlenecks

you have many groups. Often it's a very tempting thing to limit spend because it is an OpEx, so forth, to do patchwork in terms of we'll uplift here, we'll uplift there.

That doesn't work. That doesn't work. You need to uplift at scale because they're integrated.

I think that's a big part of what you're talking about. It's integrated so you one word one person is doing great work but the next person isn't there's a bottleneck so without that you can't you can't you can't scale

Working With Agents: Where They Help—and Where They Don’t

and you can't uplift productivity so this is that AI part now we'll go into that how we work with agent part as well what we're finding with the agents and there's a lot of other stuff we're finding but just for time sake and we've We've kind of limited to the three things.

You Can’t Use Agents for Everything

So this is something we found from having a few failures. You can't just use agents for everything.

And we're finding that off -the -shelf agents are good for, like, you know, maybe a normal tasks or individuals using it but to get the kind of productivity that we want to get we cannot use off the shelves has to be bespoke has to be very specific to the function that you're you're using the agent for that's what we're finding

next one so building on that part of the reason that you can't use it yeah

Human–Agent Composition and Optimal Team Patterns

because we're finding that the agent -human composition makes all the difference because agents aren't good at everything.

How you use agents, what you use them for, the roles that they play and the roles that human plays is critically important to scaling productivity.

And then building on that, and these are all compounding. They're not isolated.

they kind of build on top of each other building on on the composite teams we're finding there's optimal patterns right we're finding that you know four plus one three plus one in some in some cases and what's happening I think you kind of covered that in a bit and talk about it what

we're finding is if you have three agents and you just have three agents and then you have a human leading them the cognitive load on the human is far too much because of the speed that the agents run so you need a fourth agent to synthesize aggregate summarize provide the human with the ability to go okay what do I really need to look at in order to drive that type of you know to be effective in terms of productivity these are all the things that we're finding out as we as we go so

Human-in-the-Loop: Judgment, Governance, and Context

three key things human in the loop human needs to mean the loop judgment is huge governance is is what keeps you out of the courts out of newspapers also humans provide context I think someone mentioned context key yeah agents can can do lots of things, but they don't have context. They don't know how you think of things. They don't know how you look at things. But a person would, and that's needed in order to get the productivity.

And now I'm talking about like 5x productivity. That's quite, at an organizational level, that's a big target, right? But raising a little bit of productivity doesn't move the needle. You need to raise at scale. scale. And a structured approach. That's the other thing.

And I think, I don't know how much time we have. Five and a half more minutes. Oh, awesome. So Rana will show you what we mean by the structured approach. It has to be a structured approach. You can't just do it nilly -nilly.

Training and scale, I think we mentioned that. So I think, Rana, do you want to? Can you flip over to the other?

Enterprise Example: Scaling Test-Script Delivery in Consulting

Thanks, yeah, so obviously Avik has kind of covered the high level flow of the important things to consider when you're thinking about human productivity and then the human agent productivity. Avik, do you want to just do the slides for me next? Sorry, the clicker doesn't work when it's done in this way.

But when we're really scaling and we're looking at across an enterprise really massive productivity activity gains, this is an example of one where we're starting to work with a lot of

consulting companies, because consulting companies are in this weird flux at the moment where their whole business could be disseminated into AI, or it could be totally collapsing in some ways. So they're trying to really move fast in this area.

And so the consulting company we're working with, we're going to take you through a little bit of what the exercise was, was, if you go to the next one.

Step 1: Break the Work Into Scenarios

So the first thing was, think of it as an IT consulting company. They were looking at, OK, developing SIT test scripts or developing, basically,

test scripts to test an application or in a project to test parts of an application. So this is a scenario.

If we go to the next one, what we did first was actually break down the work into scenarios. So think of this as, you know, the core granular fine detail of that task, essentially. And I call it a scenario because let's say it's translating functional design into a use case, or it's selecting and ordering agent inputs, or it's doing some kind of a work unit here.

Step 2: Build a Cognitive Map and Capability Model

And what the system does is it actually takes all of the data from a variety of different data sources. it takes interviews with the people who are manually doing this and it then kind of creates what we call a cognitive map and that cognitive map is based on the work that is done work that is established yeah so this is the work if

you go to the next one having what we do then when we have the cognitive map of of the task itself or the scenario, we then kind of look at each of the individual capabilities required.

So on the left, you'll see solution synthesis, requirement, fidelity, and these are all related to the building out of the test scripts in this particular case.

Not to get too technical, but think of it as subtasks, essentially, that are done. And then what the system does is it knows what is good.

Step 3: Define “Good” Using Top-Performer Benchmarks

And it knows what's good, because what we've done is, is that somewhere in the organization, you've had performers who really knew how to write a really good test script. And we've used that as the benchmark. Now, you can create your own, but that really good benchmark, think of it as a Warren Buffett investment piece, that becomes what good looks like.

And then when we get the individual tasks, we can kind of start to see where the gap to that benchmark is and and that's really important because you're trying to see how far away you are from this this benchmark and if we go to the next

Step 4: Close Capability Gaps With Reinforcement Training

slide or the next section you kind of see what happens then is is that then you there's a gap to that benchmark I've got a as an investor I've got a gap to Warren Buffett I'm gonna go through a lot of reinforcement loops and continue to do training until I can hit that benchmark or at least uplift my

capabilities right and and that's typically what was done in training HR training programs back in the day but now we're leveraging AI with it and this

is where we start to see that my first test exam was a red because it wasn't that good when I did it again I got better and then when I got better the third time I crossed the benchmark so it's kind of then starting to make sure that you can uplift your capabilities okay next one and then what we do in

Step 5: Map Capabilities and Redesign Human–Agent Work

this context is is these are the employees within your organization these are the kind of scenarios we talked about and these are the the individual kind of scores based on your capabilities so we start to build a whole matrix of the capabilities within the organization and then if we go next

what we're doing here now is is looking at okay well what of these capabilities could be done by humans and then what could be done by agents and then what could be done as a hybrid and then we start to look at the organizational redesign of what that mix of humans agents and how that should look across functions and across the organization and the last thing probably because

Conclusion

I'm running out of time as well, is essentially if we use this approach we can start to actually really get productivity improvements at scale within enterprises and in a quite effective manner as well.

So yeah that's all for me.

I just wanted to kind of close with that example But thank you very much for your time as well.