Introduction

My name is Nick. I'm a product manager at ThoughtWorks, and I've been doing product management now for about seven years, and more recently at ThoughtWorks for about two years.

So today I want to talk to you about a product management playbook. So today I'll talk to you about a product playbook that we use, and specifically within the life sciences industry for a client that we're working with.

Is that better? Okay.

Client Context and Goals

Okay, so I'll give you some context on the client. It's a pharmaceutical company. I'll talk to you about AI -first product development practice that we're using. I'll get into a spec -driven approach that we're using to execute that. And then also the client's solution that we've built so far.

So just so you all know, ThoughtWorks is a global product development consultancy. So we're working with consultants around the world. we have employees across the globe we're working with distributed teams and we're not just building we're not just like delivering strategies and ideas it's also building those things for our clients as well so with the client that

we're working with they're a pharmaceutical company that's global and they have a lot of problems with their data and so they came to us to help them connect their data and also build and transform the digital digital ecosystem system, and specifically we're working with the R &D department within that.

And within R &D, it's the kind of KPIs and success measures we're driving for, reducing the time that scientists spend on non -scientific work, helping them accelerate their projects and deliver projects faster. And also data quality is really important because scientists, which you won't be surprised to find out, are obsessed with data.

and so we made a product bet and how could we deliver the these new digital solutions and data products for scientific workflows and also keep up with the pace of science and so I also like this image as well this image

really reflects the kind of environment that these people are working with our customers they are using computers but then they're also using like files you can see materials and that and they're working in a physical environment and and having to then come back to their computer to complete some of their workflows.

A Quick Chemistry Pop Quiz

So I wanted to give you all a pop quiz. I don't know if anyone remembers back in their high school days anything they learned in chemistry class.

So which element is the fundamental component after carbon that forms the basic chain linkage for nearly all organic structures? Just yell it out, which one do you guys think?

I don't think I heard the answer yet. It's oxygen, yes, maybe someone said that, but yeah, the answer is oxygen.

okay so I'll talk to you all what we've learned and throughout this process so

What We Learned: Data Silos and Workflow Friction

we learned about siloed data systems across all these labs so all you can imagine these everybody has a different lab that's solving a different experiment or a different area of the pharmaceutical lifecycle but none of that is connected and they're not following the same processes in this workflow so everyone's doing something differently they all have their own spreadsheets in their own way of working and it's all trying to glue things things together using email and sending the spreadsheets.

And a lot of these labs are having a lot of back and forth where they're just communicating about specific things but spending a lot of time on that.

And so the opportunity for us was trying to de -risk all of that time and remove all that project delay, standardize the way they do their work, and track their work between the data and all the different departments.

AI-First Product Development Approach

Okay, so I'm gonna talk to you about the way that we executed this using an AI -first development approach. So for us it wasn't a, it didn't really matter what the tools we were using, it was more about the operating model of how we executed that.

And so while we were using Cursor, while we were using Cloud to actually execute the tasks, it wasn't about that, it was about how we actually worked together.

And AI sits across the entire flow of how we do things, not just at the coding stage stage to generate the code. And so it started with the discovery, turning what we learned into insights,

and then transferring that into requirements, product requirements, designs, and then coming to the coding stage.

Why Guardrails Matter

And then also, I wanted to raise the importance of guardrails. And so this is something I'll touch on a bit more,

like how we incorporated guardrails to keep the output in check from hallucinations and other things we didn't expect to come out of it.

From Problem to Production: A Spec-Driven Model

Okay, so from a problem to a prod. So this slide kind of illustrates the way that we kind of structured this.

Guardrail Specifications as a Foundation

So one, we had a foundation of these guardrail specifications. So what that meant was we wrote these documents, and these documents would outline specific things.

Like, for example, our design system would be outlined in a document. It would say exactly what the colors, the brand, the UI components components that we wanted our applications to have and look and feel. And so then whenever we're building our features and our products, it would reference back

to that. And so on is in the same with the platform. We had some platform capabilities that we wanted the AI system to be aware of. And so it could reference that.

Same with our data models. We had very specific data models that we had architected. And so we wanted to make sure how we were developing things was all coming back and and referencing that.

And so on top of that, then we came to the product development of bringing the different designs and problems and turning them into a specific feature spec

or a specification for what we wanted that feature to look like for the end user. And then the third piece there is just what I mentioned before, the collaborative operating model.

Collaborative Operating Model

And so for us, the way that worked was having a daily cross -functional meeting or session with all of the people, the engineers, the designers, the product people and just looking at exactly what we're trying to build together making sure

that we're talking about the prompts that we're using and and kind of refining that on a daily basis we would do even like breakout rooms or some people talk about certain topics but that was kind of the gist of how that thing would we would move each you know day by day okay and so this this kind of just illustrates the the the hand off that we did across that that feedback loop so

So on one side, we took the input from the guardrails that I mentioned, the product requirements, the wireframes that were designed, and the human in the loop. And so as we went across the time of building something, we would pass that along to the next person or to the next step in the flow.

And so that's why you see going from the business problem, or the business case, going then to the design, then to a prototype that we can use and touch and feel, and then going going to those features specifications, and then finally going to the code

and creating the actual deployable product in the end.

Case Study: A “DoorDash for Scientists” Workflow

Okay, so I'll just talk to you about, just highlighting one of the solutions that we built following this process.

So for this solution, we worked in one problem space with an analytical sciences lab. This lab basically handles requests from other labs, processes that request in the lab, I mean, this is similar to how you saw in the photo earlier.

And then they would, rapid turnaround was really critical for them. They needed to finish doing all these experiments and tests and then return the information back to the requesters.

And so what we built was like a DoorDash app for scientists. And so this covered both sides of the equation. The ability for requesters to, like ordering pizza, would go through and order the specific scientific experiments that they needed.

and then on the other hand you have the the fulfiller persona that would then process that information and then take that and then deliver back to them and we would be tracking that whole process like you would see with your order or your delivery date for something that you got from your pizza app so this is

processing thousands of samples samples being like those scientific vials you saw in the photo earlier and then standardizing that workflow across many many different labs.

Key Takeaways and Ongoing Lessons

OK, so I'll wrap up here with some of the key takeaways that we learned.

So one was those guardrails. So I mentioned before what kinds of guardrails. I didn't go into a deep dive of what was actually inside of them, but those guardrails

are really important as a foundation. They set constraints for our AI that would be actually executing and generating code.

But those guardrails are really, really important to set the foundation.

You need to work with not just a small team, but you get alignment with all your leadership as well and what that would look like.

The next piece here is the testing and the design still needing rigor. So what I mean by that is just because you're able to generate some kind of software or digital tool very quickly,

it doesn't mean it's actually going to work the way that you intended to or it doesn't mean that it's going to resonate with your users or it doesn't mean that it's going to really be capturing all the test cases And so you can't forget that,

and you still need to budget time and effort to handle that.

The third piece here is just the collaborative side of it. Because I worked with my team cross -functionally, we were looking at the same kind of things together and able to kind of find problems.

And it also just became a habit of being able to build that habit together and finding problems together. So things we're still learning.

So this is fairly new to a lot of people that I've been working with. with it's fairly new process for me as well and so we're still learning like what's working what's not working how to make it better like and all that and iterating so one thing is on the design

side and the user validation side that still remains the same you still need to do that you still need to have good designs good design is still really important can't rely on just AI to generate the best design ever for you also with user validation you know a lot of times we we get

get excited about a design or a prototype we share it with our users and and it's just not working for them you know something's missing something wasn't captured so inconsistencies so there can be inconsistencies in what you've imagined and what you thought you were delivering

and then you know after doing even extensive testing you still might find things are not being captured that's maybe some kind of small hallucination here and there and the last piece piece here like edge cases still need product judgment.

What I mean by that is sometimes we would find an edge case where we didn't really know how to handle that or a user came across a scenario we didn't really expect.

We still need someone in the product, from the product side to make a judgment call on how to handle that or whether that's important or not.

So yeah, that's what I wanted to cover with you all on this presentation. That is all I have.

Audience Q&A: Guardrails, Compliance, and Build-vs-Buy

So thank you all for and open the questions Okay, any questions on the technical side?

Oh, I can move around the mic With your name

I'm sure bum and thanks Chetan. I have a question for you as well But I'll come to it afterwards Thanks, Nick.

So basically it's the Something that will help us generate a product using SDLC flow, right?

How are you ensuring in terms of guardrails that the token usage limit is within the limits plus what guardrails, how guardrails change with different domain? For example, you gave us in life science. Let's say I want to develop something for a banking domain or a retail domain. So are these guardrails only for software development or also for the product?

So yeah, the guardrails cover like a wide variety of things. So there's definitely, like, a product and domain guardrail. That guardrail specifies, you know, what is the domain? What is that?

What is, like, some, like, key acronyms, for example, that domain is using? What are, like, the terminology they use? What is the product vision? What are the, you know, describing the users and their own, like, workflows?

And grabbing publicly available information about the types of user personas. All that stuff is kind of embedded as well in the specific guardrails with regards to like domain or product requirements.

And so everything stays within a cloud code boundaries or a codex boundaries or it's like someone gives a prompt and it comes along with a software. How does it work? Like the guardrails we stored in specific files that then live in a repository inside of like the base, the code base.

So basically, it's a combination of agents and skills that are helping everything gets developed. So I would say we had the guardrails living in this core repository, and then the engineers who were then focused on implementing new or iterative code on top of that would then

have their own prompts that they're prompting and referencing the specific guardrails. Thank you.

can I ask you a question Chetan you do mentoring for storytelling because I believe apart from AI

I was going through a podcast and I heard that storytellers are also making good money but yeah everybody needs a story

I think everybody has a story and I love just hearing people's story and try to translate that

to the investor audience because I know them also well So that's why I'm in that business of storytelling for founders to pitch to VCs. That's a specific type of storytelling.

Yes, and I do mentor a lot. Thanks for asking that.

You have a question, sir. Good evening. My name is Izzy, and I'm studying for my governance certification. I'm just curious, when you deal with your company and other companies in the medical field,

you're sharing a lot of data. And so I guess that's what you're talking about. Guardrails. Are you on the deployer side or are you on the...

I'm a contractor. Is there a question too? Or is that a question? Okay, no. But is there a difference when you're on the other side, Nick, on that part, to build upon that thing?

So as contractors, we do have some limitations. with how we can use AI with our clients' proprietary data. So we actually do start with a lot of, let's say on my, for example on this computer

I would just work with synthetic data, do some mockups and prototypes and so on, and then I would move that to our client environment in our separate, like we have a virtual desktop,

then we move it there, we do then basically integrate the work. So there's a bit of a bridge that we have to cross as contractors, but we're trying to recreate that workflow for our customers. Sure, sure.

So you are a Canadian company, right? It's a Canadian company. So how is the EU AI Act coming into Canada, and how are you working with the new regulations?

I wouldn't know about that, to be honest. I haven't noticed any impact. But, yeah, we all know we need some sort of regulations that's another topic for another day thank you so much

Nick for sharing what you're working on it's amazing I think you're part of that new wave of

companies that are actually developing their own software rather than just going for available software as a service that's out there and I was wondering was the AI also prompting you when you were working keen to use software service solutions that are out there available, rather than just creating one from scratch using AI?

Yeah, so as I kind of mentioned in my intro, I work at ThoughtWorks, which is a consulting company, and so when we look at the problem that our client has, we look at how we would solve that problem with either building something custom with our own team, integrating with with existing solutions or something in between hybrid.

So in this case, the client wanted us to build them specific solutions for them to take and maintain. We also had other solutions that are in the industry. Why don't we just buy those and integrate with that in some way? So those trade -offs were something we made earlier and this is how they landed in this direction.

And a question from your experience, like have you worked even before AI in building this type of products for clients or do you guys just started more with the AI wave?

So yeah, we basically took a bet, like I said earlier, like we took a product bet and said can we just do this more rapidly? Can we take the ideas that people had and kind of put it on the shelf because it would just take too long and we're kind of bringing all those ideas out and we're trying to more

more rapidly deploy digital solutions like the one you just saw a screenshot of earlier. So, yeah, definitely.

And can you talk to us a little bit about that trend? Like, how do you see it going on in the future? Like, are companies more drove now towards building their own things with AI rather than getting what's available there? Like, what are you seeing in the market when you see requests?

I think back to the earlier point of, you know, evaluating whether you should buy a solution or make a custom one. I think with custom ones right now, everyone feels that, oh, I can just vibe code something so quickly and easily, then why don't I just go in that path?

And then as you saw earlier, you're probably not thinking about all the guardrails. You're not thinking about the different test scenarios. You're not putting in that rigor because you're thinking more, how can I be more rapid with something?

And what we're seeing is that people are just now prototyping their ideas so quickly, and then we're oversharing to our client, for example. So you're saying, oh, well, we can build you this amazing thing. And look, I just built a prototype so quickly. And they're like, okay, let's see it. And it takes a lot more than just...

Now the hard part is actually estimating and predicting how quickly we can actually build things that we're proposing.

Oh, thank you so much for sharing. Thanks a lot.

I'll take his question first and then come to you. You had a question too. You still have one? Okay. Okay, we can take one. I'm so sorry so that we get the other speaker also done,

but Nick is going to be around. Not like Ashna, he doesn't have another commitment. Right, Nick? You're here, and we can all chat with him later on. But if you want to ask a question, go ahead.

Hey, great presentation. I'm Adrian, co -founder of Panorad AI. For the past three years, my company has been delivering sovereign AI enablement

and transformation for regulated businesses.

So some of the questions that the gentleman over there had, it's home. If there's a client that has HIPAA compliance, HIPAA, HIPAA requirements,

the data cannot leave the country, how do you go about that? Do you have sovereign delivery, like you have AI that can run inference

within the borders or other solutions?

I really don't have a good answer for that. We have like data experts on my team and we have like a security expert, we have a data governance expert, like those are the subject matter people that I would refer you to but unfortunately I cannot.

That was just a trap, I'm trying to promote myself. I'm kidding. But it was a good question. So basically there's a framework that I follow.

So first we have to establish a POC or MVP, the synthetic data. If that works, then you go to the customer, tell them that we need this data for this product to actually function and work, if they are compliant. That's how it should work.