Introduction

Hi, everyone.

If you didn't hear me at the beginning, my name is Stacey. I work at Lyser AI. It's an AI infrastructure company based out of Jersey. Also has a big office in India.

I'm one of two Canadians that work there. I'm a client relations manager and solution strategist.

So I think a lot of people have to have a hard time wrapping around what their heads around what Lyser does

What Lyser AI Builds (and Why It Matters)

So does anybody know what Langchain is? Crew AI? Agent Force? Heard of those? Even Make .com or Zapier .com?

So basically We build agentic solutions primarily for enterprises.

So there's so many trends going on Open Claw You have well, we've been talking a lot about vibe coding lovable

So, the way we look at it is, what do enterprises and businesses need that will make things more secure, responsible, but also built in a way that they want? One way is open source. So, we're very much open source.

We're not biased to cloud or infrastructure. We work with any cloud service. Our platform goes right into their environment on -prem, so they have full control.

LLM agnostic as well. So, that's what we do.

From Platform Context to Practical Advice

The reason I'm telling you about the company is that it makes sense to what I'm going to talk about here is I've been there since August, but I've been building agentic workflows for over a year right now from a strategy side, and I think a lot of people jump quickly to

I just want to build something, but they don't really do the thought process leading up to it on making sure that the use case that they have in mind is actually going to survive. And I think you've all heard about that.

So I don't know which company was McKinsey I can't remember who put it out 99 % of agentic use click cases fail So I'll just share some success that I've seen over the cases over the year of how to make one survive

What “Agentic AI” Means (Beyond Chatbots)

So I guess just to start off just to make sure that we know what agentic is.

It's it's not a chat bot It's really it's a it moves through work It moves work through a process.

So I almost like look at it as like if you think of an org chart and you've got a bunch of people, think of each person as an agent. So it's like a series of agents, each have a role that is dedicated to what they need to do, and they're all connected.

So that's how we kind of look at agentic AI. It's a connection of a bunch of different agents.

And there's, you know, specific things that they go through. You've got your goal, your context, you connect to tools, you have a bunch of rules, and then you have your ultimate outcome.

A Concrete Example: Billing Automation

So for instance, if we look at a use case, I'm going to bring up use case quite a bit. I want to make sure that everybody kind of understands use case.

So one we worked on with a brokerage firm, it's billing automation. So think of somebody that takes invoices in from email, and then they have to generate an invoice. There are a ton of steps. They have to cross -reference so many things.

So we created an end -to -end solution for this brokerage firm on how to get an invoice from an email and then generate a bill to invoice the client. So it's very detailed. There's about 20 agents running in the background.

So consider that a use case, something that you want to solve and you probably are bored to death of doing it. it. The big goal for them is that it was very time consuming and they had so many people executing this process. So they wanted to see what they could automate.

Start With the Workflow, Not the Technology

And so when you think of it, like, the wrong answer when it comes to building a genetic

AI is you don't start with can AI do this? No. Because more and more often the answer is yes, you could probably get AI to do it, but why should you?

You don't do it for the sake of doing it, you really have to start with is this workflow the what you want it to do?

Is it clear? Do you understand it? Is it enough to delegate to AI? Can you measure it?

Can you govern it? And can you scale it? So that's how you kind of got to look at a use case when you have something in mind. I think we work with a lot of clients and they're still in this like, oh, this would

be a cool thing for AI. We build it for them, and then it's crickets, because they haven't developed the ROI, the adoption rate is low, and there's actually a lot of services that already execute what they want to do.

So, and another thing that we noticed is that not every use case needs full autonomy. So, another thing that we kind of noticed, you know, in the early stages, they wanted to automate every step. And we realized that not every step should be automated. automated.

How Autonomous Should the Agent Be?

And so we broke down into asking, you know, our clients or even ourselves, like, what do you want the agent to do? Do you want it to observe?

Do you want it to recommend something for you? Do you want it to prepare something like a draft?

Do you want it to execute something and approve? Or, you know, you want to execute with guardrails? And you can do all of that within an agentic workflow.

But you don't have to automate every single step and I'll talk about it at the end that's why because I don't know if we all know that token efficiency is becoming an issue right now so if you're gonna automate everything get ready for a big pay bill at the end and so seven

Seven Survivability Tests for Agentic Use Cases

survivability tests that we kind of go through or I've kind of learned over time is work for reality can we actually map what happens today and this is an interesting thing.

If I go to an SME, a subject matter expert, on a specific use case, I'm gonna be asking them, okay, how do you execute that task or that use case today clearly? And if you don't know how to do it clearly, then you're gonna struggle in building a gentic solution.

You have to know the existing use case today.

Delegation readiness, you know, you know, what are you asking agent to do? Again, what I said before, observe, recommend, prepare. You have to really understand what you want the agent to do.

And then data access. If the data sucks, so does your agent. So if you don't have good data and if you don't have access to data, then you're going to have a really hard time building a successful agentic solution.

And then decision clarity.

Another thing is that humans need to explain this. We've got vibe coding going on, and if you can't explain your agentic workflow, you're in trouble when it breaks down or when somebody asks how it works or security or, you know, privacy teams.

Another one is risk boundaries. So you built this agentic solution. What are the risks attached to it? What if something goes wrong?

These are things you have to think about even before you build the solution. You've got to have a crisis management. Like what if your LLM, like if you're using Claude, it crashes. Well, what is your solution to that, right?

And you're like lost. And I know a lot of people are lost today when Claude breaks down. I'm not. I have other tools that I can use, but I know some people freak out.

So there's the risk boundaries.

Another one is value realisation. This is huge. I think, again, we get caught up in wanting to build something, but how are we going to measure it?

What is the value of this use case? Again, don't build it just because you can. You're really looking at ROI.

You're going to be spending money to maintain this. So you got to be prepared with like understanding what goal and objective and I have a formula at the end that I can share And that also goes into cost to serve, you know can models tokens and retrieval

You got to look into that like which LLMs are the best you don't have to overdo it A lot of people are loving the anthropic models because the reasoning bottles and everything But if you really come down to do you really need a reasoning? Model to execute that agent. Can you just not swap in a cheaper Gemini model?

Those are things that you've got to look at, you've got to plan for at the beginning, and you've got to test those various models, because at the end, if you're paying a lot for that cloud model and you can't afford it, you throw in a Gemini model and it doesn't do as good of a job, you've got to do this testing up front. Be prepared when the LLM is going to go down, which other LLM is going to take its place, and how quickly you can do that.

A Simple Framework to Choose the Right Use Case

All right, now it's coming to choose the right use case. This is a model that I kind of developed because I realized that some of our clients didn't quite know what they wanted.

So I don't know if you can really see the chart, and I apologize. There's two charts, but it works really well.

So let's say you don't know what use case you necessarily want. So you kind of come up with five of them. And then you kind of define, well, what is the agent role for the specific use case?

First one is a customer support triage. So you kind of have to define the agent role. rule.

Then you've got to look at the value and feasibility. So how much value would I get out of this, again, going back to the value realisation, if I build this? And how hard would it be? What's the feasibility?

Do I have the right data? Do I have the, you know, connections all set up?

Next, guardrails. Like what issues might arise? What guardrails are going to have to be put in place?

So you answer that for all the different use cases, and then you kind of come up with your verdict. it.

And that's a way to kind of really think through, you know, what you may want to build, whether it's for your customers or whether it's internal and you work for, and you have a bunch of clients internally.

Discovery: Mapping the Current Workflow Before You Build

So now you got your use case. And again, sorry about this, but this one works really well.

It's mapping the current workflow you build. And this is interesting, very challenging. We call this our discovery phase with our clients.

And this is where they have to, but the chosen use case, this one's the customer support triage and response preparation agent, so it's a customer support agent.

So we need to go through every single step, right? If you don't know the step, how will the agent know what to do? So we have to dissect the steps.

The agent doesn't have to do every single step. It's all about the outcome, but you still need to understand how the work is done today.

And why is that? Well, we got to know, okay, what actually happens at each step. Who is the individual that currently does that step? And then system and source is used today.

Well what systems are you connecting to? Are you connecting to, is the user using email? Are they using a CRM? Why do you need this? Well the agents going to need to have access to those system and tools.

Is there API access or are you going to use SQL query? You have to know all of this stuff because there's a lot of systems that you use they're just going no, sorry, we don't have an API, or it's going to be, you know, super hard to connect.

A big thing is that because of vibe coding and using platforms to create code, the hardest part is the integration. That's where we struggle. The integration part.

Coding's easy. The hard part is integrating with these systems. And you're probably going to connect with a lot if you're building agentic AI. And the more complex, the more headaches you're going to have.

So that's the the systems and sources. Then you need to understand the input and output. What is the input going into that step and what is the output? Because you do want the agent to do that as well.

And then, you know, at last is success metric. Like, what would be success for that step? So, when you get that, you know, mapped out and written, then you build your architecture. Your architecture roadmap.

Then you kind of understand. You can build your agents, connect them to what systems, and then you can have a visual. So, that is is the goal and you need to do that before just building because sometimes you build something you go i don't even know what i built i think i built this but this is where you have control

Proving Value: Metrics, ROI, and Operating Costs

and it's more methodical and thoughtful all right i'll move quickly um now it's proving value a working agent now we're talking about roi and and value realization so a working agent still has to

move a business number and what i've also what i've noticed at the beginning is weak metrics metrics, like, you know, successes, demo feedback was positive, yay, users opened and tried the tool, wow, that's fantastic, the agent completed a sample task, wonderful. These are not metrics, because that's going to die pretty quickly.

You need to look deeper. How is the business, again, going back to how the current workflow as, you know, using humans and other different systems, how is that measured? How is whoever was executing that, how are they being measured?

you got to look at that and you got to find your benchmark. So a big thing is cycle time has reduced. Like I talked about that billing automation agent for invoicing that reduced a ton

of time, which equate to saved money. Backlog decrease. You know, if you're an IT and your backlog increases substantially, there's a lot of money saved and time saved that way.

So you got to kind of look at that when it comes to value realization, because what you're going to enter is this you're going to be entering token world and if you have a bunch of different agents they're connected to a bunch of different llms and you need to know what llms are best for each but

there's the the order to survive you got to look at your value realization what is your oi what are you saving money wise or time and then you got to minus the cost to service it how much is it costing for the compute the tokens and everything like that and then obviously minus the operational risk as well so if something goes wrong what is

the cost to you so you really need to look at all of that upfront and not too scary in order to survive sometimes you get lucky and it's just wonderful just like the last speaker there you know congratulations on that but like when you are working with clients or you're working with internal stakeholders you

You know, you can't really just wing it. You do need to be methodical because, you know, it takes a lot of effort to build an agentic solution. And if they build it, it launches,

and then it doesn't work, you know, you're not gonna have happy customers. So like I said, for this, you gotta make sure

you use the right model for the task. And then only retrieve contacts that you need. So another thing is is that you don't wanna, you know, to retrieve more info than you have to because that costs tokens, right?

Another one, avoid unnecessary multi -agent chains. So when agents are talking to one another, that takes time and money as well. So you've got to, you know, kind of be methodical and thoughtful in that.

And then, yeah, I'll just close up. It's just really kind of understand which LLM does right.

And then if your system goes down and that LLM goes down or it's too expensive, do test test out the other cheaper models, so you can quickly swap one in.

Closing: Operational Resilience and the Lyser Platform

So that's what we do at Lyser AI, we're an open source platform. If something's going wonky with ChatGPT, we just swap it to Gemini, because we know Gemini will work just as fine for that agentic solution, just in order to, you know, be safe.

But that's it. That's me.

The QR code is to Lyser, you can learn more about it.

The studio platform, you can build your own agents. It's a low code, no code type of environment, but we also have a vibe coding architect that we just launched.

It's in beta form, but you can go in and build your own solutions, but we also work with enterprises and build them for them on this platform.

Thank you.