My name is Xenophon Vassis. I'm working in revenue operations at Yodec. Yodec is a SaaS company that operates in digital signage industry. And I'm going to present you today how we built a real internal tool with AI from prompt to production.

From Everyday AI Use to Real Internal Tools

Actually, my presentation has four parts. parts. The first part is about how we are actually using AI nowadays, all of us, versus the capabilities that AI has.

The second is the connectivity. LLMs nowadays can connect to 9 out of 10 tools that you are using on your company. So it's very important also also to talk about the connectivity that LLMs nowadays have on our systems.

The third part is about the recipe, not a food recipe, of course, but a recipe that combining those two ingredients that I mentioned before and another third ingredient that I will mention on my slide at the time, we can actually have our internal tool that we build it from AI from scratch without knowing coding for example.

And the fourth part will be at least from my experience and what my journey taught me how to use AI in the right way and how not to use it the wrong way of using AI. Let me rephrase it, the bad practices of using AI and the best practices of using AI. So

So I will also make a question and I want to ask how many of you have you used AI to write you an email, for example, or to summarize a report or something like this. I think most of us have used it. Keep your hands up if it is possible.

I want to keep your hands up if you have used it to build actual software that helps your team every day and resolve an issue that you have. Perfect. I saw some hands that dropped.

Moving Beyond One-Off AI Tasks

So what we are actually using AI today, it is, as you saw, as you saw, when I asked if if you have used AI to write an email, all of you raise your hand. The most common thing is that we are using AI for single tasks.

We're using AI to summarize a thread, to have a single report, to do a task. And when this task ends, nothing else. We don't resolve anything. We just move on with our next things that we have to do. do.

Some of us, we may also move to the next stage and we may also automate something. We may also find the symptom that it is repeatable and we may automate it. But actually are we resolving the problem or we are just automating something that if we don't find the source of the problem, we are still having the same problem.

So what we should aim is actually to find the cause of the problems and to try to create the solutions, the software let's say, through AI, and now AI is capable to do it, to resolve the problems, the issues that may create those symptoms, may create those repeatable tasks that we may have. have.

So what it can actually do, AI, is to read all your documents and your data, it is to actually write working software, is to run multiple -step work on its own, and it can actually connect to your own systems.

The Gap Between AI Investment and Impact

So I am enthusiastic with the numbers, so I will go through three numbers that I find very exciting. exciting.

The one is 92%. The 92 % of the companies, based on the McKinsey report, indicates that 92 % of the companies are willing to spend more and invest more on AI.

But the same report says that the leaders of those companies, only 1 % of them think that their companies companies are AI mature. And AI mature means that they have integrated AI on their workflows, on their systems.

And there's also the 95%. The 95 % comes from a research that happened in MIT. It started on 2023 and we got results last year.

And that showed us that in 2023 some companies that was on that experiment didn't have actually measurable impact after using enterprise AI pilots. Didn't have profits, didn't have losses.

And that you may ask, so why we are here? What does that mean? So AI doesn't work? Not really.

What was the The report indicating it was that none of those companies was using AI in the right way. They didn't have integrated AI to their systems. They didn't have actually put AI on their workflows, on their everyday system that they are using on their companies. So the limit is not what AI can do, but it is actually what we ask of it.

Connecting AI to Company Systems

As I said, our tools don't talk to each other. I am in revenue operations, I am working with many tools and I'm sure that many of you also have experience working with a CRM, with a PRM, with a billing system, with several tools that you are using and the connectivity to all those tools is extremely hard and most most probably we don't have any connectivity. The tools are not speaking to each other.

That causes frictions on the numbers that you may get, on the numbers that you may ask, on the things that you may want to do.

What happens and what changed with LLMs is that now you have a place that collects all the data from all your systems. It doesn't replace your source of truth but it actually and I don't have okay it doesn't replace your source of truth but it actually can collect all your data to one point and to be accessible data for everyone and for you actually to work on those data so in my first slide I mentioned that we have first nowadays the capability and And the second one, the connectivity. So what is missing is...

Can I ask a question? Yeah. Oh, pardon me. Questions? Can we have? We have three to four questions at the end. At the end, okay. Perfect. Okay. I do not have much.

The Missing Ingredient: Problem Ownership

So the third ingredient that I mentioned in the beginning, it is the ownership. ownership. The ownership that it is very important and the ownership that creates the problems.

So if you have, if you are the owner of the problem, if you know the problem, if you know the solution, then with the other two ingredients that I mentioned in the first and the second slide, you can actually create something useful. You can actually create an internal tool.

For example, I know I am on sales, I'm on marketing, I'm on operations, I'm not an engineer. What was happening in the past it was I had a problem. I knew the problem, I was the owner of the problem but I didn't know coding. I wasn't an engineer.

What I was going to do is actually to raise a ticket to the engineering teams and after that I needed to describe the issue to the engineering team if the problem was on the roadmap that the engineering engineering team created

so after that i was needed to describe the problem i was needed to make the problem to be understandable for also engineers that are more technical and if the problem is in another department it is very difficult to make them understand what it is actually the cause of the problem so also the output that you may also get it may be not be the desirable outcome that you may need it so right now you are the ownership of

the problem you can resolve the problem by your own and only need the only thing that you need to do is actually go to your AI to your LLM and ask some small small things.

A Practical Recipe for Building with AI

First, you don't start big, you start small. Second, you need to be precise on the problem that you have and you need to set the rules.

In a language AI, in the language LLM, the problem is that the output that every time can give you can be different can be different every time so you must to create you must ask to create your llm to create actually coding to follow those rules every time with the same way if you don't ask it and if you don't make it create you a javascript for example that it will follow those rules every single time every time that you may ask about something the outcome may be be different from the last time.

The rules may not apply to the next agent that you asked for.

Case Study: A Billing Consistency Tool

So in my case the problem was in billing. We had an issue in billing. It was very complicated for the partners.

So what was happening it was that sometimes we may have different numbers, we may have wrong numbers to be be invoiced and that was costing not only money but also the trust of our partners and this is the most important that it was costing to us and as a company that we are, we want to be close to our partners, we need to have consistency to what we are invoicing and what we are billing. billing.

So what I designed is a tool for GDPR reasons I'm not going to go through the actual tools I have created a video of it.

So what I did is I created a tool that peels with consistency and also because I know billing is not so appealing for the reps I created it also to other sections expansion and outbounding that make it more appealing for the reps to go through the tool to find opportunities for their own so to also you know if you make someone to open the tool because they find it appealing you make it use also for the course that you need it for the

consistency of the billing so I'm going to play the video sorry it's a little bit small.

How the Tool Works

So, every code has many rules, as I said before, we have proration, code terms, you name it, many rules. And that causes sometimes to build wrong and that was causing us, as I said before, not only money, but also trust.

What it does, it is that it can create quotes and it can actually calculate by cent what we are built to with consistency what we are building to our partners.

It can analyze invoices so if you have a wrong invoice it can go through all your line items and find where the wrong is coming from.

It connects to our systems to for example to our CRM so So it can actually go to our CRM and you can actually add a note on our CRM about that calculation, about what you calculated.

And it also, as I said, it funds expansions and outbounding opportunity that it also connects to other systems that we are using internally in our company.

So let me move to my next slide.

Best Practices for Building with AI

So what my journey taught me about building that tool, it is that it is real that we have wrong ways to use AI and we have actually right ways to use AI. And the wrong way to use AI is to trust the first answer. If you ask something AI and you get that answer and you think that the answer that you are getting is true, then most probably you are getting an answer that seems like true, but it's not. You have to check every single time the outcome that you get from the answer that you are getting.

creating. The second one is to ask AI for the answer every time. You don't need to ask about the answer every time. This is the task that I mentioned on the first slide. You need to create something that produces the answer by its own. You need to create the tool that resolves actually the issue not to resolve every time every day the task that has been created

from that problem the third one is build something you cannot judge if you build something that you think that you know but you don't actually know then you cannot review it so this is also something that for me at least I think it is a wrong way of using AI.

And the last two is expect it right the first time. AI and the right way of using your LLM is actually to have patience. You need to actually make many many many questions. You need to actually have conversation with your LLM.

And also the last one is to let it act alone in real systems. As I said before yes it is we need to sync it to our systems but we need also to have control on what it actually writes back to our systems because we are not changing the source of truth we are collecting data at one place so that's all as you can see that

Iteration, Validation, and Control

for me it took 93 messages to build it 27 versions no sorry 30 versions of my my actual tool and 27 times that I said that something is wrong, something needs to be correct.

Conclusion

So that's all, thank you, thank you for your attention.