Introduction

We are here in an application called Enriten, or NNN in English. It's an application that makes up automation. Automation is basically the process of automating processes workflow.

Overview of NNN

And what is cool about NNN is the fact that it allows you to embed some LLMs easily. and to build agent. Here you have an example of a workflow.

Basically, you can think about it as a kind of customer success manager for e-commerce. So this is a trigger, and this trigger is an event. 1An event is triggered by one action. And this action in this Google Sheet is the fact that you add a row.

So each time that there is a row, it will be some steps. Each step is represented by a node. And you can have, for example, the if node. So it's a condition. So given some condition, it will pass all the information to the next node. where there will be some update on the Google Sheet.

And then maybe you recognize the OpenAI icon. So this is an LLM that is embedded that will summarize with some instruction the info that the precedent node gave to the following one. And then we will send an email.

How NNN Automation Works

So basically, when I trigger the flow, for example, you will see at the beginning that five items go in the fourth row. Why is this? It's because when you look at this worksheet that represents the orders, the condition, check the status. 1And if the data has already been processed by the bot, it will not trigger the action.

But if, for example, I modify this row, for example, so imagine that this is the new row. And then you activate the workflow. You will see that one item among the four, so among the four, go to the next step. What does this next step? It updates the sheet.

So you will see that unprocessed is made processed. So the workflow processes the information. And you will see that I should have received some email. But I think there is some presentation stuff. So I will just go to my email to show you what's happened.

Workflow Example

You will see that you have the message that I encode in the LLM. I will show you what happened. That just gives you, for example, that send to your team, your customer team, the new order that has arrived. And this is compute based on this effect. And as you can see, it's pretty relevant to

the information that we gave. So if we go, for example, in the LLM, you will see that I connect just my OpenAI account. The resource is just text. The operation is to provide a message. There is the model I use, and there is the instruction. So I give some instruction with all the other status and info that you have in the Google Sheet Triggers. OK?

So this is a... Sorry, I don't see the mouse. So this is a basic usage of how you can implement LLM in a classical workflow. Usually, before the advent of AI, these steps were pretty difficult to manage because you have to really make insight about Google Sheets, retrieve information, and do programmation. Now, as you can see, With one, two, three, four, five, seven clicks, it's possible to manage it quite easily.

OK. So this can be a real use case, in fact. And you can imagine, for example, that your bot only send email to your team when I don't know the shipping status is canceled, for example. And you can say, OK, for each order that has been canceled, we want that our customer success team contact the customer and ask why, what's happened, how can we improve, and so on. So it can be, I think, one pretty good use case.

Advanced Workflows

OK. I will then show you a different workflow that is a little bit more complex. Just before, for everyone, was it clear? OK. Do you see the interest? Are you excited?

OK. So maybe I will show you Another workflow that is a little bit more complex, but also more interesting. There is no huge effect. I just will explain to you what's happened here.

So each time that I click, It takes one document in my Google Drive that I put in it. And then this system, so four nodes, are building what we call in AI a rack system. So it's a retrieval augmented generation kind of component. So why all this barbarism? So this term just you can think as a specialized part of an AI component that will allow you just to get the information you want given some documents or some information.

So instead of providing you answer based on the previous training, this system is like a kind of layer that allow you to really target and be really precise about one document, for example, or one set of document, one set of representation. So this type of system ensures you that your model will not, in a more difficult way, hallucinate. So it will help you to prevent your model to write you bullshit, basically. So I give to it some complex document that is a sales report. And no one loves sales reports because sales reports are pretty much, there is a lot of number, there is a lot of complexity inside of it. So I think this is an interesting use case.

And you will see in the next workflow how we can reuse this system to interact with the document in a more convenient way. So maybe you want that I go into details. You don't want to. I don't know. Do you have some questions regarding this system?

Detailed Explanation of the Rack System

What does the system have? it's a rack system so here you have what we call the vector database so is pinecone it's a system that allow you to to basically store the data in the same representation than you have in your model so this is embedding and it's just a bunch of numbers that represent what you can think as a point. And this point in your space represents a meaning. And this is the way that the two systems communicate by meaning. So this is how it works.

This is the data loader. So this is the way the data is loaded. And this is the way to cut, for example, all the vectors in small parts. So you have all this type of system. But you see, it's quite easy. Yeah? You have a question?

When you're trying to figure out the action of what you're supposed to do, how do you know what AI model to use? When I build the workflow? Yes, when you build the workflow. OK. For example, if I have, for example, a sales comes to me and say, OK, I want to provide my report to my sales vendor.

They don't know all the details, but sometimes there is some customer that ask precise question and I'm not able to answer to them. So I will ask to the sales manager, okay, do you have a report? And he will say, yes, I have a report. This is the report that you see there is a lot of page, there is 84 page and it's really complex.

Then I say, okay, if you want to interact with a document, a special one that is really complex, then I can provide this type of system. If you want to just chitchat about mail, stuff like that that are not so complex, I will not recommend to use this type of system because it's overkill, basically because it's short document and you don't need to have a deep understanding of what's happened. This is how you should process, but you should be able to have some, yes? Practical insight about what you want to do, basically. Yeah?

Building Workflows

Yes. I mean, so far, when I look at this diagram, I just see the general idea of, say, a rack system. Yeah. OK. These two, you can think, for example, I don't want to go too much in detail, but you can think as a node like a mix of API call system and also with a lot of bunch of code inside of it.

So for example, when you see Google Drive, it's just called the API of Google, so the service, and make some coding stuff in order to extract what you want to do. And for the rack system, it's the same stuff. This model is used by a long chain. I don't know if you know this platform. And this one also is by default provided by long chain. So if you go inside to the code, you will see that it's long chain, basically.

And they use the same concept as longchain. So if you understand a little bit the concept behind longchain, you will be able to understand it because it's the same system. And then it's just basically longchain. This one is just basically a mix between longchain and OpenAI API.

OK. So when I want to use NAN. Yeah. No, you just, yeah, I will show you after all. But you just have to, for example, you go there.

Steps and Tools

And for example, if you want to, I don't know, you know that because you have an understanding of the concept of RAG, you know that you need a loader. So then you have a default data loader, OK? And you will just take, for example, this node. And you will see it will explain a little bit what is going there. And usually, it's well-defined for simple case. So you don't have to add option and stuff like that.

For this case, for example, if you go how I build it, you will see I I just formatted the input. So I just tell the model, OK, I want MIM type in order to be able to recognize if it's a PDF, a JPEG, stuff like that. So it's really easy to manage. But if I have time, I will show you the last.

I think it's quite tight in terms of time. I can use the final step. So you will recognize that I reuse the pine cone vector store, so the DB, just with the embedding because I already process the data in the other workflow. So now I just need the representation of this document. So I just call the embedding of OpenAI.

You have seen that this is the agent. I provide him. a model of OpenAI, and just some memory in order that when we chat, he didn't forget what I just tell him. So as I mentioned, you will see the report. So it's a public report about .

Practical Applications

You will see there is a lot of number, case study, background, the growth, the expansion, the recap, a lot of number. So OK. BFCM, I don't know what it is, but maybe you know. So we will ask to the model, for example, some stuff about that. And we will see what it will respond, and we can check what's happened. Then you have basically an interface that chat.

So when you chat, it's trigger the agent. And I will ask in the report of what is, I don't remember, what is BFCM. OK, and you will see, OK, so it's called the first time the agent to retrieve the information in the question. It's go to the pine vector. It's go to the embedding. Then it's go to the memory. Then it's recalled the OpenAHL pod. And then, basically, here you are. So you see all the data with all the number.

And if you check on the report, I think this is there. You will see at least all this number, I think. And yes, basically is that. OK.

And you should say, oh, OK, but it seems really, really, really complex to me. But basically, what you do is that. I just will show you. For example, you have a trigger, so the chat trigger. Okay, so nothing to do.

Then I want to add an agent. Okay, I add an agent. Okay. This takes the input. Okay. I maybe want the system message so that in order that my agent answer. OK. And then I want the chat model. So what is cool also is that you can use all these models.

And you can also use what is called Oyama. So if you want to build a system that is local, you can do it. And I will show you that I have the same system but in local. So here I use the cloud system because I think it's better for this type of presentation. But you can run it in local. So it's GDPR compliant. And you can choose also the application you want.

So basically, then I will just take one model up. When you use your local model, is the database, the vector database, also local? Pinecone is in cloud, but you can use Quadrant, which is also in NU10, that can run locally also, or on your own server.

And basically, when I work with Swiss industry, they always want local system, so this is Pretty nice for that. So you can choose also the model. I didn't show you, but there is a lot of model that you can use. And then, OK.

Creating Personal Assistants

I will hide the chat. And I will open new chat. And you will see that, for example, without noting, you already are able to

to provide you something, you know? So you just put your model, your agent. You don't need to add some memory and stuff like that. And what is cool also is that you can build tools that are related to agent to make your action. So for example, if I want to build a personal assistant with my calendar,

Okay, I want to get some event. I don't want to think too much, so I will let the model define what I need for the event and the ID. So I don't fulfill because I don't really understand what happened. And you will see that basically you have then a model that allows you to get some information about your event. So what are my events today?

Ah, yes. Why? Ah, OK. OK, OK, OK, OK. So there is an error.

It told me, oh, you are too precise. So I put more ability to the model. And maybe I just want to tell him that he has to check there. So get event tool. And then he will know quite easily OK, so I don't know why, but there is some error. I don't know why. But basically, it should retrieve my today event.

And then you have a tool that is able to, for example, tell you how, OK, you have one event this morning or one event, I don't know, tomorrow. And you can also edit, create, delete. And basically, you can build your own assistant. And last thing, you can connect it to WhatsApp. So if you want to chat, for example, you can send your message directly to WhatsApp. And you can also trigger all the information using WhatsApp. So for example, you have one, two, three, four, five, six, seven agents on your phone, basically. And you can do it for Teams and stuff like that.

User Experiences

And I think this has ended my presentation. Do you use this for your, like do you set this up for the company you work with, or what kind of companies use it? I work with small companies, with independent, and also with artists, because they have a lot of administrative work, and this type of tools help them to manage all the staff with all the team. For example, I work with a theater company,

and to manage the administration they use agent for example for invitation meetings for also making some projects also for digging for information regarding open call and stuff like that any other questions or And also e-commerce. I work also a lot with e-commerce people. That's why I take this. Is it big e-commerce or like it could be individual? No, no. It's quite big. Last year I worked with Decathlon. So it's quite big. I don't use anything for it.

Just to understand because you didn't give your introduction. Yes, I have a background in development, so I'm a dev, but yeah, since two years I only work with this type of automation and topics. Also making some custom agent for

in-built company. So I also help to the customer support systems and stuff like that. So how do you say that? Call centers?

Getting Started in Automation

To kind of get into this field, what steps as a student do you think I should take if I was interested? I think you can start with that because it helps you a lot to understand the general concepts. They have quite good tutorials and this is open source so in Switzerland I think it's also important to have at least one or two tools that allow you to go to company and say okay we can develop it locally. Without using chat GPT, I never used a GPT, for example, when I work with industry.

So this is hardcore engineering? This is. The question that she asked, like where she said that because she's a student and how can she get into this? So is it like? How can we have our steps into this? Like is it hardcore engineering or is it... No, no. It's just plug and play as I show.

You have, for example, business cloud. Okay, I want to send message. You have the notes and message.

And then you have all this documentation that they are quite good at it. So you have all the documentation that will explain you how it's managed. OK? And for example, if you are interested about advanced API, you will learn the concept that I use, for example, based on long chain that I show you. It will explain you all the things you need to know. And if you are really interested about, for example, deep dive in the code, there is always a reference to what they use to build their systems.

Using Local Models and Systems

As you mentioned, when you work with companies, you never use ChatGPC, but basically always local stuff. I mean, from my first experience, when I tried to use Oulama on my local machine, with DeepSeq and so on, I mean, I was always disappointed with the results. I was never able to run the 70 billion parameter models, but always like the 10, 30, whatever. So what kind of computing It depends on what they need. Since last year, I used mostly Yammer 3. Usually, they have their own server, so they can allow some resources for the model, more than your local machine, I think. No, no, no, no, no, no. No, no, no, no, no.

Usually what I use, it's not the small model, not the quantized one, just the small and medium model. For example, from Yamaha, Yamaha 3. And I use a lot of RAG systems.

And I plug also the DB. On their own system, I almost never fine-tune the model, for example. Yes, basically that's all.

For example, I worked for a ARash company six months ago, and they were quite pleased with the model, and I used some tiny ones, like I mentioned to you, Yamatree, and JustFrag system, and it does the work. So when you build your own drug system, do you take it off the shelf or do you really build it from scratch? No, no, no. I build it from scratch. I was thinking of doing that once because my brother had a project that I wanted you to do.

But then I read that it's actually quite complicated to have all the edge cases and so on, like when you chunk the PDF, like that you get the figures and everything. For example, is this not really a problem? If you have a good model that is back up, but Yarmoutry does the work, I think with Enwritten also it will be enough. It depends also of, for example, I build several tiny agents. I don't build a big one, for example. Basically, one use case, one agent. I do not try to do some general.

Vertical AI Solutions

So you know the concept of vertical and horizontal AI. So I don't build horizontal AI. I only build vertical ones, which means, for example, I develop one agent for one use case. Not more than that.

And for example, the business case of the e-commerce, can you explain a little bit more about the workflow and For example, at least the customer order that I showed you, it's used. I just replaced in the workflow, it's triggered by Google Sheet.

But in the real system, it's just connected to Shopify directly. And I take the order from Shopify. I just take the new order and each time that there is a new order that is cancelled as shipping, the customer success management takes this case and they contact the customer for example. So this workflow exists in real use case.

And same for the call centers? For the call center, it's more complex. I use this system for allowing them to build dashboards and make basic data analytics. So, for example, you can... It's not a drafter of an email. There is no authorization for the call center. It's more to get information of what is going on.

Case Studies

Yeah, it's kind of a system that do data analytics for them. And also, and one other that answer to 80% of the case in an automatic way. For example, because it was for an energy company, and it's almost all the questions are related to invoices. So I built a system for that.

Yep. based on the expertise of the call center agent. So the first step is to get the knowledge from them. So basically, you build the system with them, not with the IT team and stuff like that, really with the users. And they explain you how they work.

For example, I go one week with one girl just to follow her and see what happens. And then I build the system based on that. Yeah, it's a chat-based and there is just one tiny dashboard made in Tableau. That's all.

Challenges and Solutions

Is this related to MCP or is it when you provide calls? I did not explore MCP for concrete use case so it's a new way we will say to manage AI in broader and yeah Yeah, I have a question because you mentioned about the invoices. Because typically what's happened is the invoices, it can be a paper. And then I'm just wondering, I tried to make something by myself, like hard coding in Python.

But I find there was an issue even when you are trying to read the PDF file. It doesn't really pass through the tables. Yeah, I will show you, for example. Yeah, there is a way.

Basically, if you work on PDF, you can use OCR techniques. I don't know. So basically, for example, you can try this.

And even, for example, for this case, it's really useful also because it allows you to prototype very quickly. So for example, there is a node that do OCR. I don't know where it is.

I don't know. I don't know where it is. But there is a node that allow you to basically do optical character.

is optical character recognition. So it's allow you to detect strange. Yeah, even if there is typos, even if it's not well formatted, it doesn't go inside of the PDF code. only take a look at the lookalike. So for example, you can use a NoCR stuff inside of this and it will give you a baseline.

Okay. For example. Is it able to detect the handwriting? Yes, yes, yes.

It basically stands for that, for the signature recognition, for image also recognition. It's the old-fashioned way to process it. And even now, in agent, there is some component that are based on this type of techniques because it works. And there are all the features you showed us. So far, there are all three? Yes.

Conclusion

The cloud is, how do you say, you have to pay for it. But you can see this one. This is a local instance. And everything run in local, for example. So as you can see, it's just the local host, so using the current network and my machine. And it will process also. Basically, you are connected now to Google API and stuff like that, but it works.

It's slower, but it works. Yeah. You want to say something?

Oh, I think I... I think we'll move to the second talk. And then what happens is afterwards, you can ask questions directly to Michael.