Introduction

Hello, everyone.

It's cool to be here in Sarajevo.

I'm not from Sarajevo.

I'm actually from Banja Luka, but at least five, six times a year I'm here.

Demystifying AI Automation and Goals for the Session

What I'm doing, actually, what I'm doing is I'm consultant for AI automation system and agenting system.

And when you say genetic AI, automation AI, et cetera, et cetera, it feels like you're confusing now.

It feels like complex.

And what I want to do today is actually to show that with just a couple of courses, a little bit of documentation, every person in this room can

actually do some kind of automation.

Of course, with the practice, you will be able to do some much more complex and robust system.

But even with just a little bit of your time, it's pretty simple to do something like personal assistant, some automation of the work, even some stuff that Ayla mentioned here.

I don't have presentation.

I would just do everything here, hands on.

we'll build this automation workflow.

I call it personal assistant, something that uses the AI agents that are talking to different other AI agents using different language models.

So let's go.

Designing the Personal Assistant Workflow

User Interface: Telegram Bot Inputs (Text and Audio)

OK, so what I'm building is

I want personal assistant that I will use the Telegram bot that I can communicate with.

I can communicate with the text, with audio, sending messages.

Then I want to give him some instructions.

And this AI agent, this workflow, has to actually figure out what you are asking from them from it.

And then delegate this to other agents.

So let's start.

Of course, I prepared some prompts.

It would take much more time for me if I didn't because bunch of things.

We said Telegram.

Let's start with the Telegram.

Telegram and we want to say that when some message is received from Telegram,

and I already connected my account, Telegram account, Telegram bot, I want to do something.

Let's just execute just to see if it's going to work.

Hello.

Cool, did something.

This is JSON format.

You don't have to figure it out now.

Probably some of you already heard about it.

But you don't have to be confused.

I just wanted to test that something came from my Telegram bot.

Cool.

So we already said that we can send message via text or audio.

So what I need to do is I need to make sure that

we pass to LLM, to AI agent, the text.

So in case it's audio, we need to make sure that it's transcribed, downloaded and transcribed so it goes to the text.

Branching Logic: Handling Text vs. Audio

So let me do, this is something called switch.

And first, what we need to do, so kind of first branch, in case we have the...

Sorry, I'm a little bit tall for this.

No worries.

In this case, we want to make sure that first branch goes in case we have audio, but then we also want to make sure that what happens if we just send the text.

Test, test, test, hop.

Not that one, sorry.

Something executed.

We see our message as you see this, test, test.

Telegram sent us more data, so we decide what to do with those data.

Cool.

Now we need to make sure.

We need to add another case.

In this case, if it's a text,

Exist we have our another branch.

So so the two different cases we have these We just need to do some small preparation for our LLM Okay, we will need to do this text and just a little bit variables and

We can drop it from here.

So in most cases, you will not need to type any kind of complex expressions or anything like this.

So text is simple.

It's already text.

But LLMs cannot understand the audio, so we need to get this audio and transcribe it.

Transcription Pipeline: Telegram File to OpenAI Whisper

And luckily, we have OpenAI and Telegram together.

So again, Telegram together.

We need first to get this file.

This file, we already saw how it looks like, but let me... Not that.

Execute workflow.

Let's just send one more test.

Hello, how are you?

Cool.

We have the something.

Let me do get file now.

Sorry.

Telegram.

Excellent.

And get file.

Get file.

Now we have this get file.

So only thing we need to do is tell him which file is this.

And also, we have OpenAI to help us with the transcription.

OpenAI.

OpenAI supports transcribe.

Transcribe.

Transcribe.

Recording.

Cool.

We will leave everything as is.

Cool, we have this, this.

First Controller Agent: Model, Memory, and Testing

So we are now ready to add our first AI agent.

AI agent.

Cool.

This part is this part is fine.

What we need, let's let's just quickly test it.

So we need to connect our audio branch.

And our these branches, let's see if this is going to work at all.

No.

So we have issue here.

Yeah.

So we need to do two things, you know, actually more things, but we need to send some

So we will use JSON.text.

I believe that we can test this.

Don't bother about this.

Also, AI agent needs more things.

By itself, the AI agent is pretty stupid.

It needs, of course,

some prompting and you already saw this when Stefano was doing, but I prepared.

We don't want me typing this and you're waiting, of course.

Let me just paste this.

Basically, what we are telling this AI agent, let's call it controller.

because it will do actually nothing.

But what we are telling them, look, this is you, you're doing nothing, but these are the tools that you are going to use.

These tools are going also to be AI agents.

And then, okay, we need to give them chat model.

Sorry, yeah, LLM model.

Of course, you know, again, that's stupid without LLM model.

We will use our open AI chat model for this, cool.

Another thing we need to do is the memory.

Because when we are chatting through the Telegram, if we don't have memory, they will not be able to read the context.

And without context, each message you send, they will treat it as a new chat.

So we don't want to do that.

For our cases, we can use just simple memory, and that's it.

And we need to give them the key.

Key is kind of chat.

session that we need to promote to actually let them which chat, because multiple users can communicate with this.

Each of these has its own session, so we need to tell them which session are we.

I already prepared this part so we don't have to wait me testing and blah, blah, blah.

Okay, this would be the key.

OK, let's test again.

OK, sorry.

Test.

Cool.

Doing something.

Excellent.

And let's see what it's being as.

This is the file.

This is transcribed recording.

Telegram trigger.

Get files.

Cool.

Text was here.

So far, we have something.

Let's add the last so we can test it fully.

Returning Responses to Telegram

Let's also be able to send message back to the same user.

Send text message.

Cool.

And we have already Telegram account.

We already said that we have this chat ID that we need to use.

And that's fine.

Everything that this agent throws out will be in JSON.js.

Output, yeah.

This way, AI agent sends message.

It decides what to send and send of course.

Let's see if this now works.

I will just say execute and listens.

Hello.

Will it work?

This message was sent automatically.

So think not yet really smart.

Why is that?

Because we tell them in prompt.

We tell them, hey, this is you, but you don't do anything.

You just delegate to other.

And just for this case, yeah, I'll help you.

We gave them focus on this part.

We said, look, you're doing nothing.

These are the tools that you are going to use.

Equipping the Controller with Specialized Tools

We have Think.

This is something that built in just lately.

And it gives opportunity to LLM to rethink of decision they made.

But we will also give them three additional tools called Milica, Eva, and Alma.

three agents that do totally different things with totally different LLM systems, but let us see.

First tool,

Let me just do this.

Let me just do this.

Thinking Tool: Self-Evaluation Loop

I will also give him the thinking model.

Again, with this thinking model, the thinking tool just gives you, again, re-things about decisions.

It goes again and says, hey, evaluate itself kind of way.

Cool.

Agent Alma: Creative Research and Recipes (Anthropic)

So next part is for us to give him

additional agent called agent tool we will call it Alma Alma is kind of you know, really creative AI and You know works with different, you know Creative things writing content researchers if you like to cook, you know, so of course if somebody likes to cook it's probably we need some French no, it's not French but Anthropic clothes as as LM system and of course

She needs to call.

So here is Alma.

She has its entropic chat model.

And what else Alma needs is, of course, prompting, because if we don't tell them what to do.

So I prepared the prompt.

Call.

Let me share the prompt.

System message.

And call.

OK.

and each AI agent needs some kind of tool, we will give Alma ability to go to internet to research some news, recipes, et cetera, and then come back to us and give us some cool advice.

Let's say advice.

Okay, get recipes.

Cool.

Get.

And we can quickly do this.

OK.

Let's see if Alma is ready to go.

Can you give me some cool recipe?

See, the controller already figured out that Alma is the crappy one, you know, because we tell it.

So let's see what's going to be given to our Telegram chats.

Cool, I found some cool keto recipes for you, low carbon, et cetera, et cetera.

We can ask them, you know, ask a bunch of the stuff, but let's just proceed.

So what we...

need to do next.

Agent Eva: Calendar Management (Gemini + Google Calendar)

Yeah, let's add something that will deal with our meetings.

Okay, so we will add additional agent.

Additional agent that comes to, sorry, let me just speed this up.

Okay.

Additional agent, we will call it Eva.

Eva is really organized and it uses Gemini model.

Gemini model is cool for those types of things.

Let me show you.

And of course, as every agent, it needs the tools.

What we will do, we will say we need calendar tool.

This calendar tool will be responsible to get events.

Get events.

And I prepared my small calendar for this presentation.

So now we have everything here.

So let me see if Eva is doing.

Oh, sorry.

I forgot to give Eva the most important things, which is, you know.

Here is your prompt.

Here is how you, because again, I said without proper prompts, they are pretty stupid.

Even if we don't tell them, hey, this is the current time, they will not know.

So they will think that they are in 2024.

Okay, so we are doing this.

We are seeing two.

So let's test.

Okay, let me just stop here.

Execute again.

Hey.

Any meetings today?

You see, it figured out that Eva is responsible because we tell them in prompt, use Eva for calendar and meetings and all that stuff.

So yeah, you have meeting scheduled today from with 9.30 to 8.00.

So yeah, this one.

Cool.

Eva works.

But Eva, we want also to give Eva opportunity to create some.

Cool.

Let's do, again, Google Calendar.

But we will, in this case, we will, of course, use Monster.

But we want AI to decide what's going to be sent here.

We also want to add something like summary, of course.

And AI will decide what exactly.

So this will be create event.

Cool.

Get events, create events.

Again, let's test them.

We have to test them, and on each step, we have to test AI systems.

Okay, but let's try with our voice part.

Hey, can you schedule a doctor appointment tomorrow at 9 o'clock?

Let's see if this is going to work.

No, not yet.

It's still waiting.

Oh, it needed some time, but let's see.

Please go to Create Events.

Yeah, cool.

I have scheduled doctor appointment tomorrow.

Let's see if it's real, because they, oh, look at this.

This is my calendar, and it really did create a doctor appointment.

Excellent.

So let's do the third and final one.

Agent Milica: Task Management (OpenAI + Todoist)

Let's have somebody that works with my to-do stuff, what I need to do.

We'll call it Milica because Milica likes to tell people what to do.

I have one at home, but she's not like that, I promise.

Cool.

Again, Milica needs the prompting.

This Milica needs the prompting.

It's really small prompting.

She's already smart enough.

Cool.

Yeah, that's why I scheduled it.

That's why I scheduled it.

Cool.

Okay.

So here is the Milica.

Milica needs the model and we will give her OpenAI model because OpenAI is pretty cool with those stuff.

And of course we need to give them tool.

I personally use Todoist for my to-do list.

So let's connect my Todoist list.

So this one we want to say get many, get tasks,

and we will let AI decide on other stuff.

And we also want to add opportunity for Milica to actually create events, Todoist, create, excellent.

We have project I've created again, project just for this.

We will, of course, let AI decide on this and we will say create task.

Cool.

Let me tie it up a little bit because I'm already losing myself here.

Cool.

So this is our Milica.

She has prompt.

Let's see if Milica is working.

Cool.

Okay.

What is on my to-do list for today?

Is it working?

Yeah, Milica is working.

Cool.

Oh, here are your task.

Prepare presentation and read the book.

Let's see, let's check.

Yeah, read the book and prepare presentation.

Okay.

Add new task.

Apologize to Milica for using her for this presentation.

Okay, I didn't tell Melissa.

Okay.

She needs to wait.

Okay.

Oh, she couldn't create a task.

Why is that?

Mine's done.

Oh, workflow successful.

Sorry.

I have added the task Apologize to Milica.

Cool.

And let me see.

Yeah, here is my task I need to create.

Wrap-Up and Getting Started

And again, I know that this is probably first time you are seeing this.

I'm doing it pretty fast.

But trust me, within five days of learning about this, you can do the stuff.

like this, yeah, progressively start fresh.

Even now, if you want, if somebody is interested, not sure, I will activate this workflow.

I'm not guaranteed for the stability.

Oh, sorry.

I will activate it.

No worries.

I will just remove this so I can activate it.

Even now, if you go to Telegram, not sure if you're using Telegram, but if you are using...

telegram, and you go to this bot, this milestone, low dash meetup, you will be able to communicate through your telegram with all these three agents.

That's it.