From the event: Mindstone Amsterdam October AI MeetupAnatomy of an AI Agent
View event

Anatomy of an AI Agent

A Real-World Company Brain

Tonight in my presentation we are going to dissect an AI agent and it's going to be a real one. It is the system that runs my own company.

Real quick about me, I'm a data and AI consultant with about two decades of experience. I have built and led data teams and I have trained over 300 professionals. And recently, this year, I went independent and I started my own company, Float AI. and float AI is helping companies to build a solid data foundation and drive

AI adoption okay so that was about me and I have also a quick polling quick show of hands please who has heard about concept company brain or second brain before.

Okay, half of the group, half of the room knows about it. That's really cool. The word is spreading.

From Second Brain to Company Brain

But for the ones who are not familiar with this concept, it's a team of AI agents that read your company's memory and carry out the workflows.

For me it started as a personal productivity idea. I wanted to write everything down in one connected system so that I do not rely solely on my memory.

But this quickly escalated after seeing this boost in productivity I wanted to create a company version of that and I used credits here, I used a seed from a company called Now. It's a company based in France and they are producing tools for analytics engineering and these tools are open source so if you want to know more about it just check online.

So then I created the company version of this tool and right now it remembers plants and acts on its own with me approving before anything ships.

The Three Pillars of a Company Brain

So the company brain has three pillars. It's three systems working together.

The first one is the repo, repositories, GitHub repositories, I hope you're familiar with that. Repo is where the agents are defined, roles, skills, prompts as plain markdown files on the version control and I hope you are familiar with markdown files but for the ones who are not familiar those are basically text files in plain English full of instructions to the agents. So in the repo we have like which agent who is doing what and how. This is what we have in the repositories.

And then the second one is the database. This is the the persistent state of your company's projects, tasks, deliveries, insights, and documents. And what has happened, what has been shipped, and what's still open are being recorded in the database.

And the last one is the front end. This is where a human sees and steers it. It's like a command center for your company, for your company brain. This is the screen I actually look at on a database.

So now that we are familiar with what a company brain is, it's time to cut open an AI agent. I will go full screen.

The Eight Essential Parts of an AI Agent

So, first thing is that there are eight parts in an AI agent, crucial eight parts, that make an AI agent work properly. I will go full screen here and the eight parts listed here.

1. The Brain: Models and Reasoning

I will start from the top, so the brain. It's the model, it's the reasoning core and this is not something we built it's something we call as a service from all the providers well -known providers like cloud AI, open AI, Parplexity and the model reads from the context MD file so that it can know the context of your company and it operates within that context.

All these providers are also offering a variety of models, foundational models, but in my case I use Opus for voice and judgment, SONE for for coding, and Haiku for easier jobs, easier tasks.

We go to the second one.

2. Identity: Role, Mandate, and Tone

So two is the identity, the persona of your agent. This lives in the role MD files. So this is basically the persona, the identity is what the agent is told to be. It's a mandate and it's tone.

And a few examples from my own system, chief of staff, for example, is told to be a strategic filter, not just a secretary.

On the other hand, another example, head of sales thinks in in pipeline stages all the time.

Head of data speaks in evidences and facts, not opinions. So these are real boundaries for the agents to have an identity and persona.

3. Orchestration Loops

Three is the orchestration loops. So these are the routines that the agents repeat on its own and on a schedule.

The best one I like from my system among a lot running during the day is the one that the chief of staff is running every morning at 6 a .m. and it gathers the day the context of today and then writes me a briefing about the priorities of that day and when I start my day I start by reading it through.

It lives in the loop folder in the readme .md file.

4. Memory: Durable Company Knowledge

Four is the memory. Memory is the durable knowledge across runs so that you don't need to repeat saying the same things to the agent and examples are the head of data should always keep in

mind that it should be running the company's own super base, database or the data app which is the analytics engine of the system, not a third party tool.

or another example is the designer should always use the house style spec for every artifact it creates.

So this is the specification I have given to the agent so that I don't repeat every time and it uses whenever it is needed.

Memory is living in two places either on the memory files in the repo or records in the database. So the agents can find the company's memory from these two places.

5. Skills: Callable Playbooks

Five is the skills, like hands. These are the callable playbooks for specific jobs. These are the callable playbooks for the agents whatever they are doing they call it and use it.

Yeah, it lives in the skills folder. There are like one night I I've gone a little crazy and I've loaded more than 80 skills in there. I wouldn't recommend that so we try to be picky and whenever you are getting a skill from an open source, try to take it from a security check so that you don't get garbage from open source.

but an example for the skills is that an agent can audit a web page using the front -end developer skill for instance or same way an agent can create a money board entry using the accounting skill.

6. Tools and Connectors

Six are the tools and connectors. These are outside systems that agents can read and act through.

My system is connected to a lot of outside systems but the cool thing about these connectors is that you don't need to leave your own workspace space to do a work outside of your system.

You just use your own company brain and thanks to these connectors you can basically carry out a task in an outside system.

Some examples again among many are the Telegram, Google Calendar and Moneyboard integrations and yeah thanks to the engineering team if there If there is not a ready -made connector, they can write one suitable for our own system.

7. Decision Boundaries

Seven is the decision boundaries. This is the clear split between what an AI agent can do on its own and what it needs to escalate to me.

Very nice examples are emails. They can draft an email but they can never send it out.

Or they can create a money word entry in the accounting system but they will never file this to an official.

So these are the boundaries that agents can do and cannot do.

8. Output and Escalation

Last one, eight, is the output and escalation. This is basically the communication channel between me and the agents.

I have created a special kind of approach while building this company brain. I have a report, sorry, I have a delivery oriented approach company brain which means every time an agent produces a result, it will report a delivery on its own and these deliveries will be, I can follow them from the dashboard, from the front end.

In other cases, they need to escalate the situation to me. In those cases, I'm receiving an alert on my watch or in my mobile that the agent is is requiring me to be involved in the task.

So these are the eight, let's say, parts of an AI agent so that it can work properly. So eight crucial parts make one proper agent. And a team of agents make one properly working company. way.

Meet the Agent Team

So how does it look like my team of agents? So meet my team.

Thanks to them I can build great stuff, I can grow my abilities, capabilities, I can learn more and more in this rapidly changing environment and I can also run the operations of my company.

And before you ask what the CAO stands for, it's a self -appointed title which is Chief Agent Officer. Yeah, funny, nothing serious.

A Look at the System in Action

Okay, now we are going to see how it works. Basically I was planning a live demo but to be honest just for the sake of time and also I didn't want to expose too much personal information I shot a video and I will start it now. It's fast forward so double speed.

What happens here is this is my observable screen and I can see whenever I ask a question I can see what agents are doing what which agents are doing what and right now the question here is give me two three updates from the agents who were active recently and as you see

these were the agents that were producing results, deliveries in 24 hours, in last 24 hours and chief of staff is coordinating them to get the updates. So normally these kind of turns are running about 3 -4 minutes.

Now with the double speed it's less about one minute but whenever the task is completed it goes green so that I know the agent produced a result.

Yeah I think yeah that was it. Thank you, thank you.

Key Takeaways

Just to conclude three things to live with. There There are three systems working together to make up a company brain.

An AI agent has eight crucial parts, not just one model. And a human still approves before anything goes out.

I hope it was clear. Thanks for listening. And I can take questions.

Finished reading?