Introduction

So good evening everyone, my name is Marcus Klug. I've been a software developer for 30 years.

Sorry, I think you need to speak a little bit louder. Okay, I'll try that. Is it not? No, that's fine.

All right, okay. Sorry folks, I'm starting again.

Thanks Michael for the nice introduction. My name is Marcus Klug. I've been developing software for 30 years.

mostly had small companies, largest I think was about 30 people but then for the last five or six years I've been very obsessed with AI.

A little bit before the JetGPT era started I basically went solo and started to do some consulting around AI and custom software development for the things that could be built at the time and today I want to talk a little bit about

first a bit of history, how it has evolved from the early days, how AI -assisted coding has evolved from the early days to where we are now, a little bit of the outlook and then I'm showing a few tools that are now basically the new crop of tools that come up because we have completely new problems to solve that basically have not existed in traditional

From Autocomplete to Chat: The Early Evolution

additional software development at all yeah so um it started in 2022 there's a yeah there's a

GitHub Copilot and the “Glorified Autocomplete” Phase (2021–2022)

timeline here technically it started with the technology preview of GitHub co -pilot in 2021 it was a actually open AI based model that was trained on a lot of the GitHub code base at the the time even then there was quite a bit of controversy because people were worried that their code from their private repos is used for training and might leak into the suggestions that

users get but what it was at the time was really just as I say here in the slide glorified autocomplete so the user experience was that you had a file open in your editor and then because you were writing a function debounce which is something that the model has probably seen a million times in the GitHub code base that it would suggest you some ghost text, some

so -called ghost text that you could then accept via a tab and then it would basically give you that code that you would have more or less written like that anyhow. But it was pretty basic.

Yes, it would help with some boiler plates and some super standard stuff instead of copy -pasting it from somewhere else, you would have it auto -completed. But the limitations were of course that it would really only have the context of the file you

currently had open. It was not a very strong model by today's standards, so it couldn't really do a lot of interesting stuff. But it was a start and that got the first people

hooked on, hey, maybe we can use this AI thing to write software at some point. Then came

ChatGPT Arrives: Explaining and Editing Code via Chat (Late 2022)

the JetGPT era, which was in, I think it was, what was it exactly, November 2022, I think JetGPT 3 .5 was released and suddenly you could do things like and that would be the first thing you would suddenly have the ability to paste in your code file and ask JetGPT to add comments or maybe explain something that you wouldn't understand, maybe it was

in a language you usually didn't know and then instead of going to Stack Overflow and asking there you would just ask JetGPT and it would give you a pretty good answer which

which also led to the quick demise of stack overflow, which I don't know how many here are developers, but there was a software where you would basically, there was a platform on the web where you'd go to ask about coding problems, and it was a super popular platform for about 10 years.

And then there's this graph where you can see from 2023 it really pretty much plummeted, which is a little unfair because also a lot of the training for JetGPT or GPT -4 and 3 was done on content from Stack Overflow so it definitely contributed to its own demise.

Anyhow back to the slide, so the dance was then you would have some code, copy it over in the chat GPT window, get some answer, copy it back. Was not a lot of fun but at the time it was the best we had and we thought it was already quite the trick.

Chat Inside the IDE: Sidebars and Workflow Shifts (2023)

and then we started to have the first chat so basically and then then somebody said hey why don't we have the chat directly in your in your IDE basically in your visual studio code or whatever you were using where you were writing your code so you then had your one window with the code and you had a sidebar with with the chat and the good thing about that is that yeah you

You still sometimes would copy and paste stuff over, but also the integrated sidebar could then sometimes also see the code that you had currently selected and you could directly ask about it. It would have the context of your currently open file.

If you were really lucky, it could also see the neighboring tabs. It was still relatively basic, but it was the first time that you actually had some feeling that, hey, there's a real tool now, there's something that really changes my workflow. flow.

I still write most of the code, but I really now spend at least some of my time in that sidebar. That was about, yeah, that was 2023 after JTPT came out and then stuff got bolted into all the traditional IDEs.

So the IDEs that were already around for the non -developers, it's integrated development environment. It's just the program you used used to usually write software. It's called an IDE.

And yeah, so there was a lot of basically existing market leaders that bolted that stuff in.

And then we had a new crop of IDEs, which were basically rebuilt with AI.

Oh, there's something, somebody at the door.

AI-First IDEs: Cursor and Multi-File Edits (2024)

Which were basically rebuilding IDEs IDEs reimagined on first principles around AI so you didn't only have a sidebar that you could use and chat with you suddenly had had a built -in AI assistant that would know that could see the whole file tree it could open

and close files it could open files make changes then present the changes to you you just had to say something yeah accept that change accept that change so So it would touch multiple files, make multiple edits.

It was a really good experience already compared to what we had before. Still pretty bad compared to what we have now. But it was a new set of tools that didn't exist before.

And these were called, the most capable one was called Cursor, they were also the first. and Cursor you might know has recently been integrated into SpaceX so I think it was even only a few days ago that they that Elon basically pulled that option to buy them for 60 billion so I think the guys did quite well and now they and

now they are basically helping Elon making croc better for coding it's still a very good tool. I use it but as we will see later in the talk it's just getting more and more

obsolete to use. IDEs are getting more and more obsolete.

Agents Beyond the IDE: Cloud Code (Early 2025)

And then finally early 2025 we had the first version of cloud code because maybe if I go one back you can see here the the limitations that

you still had even with the AI -first IDEs they would be able to work inside of your code base in the IDE and open and close files, but they couldn't really much jump go outside of it. They couldn't do random bash tool calls outside of your project route, so they couldn't basically control your computer.

They could not do arbitrary stuff, surf the web and do stuff and come back with solutions. They could really just work inside your IDE, but not really outside of its boundaries.

With Cloud Code for the first time that was a completely new form factor.

You would have a terminal, you would start your Cloud Code session and Cloud Code would literally be able to do everything that you could do yourself on the computer. It could open directories, look at the listings, look at

the files inside, come up with, basically explore your system, suggest solutions to you based on what you already have or what you've done in other projects.

I I remember the first time using Cloud Code, for me I was almost shocked how profound the experience was and how helpful it was and how capable Cloud was.

And this was back at the Sonnet 3 .5 times, so compared to that it was again super basic but at the time it was the most capable model when it came to computer use. use.

Cloud Code Limitations: Visibility, Context Windows, and Repetition

The limitations that Cloud Code still had was that it was sometimes, so you wouldn't sometimes know, it would just go on for a long time, you didn't have a lot of visibility what was going on, what it was doing and it had a relatively short context window which means that after working for 10, 20 minutes on something it would no longer know what

the beginning of how it all started. Basically, the context of what you were working on and stuff you discussed a couple of conversation turns before was no longer in the memory. So it would

keep doing, for example, when you're working on a longer bug after 20, 30 minutes, it would try the same fix that we already found doesn't work like 30 minutes ago because it simply entirely forgot about it. That got better over time because then it had memory and larger context windows,

but the first iteration was relatively basic. Okay, I try to speak up more. I already know tomorrow I'm gonna have a very sore throat. All right, so the thing was,

then when working with Cloud Code, as I said, sometimes those turns would take really long. You would ask something, you would type a command, and then you would wait for a long time

for Cloud to walk through your system and list directories and open files and look at stuff.

And the thing was that, you can see here, you would do something and then the agent would do its thing but in all those times that you were idling and doing nothing at all and it was basically like this you had one cloud at the code at the time and there was just not not good enough so what you would then do obviously you

would add another agent then while this agent was working you were talking to the next agent to talk talk about what they should do, and the next, and the next, and that then came to a point where you were pretty much

all the time either checking output from some previous agent or prompting a new one, but it was relatively good. So you really felt that you were very productive because you had a lot of cloud sessions going on.

Human Factors: Parallelism, Productivity, and Mental Load

The problem here was, for me personally, but I also heard it a lot reflected in other people on youtube and and where not that um the um the mental load that it puts on you is quite immense if you work like that for a couple of hours you're really completely exhausted i remember in the

beginning when i just thought i can still do like i always did i at 11 or so in the morning i was already pretty exhausted the thing is when you normally code then you code for a while and you're in the zone and you do stuff and then it's really the one thing that you currently have to focus on

and it you have something we also as humans have some kind of a context window and we have a certain amount of concept that we can have on our head at the same time so you work on this one thing and you you do it and you work in line line by line and that's fine sometimes you go off to stack overflow to check something so that was the classic flow and then you would go down some

rabbit hole that was entirely unrelated to what you were working for for 30 minutes. But it was all good and healthy because then after eight hours of work you got a lot done but you also had those quiet times where you really just had to keep one thing in your mind.

This is entirely different now when you orchestrate agents because basically you are presented with questions that have a lot of consequences constantly. constantly.

So the agent is presenting to you, I've changed those 20 files and now you have to tell me, shall we do it this way and this way? And then it comes up with a really complex explanation what the outcome is if you choose A or B or C.

And you have to make those decisions constantly. So it's actually quite, on the one hand, the productivity really is increasing by a lot.

And on the other hand, at least personally, I can do that maybe for four or five hours a day and then I have to do some other stuff like, I don't know, read emails or whatever, I certainly cannot do that for eight hours every day. But hey, I'm old, most of you are young, so you probably don't have that problem.

Scaling Up: Sub-Agents, Background Tasks, and Isolated Work Trees

So this was basically the moment where you realized you need to do more thing at a time because it would be a super waste of time if you just wait for your one agent doing its thing. So there were all those new techniques that you would do in order to go parallel.

There were sub -agents and background tasks, and you would have multiple versions of your code base checked out at the same time. So the different agents could work on different parts of the code base. Maybe you can point them to GitHub.

They will look at the open issues and then just from the top spawn four agents and work one issue at a time. And in order to do that properly, you need to give each agent its own isolated work tree of your code. Otherwise, they will start editing the same files and it's going to be a huge mess.

Remote sessions is another, an unlock in the one sense and in the other sense it means that even now when you're not on your PC but you're on your phone, you can keep doing cloud code sessions, which again is a very productive thing. it's not necessarily a healthy thing but the good thing is you can do it you can now basically

i can open my phone and i can tell claude to work on something my laptop doesn't have to be running it will be it will be um spawning up a virtual machine in the cloud it will check out my github code it will work on it and then later it will create a pull request and i can do all that while i'm sitting on the beach it's a great time a great great way to spend your time on the beach

From Single Agents to Fleets

Yeah, so we are now in the fleet era where you no longer have one cloud doing stuff, but you basically write a prompt, which then spawns up a couple of clouds. We're going to get there in a moment. And so I just want to show something here.

So that is basically the timeline, just as a recap for what we've lived through. So we had the glorified autocomplete, then we had the chat, then we had the IDE sidebar, which was basically just a chat integrated into your IDE. Then you had those IDEs that were reimagined for the use with AI.

At some point, we had agents like Cloud Code, and now we're talking about no longer single agents, but we're talking about fleets. The two new things are here, which are both from this year.

We are now basically also in an accelerated timeline where a trend is no longer measured in years or sometimes even months.

it's more like what you still do loop engineering that's those three weeks ago we have all moved on to graph engineering i'm only half joking because i'm going to show you some tweets that that show

Loop Engineering: Automating Task Pickup and Completion

exactly that so just first loop engineering that was the idea now hey why why should i still be the guy who who basically goes to github and then tells claude what issue to work on why don't i just say hey here's a bunch of work to do it's a list of 100 issues and i want to now create a loop that an agent can go pick up the next thing that is to do there's somewhere

there's a definition of when it's done the agent will check whether or not the goal is met and if it's done it will find the next thing in the in the batch to do so rather than you sitting there and and claw and prompting coding agents one at a time you now design a loop that allows agents to basically without you do the thing pick the next bit of work that needs to be picked up

and loop engineering has been around maybe well it started with the end of last year there were people who created this Ralph Wiggum loop that some some of you might know it's a Simpsons character which is very single -minded and really is only after exactly one thing okay and I'm gonna keep it up so the Ralph Wiggum

loop was a great thing because it it was yeah people came up city hey why can't we just make our agents as single -minded as a ralphic and they just you give it this one thing to do and it simply will not stop until it's done it will try again and again and you can it runs overnight it

became a little less popular when the big model providers started um not so much subsidizing the tokens anymore and you couldn't do that forever just on your cloud code subscriptions suddenly Certainly you had basically usage limits and you could only do so much and after that you would have to pay per tokens and then that basically caused a lot of those crazy loops to go slower again.

You might have seen in some news that a lot of people have blown their whole AI token budget for the year after a couple of weeks or months into the year and people became much more token aware. So there was a short period of time where this word token maxing came up a lot.

So CTOs were basically bragging to each other how much tokens their employees are using and the tokens per employee was basically a measurement of how hot and hip and productive your company is. But it was always a little bit of a silly measure and it very quickly ended when they realized that they actually have to pay for those tokens.

tokens and a lot of tokens used doesn't necessarily mean a lot of value created.

Graph Engineering: Role-Based Teams and Orchestration

Then the next iteration is graph engineering that is you don't only have a loop you basically have a team of agents that have different roles within your different roles within working on a task.

There's guys who explore and find facts in the code base that are needed to do the work. Then there's going to be guys who implement. Then there are things that do the review. They check whether or not it complies with your coding standards, with your conventions, whether or not the tests run through, whether or not everything is basically inside the performance budget and so on.

So there's a lot of different roles that agents get and then they will report to each other and then there are orchestrator agents that have the job to just, well, orchestrate the agents and tell the guys what to do in which order and in the end everything goes to a so -called merge queue so all the different changes that have been done by different ages are put in a queue and then somebody decides in which order to merge it in the code base.

So it's getting more complex and the graph itself is now basically basically a thing that you think about first and then you have created like an artifact that you can change from per task. You can for different tasks create different kinds of graphs and let agents run through the graph basically.

We've seen this slide. This is is the slide I've been taking for this. Okay, I'm going to be real quick. Let's skip that.

This just I want to quickly show that because so this shows you basically how we went from the green area is everything that is still that is still code and then a very little bit is agent. So the green stuff is code. And the blue stuff is agent.

In the beginning, it was all code and then a very little bit of fear. Do you want this auto complete? complete. And then we started to have a sidebar and then the sidebars became bigger and then the code was only, the agents would only show you the code that changed and you would basically just have a conversation.

Then you would have cloud code which was all code and all agent and almost no code. Then you had multiple agents and in the end this is where we are now. You have an orchestrator of agents and we have a hierarchy of clouds that are orchestrated.

There's a very good blog post from Steve Legge, who's a complete legend.

It's called Welcome to Gas Town.

Gas Town is one of the many tools that have been created in order to solve this, how do I orchestrate agents?

And here he talks about those different stages in which you can be as a coder and you should locate yourself. And if you're not at least in stage seven, you don't need his tool.

That was basically the reason why he came up with it. But I think it's a nice

The Overton Window Shift: Changing Beliefs About Programming

Yeah, and then this one is that slide that where I want to talk a little bit or show how the how the overton Window has changed from how we got from this is 2017 and Rekha party the the guy you might know as a co -founder of open Ai head of I and Tesla He wrote a blog post software to all in 2017 and it was considered complete science fiction.

They didn't somewhere along the line there will be be, code will not be handwritten and software will be done in a different way. This is from January 2023, the hottest new programming language is English.

The guy from Stability at the same time said in this year that there might not be human programmers in five years, I think he's still on time.

Yeah, we had quotes from Hans and Jan, I'm not going to read them all for you but you can see for yourself how quickly it's changed and now we're at the

the stage where here you have this guy who that's the open clock creator Peter Steinberger who said in June hey you should not prompting your agents anymore you should just design loops and then a few weeks later are you still talking about loops or do you already do graphs yet and this is this is really only half joking because this is how we are how we are moving

forward and things that have been considered and ridiculed like things like what Dario Amore I said middle of last year that 90 % of AI's code is written by AI, so 12 months later I think, at least in the people who are locked in, it's more than 90%.

I don't write any of my code anymore and I only know a few people that do. It's funny, I still know a lot of developers who've never written a line of code with AI, but the ones who use it, use it now almost exclusively.

Closing Thoughts: New Problems, New Tools

And we are now at a point where it's not even about you look at what the code the agents agents do anymore because you can't they generate so much code that you cannot even feasibly look

at the diffs so we now need to solve new problems and have new tools but I think I ran out of time

so we're gonna we're gonna have to do that next time but come to me after the later in the networking and I can show you a couple of tools thank you