Introduction

Great to be here.

That was a fun little exercise. Got to meet some people.

And now I get to say for a longer period than two minutes what I've been up to.

All right.

Ancient History: Early Attempts at AI Programming

I'm going to start with ancient history.

Go way back to circa the 2000s and people started to say, can we generate programs automatically using AI?

And then it was mostly things like genetic algorithms and they made sort of like fake simple programming languages I

played around a bit with that then in 2017 we had these neural nets and I was with the team at Google motion much on the Leslie and we said let's take advantage of those but the way we got to take advantage of them is we got to have

them build trees and then manipulate those trees because programs are trees trees, right? And so we built this dynamic system for manipulating trees of different

size because neural nets were built to have things of one size, and we built that. We thought we were going in the right direction, but we really missed the boat.

And this was in February of 2017, and almost at the same time, or a couple months later, in a couple doors down the hall, Noam Shazier and his team were saying, ain't anything fancy like

trees just feed in a bunch of sequential text and then we'll do something a little bit tricky with where you pay attention to the past and it will all work out and so he was right we wasted our time trying to parse programs into

trees and just feed in the text that's all you need and Rich Sutton's bitter lesson is that's usually the case it's just end -to -end having more data you're you're going to win.

From 2017 Missteps to Modern Breakthroughs

So we were going down the wrong path, now the field has recovered. Let's do a little bit less ancient history, bring us up to 2023.

So DeepMind released this alpha code that scored sort of in the middle in programming contests compared to human coders.

This was the key example they put up of saying there was a problem, here's the code our system generated.

It got the right answer. That should be good. And yes,

it's good that they got the right answer, but that's not all there is to programming. And before anybody can check in anything into my code base, it's got to be code reviewed. So I did a code review, and it didn't do very well, right?

There's a lot of problems there. And I won't go through all of them, but I'll go through just one.

A 2023 Code Review: What Went Wrong

So right here, this is a problem it doesn't really matter what it is but you know you they're manipulating two stacks and popping things off the stack and right here they say well i'm given two input stacks but i'm going

to introduce the third stack c and whenever i pop something off of b i'm going to store it into c because you know it's sometimes it's good when you pop something off not to lose it and then it never

uses stack c anywhere else in the code and i know programmers who do things like that and i don't want them touching my code, right? So that's where we were in 2023.

I think of this as a Lake Wobegon effect.

Part of the problem is half the programmers are below average, and that's what it's training on. So that's what you get.

But modern history, 2025, next week will be 2026. It's a lot different.

Vibe Coding and the 2025 Shift

So Andrej Karpathy famously said there's this new way of thinking about it, of vibe coding.

I went in and asked Claude, solve that problem from 2023. It gives a great answer. It has no problems whatsoever.

You don't need to code review it. It's efficient. It has a good analysis.

You can ask questions and it can come up with arguments for why it's correct. You can ask it to optimize itself and it'll say, yes, here's some things you could do to make it better, does everything you'd want an engineer to do.

Advent of Code: Human vs. LLMs

Now, some of you may have been playing this game called Advent of Code, which every December there's this sort of programming contest and there's puzzles to solve.

I've been doing that most years when I have time, but this year I said if I can do it on my own, I'm also going to ask LLMs to do it, and they did just great.

They solved all the the problems. They do it a lot faster than me.

I think my code's nicer, and my code does happen to run faster, but they're definitely right in the ballpark, and they get each solution in a couple minutes, whereas it could take me an hour to do it.

So AI is definitely changing software development.

Who, What, When, and How of AI in Software

I'm going to talk about the who, the what, the when, and the how.

So the when is now, right?

If you had wanted to say, I want to use this tool in 2023, I would say, don't rely on it. And 2024 maybe for early adopters, 2025 and next week we're going to be 2026.

This is the time. It now works.

Latency and Developer Flow

Although I got to say one thing is, you know, when we have our VS Code or other IDEs, it gives us a lot of help kind of instantly. You know, I want to know what are the methods for this class or what are the arguments to this function. And that pops up instantaneously.

And that's really important if you want to be in the flow. If you go slower than that, you kind of get out of the flow.

And we don't have systems now that can answer these questions in that kind of immediate feedback. It takes a couple seconds. So if we fix that, they'll be even better.

Who Benefits and When to Use It

So who is this for? I think it's for smaller teams.

It makes experts better. It makes non -programmers better. And we heard an example of that back here.

It's great for prototyping. you make prototypes 10 times faster.

For the whole software life cycle for a serious product, maybe it's like a 50 % speed up.

And we saw a couple weeks ago, Linus Torvald says he's okay with vibe coding as long as it's not used for anything that matters. And maybe that's a good point of view.

If you have very complex kernel code that's going to be used by millions of people, you want to be really sure.

From Roles to AI Teams and Agents

There's all these roles in the software development team. I think one of the really exciting things is saying let's have an AI do each one of those roles and let's have them Interact together and now I've got a whole team working together

Can there be an agent for each role and maybe for each feature you assign a new AI engineer? Say you're gonna be in charge of this feature and then you're gonna pass it off to this QA AI and so on.

I think we don't know yet how to organize These teams of AIs and how they interact with human teams

Rethinking the Software Lifecycle

You've all seen things of the software development lifecycle. Now, one of the things that's amazing about neural nets, when we get to the implementation stage, is they can do this back propagation.

If there's an error, you can feed that error back and you can fix it. But right now, you can only do that for that one small step.

I want to do that for the whole software development lifecycle, right?

I want to say, if there's a bug in my planning or my my documentation or my maintenance or anywhere else, I want back propagation to go throughout the whole cycle and fix it up.

A Gift: Building Better Learning Tools

So we're all asked for a gift to share. So this is one thing I've been working on.

So I have been working with Steve Johnson in the Notebook LM team, and I think that's been a lot of fun.

but I feel like notebook LM is great when you have a set of documents that you want to understand but I've talked to several people who said oh you know I love notebook LM I put in all the documents from my team and now I really

understand what's going on much better and I went to share it with my team and I was so excited and they looked at it and said well what do I do with this and And it's like, if you didn't build it, it doesn't really make sense for you.

And so I think it's mostly a UI issue that we can make that better. And I want to take a slightly different stand on the continuum of possible UIs.

From Static Textbooks to Interactive Dialogues

And another way I think about it is, so I've played several roles in my career. One of them is as a textbook author, and I'm proud of that, and some of you may have read some of the stuff I did.

But I also feel like it's a strange situation where it's like I'm in complete control. textbook's a static thing that I wrote. I don't even get to talk to you, the reader, and I hand it to you, and your only choice is do you want to read it or throw it away or skip part of it.

So it's sort of, I feel like I had too much control in that interaction, and you didn't have enough.

On the other hand, the other side of the spectrum, we have these language models with a search box that's completely empty and you the user can put in any question you want and then the system will just play along the system has no no role whatsoever and you the

user are completely in charge i want something that's kind of in the middle that says sometimes the user is going to say i'm interested in this and the system will respond other time the system says well i'm going to quiz you and i'm going to get you back on track and there's a body of knowledge here that we're trying to teach that an author has put together and it's going to be a free flowing conversation back and forth but there is somebody in charge

Introducing LittlePaper.ai

to try to get you there so I built this thing with my colleague Peter Dannenberg

there's a GitHub repository there's a URL

it might look like a strange one so why is it littlepaper .ai so it's a nod

to this book called The Little Lisper from like 1990 and this was an interactive book it didn't have a large language model behind it but what it had was a series of questions and answers and the way you were supposed to read

this book is put a piece of paper over the right -hand side and read the question answer it yourself and then slide the paper down and see what what their answer was so it was making you work it was interactive in this kind of

of weird way that you can do on paper so that was a little whisper and then PA IP is my textbook paradigms of AI programming it's an old textbook so in that sense it wasn't the greatest choice but that meant that it's out of print

and I have the copyright so I can do whatever I want with it whereas some of the other stuff I felt like I didn't have the copyright to okay so how so

Beyond Programming: Toward Problem-Solving Systems

So programming is this process of defining step -by -step instructions is kind of the main definition. But is that really the way we want to define things, right? So then if you look at problem solving, it says problem solving is a process of achieving a goal by overcoming obstacles.

And problems in need of solutions range from blah, blah, blah. And it seems like this idea of overcoming obstacles is more important than this idea of step -by -step instructions.

So maybe we shouldn't be thinking about what's the future of programming, we should be thinking about what's the future of problem solving.

Maybe we don't need programming languages, maybe we need problem solving languages and problem solving tools.

Metaphors: From Micromanager to Partner

So here's a typical programmer, you might have seen this picture before. This is Margaret Hamilton and this is the source code for the Apollo mission. She was in charge of it, she's credited with coining the term software engineering and And, true fact, she was my boss in my first job out of college.

How cool is that? But whenever I see this picture, I'm reminded of this picture. And that's how it sometimes feels, like, when you're working with code, right?

You feel like you've got to strangle it, and you've got to use the force to make it do what you want to do. And it feels like you're kind of micromanaging the process, and nobody likes the micromanager. Darth never won Boss of the Month award.

So maybe we need something else and not a micromanager. Maybe it's a teacher, a partner, a community. Those are the metaphors we want to go for.

Design Patterns, Architecture, and Livable Systems

This design pattern is very influential within the software community, but the gang of four there got the idea of design patterns from this book, which is a book about architecture, like real architecture, building buildings and cities and towns and so on.

and systems of livable communities so maybe that's the way we should be thinking of the metaphor of what programming is it's a it's a way to design livable environments not a way of giving the computer step -by -step

Biology as Inspiration for Robust Software

instructions I think this is a good book by Hansen and Sussman came out a little bit before the AI boom but they say you know biological systems use contextual signals that are informative rather than imperative there's no master commander

saying what each part must do right so so I have a brain but it's not like the brain is is has an instruction counter and is having every single step it's more like you know all the cells of my body are kind of communicating with each other and that makes us biological systems more robust so maybe building

Conversational Languages and Human–AI Partnership

software should be more like that and then what maybe it's a conversational language rather than a programming language something that's good for a human AI partnership.

Reimagining the Operating System

In terms of the operating system, you know, right now I want a pizza or a car to appear, and I go out and I push one of these buttons, and now I've given complete control of my data and identity and everything else to Domino's or Uber or whoever.

Maybe I don't want a system like that. Maybe I don't trust them.

Maybe I just want a system that has one button button that says this is my personal assistant and I trust it for whatever reasons and now it's going to go out and manipulate with everybody else on my behalf rather than on some big company's behalf.

So maybe that's the future of operating systems.

Q&A

So let me stop there and open it up for discussion.

On Trees, Hierarchy, and Reasoning

I don't think the trees were wrong.

So first of all, I was a member of like your first massive online class with Sebastian, so thank you. you.

Yeah, you're welcome.

When I heard you talking about the idea of programming being step -by -step and then problem solving being the solution, I immediately thought you weren't wrong with the trees because to me problem solving is hierarchical by nature.

Do you think that there are hierarchical architectures that we can use and essentially like plug our existing models into those to help solve some of that like the latency problem with development code or Or, you know, other problem sets where maybe the whole thing doesn't fit into a context window.

Yeah. So we can have, like, a parser that breaks it down. It's a great question.

So I think for AI systems in general, not just for solving programming problems, I think you're exactly right. That hierarchy is important. We haven't done a great job with that, right? We've done a good job with sequence.

And we can kind of fake some of the hierarchy stuff. stuff, and we use chain of thought and things like that to kind of remind ourselves of higher order concepts, but I totally agree we could do a much better job of building hierarchical reasoning into the core of these systems, and that hasn't been done yet, and there's people playing with it, and you see a paper every now and then, but nothing's caught on yet. yet.

On the other hand, this idea that, you know, right, why do we build neural nets? Because we want to understand natural language and nobody knows the grammar of natural language. We know the exact grammar of Python, shouldn't we use that? And the answer is, yeah, you could, but you don't need it, right?

Because the language models we have understand Python or JavaScript perfectly without having this hierarchical structure so it's not going to help them there but I think it will help them with the reasoning so so that's a great point yeah so in the past

Jobs, Disruption, and Adaptability

every time we've had new technologies that create new wealth that's created more jobs and more opportunities and mostly I think that that will happen right that the there will be more opportunities but I think that the way in which it happens will be different and some people could get caught out

right we had when we had that in the past when you make technology switches which is some people who aren't adaptive lose out, right?

So, you know, we went from a country 100 years ago where a huge percentage of the population were farmers to now we've automated most of the farming jobs, so we only need a couple percent of the population to work as farmers.

And mostly we see that as a good thing. More people get more food for less cost. but if you were one of those farmers that lost their job then you see that as

a bad thing I think in the past that was tolerable because it was kind of generational right so you know I would be moan oh you know my parents my grandparents were were farmers and I was a farmer but my darn kids they went off to college and looks like they're gonna stay in the big city and do some job in

an office building and they're not gonna be farmers and I'm upset about that but but it doesn't affect me personally. But now it's like things change every year.

And if you aren't flexible enough to adapt to that, it can cause disruption. So I think the people who are flexible, they'll find a job, there will be things to do.

It won't be exactly the same job as you were trained to do if you were trained to be a software engineer a decade or more ago, but there will be opportunities.

And if you can take advantage of them, you'll be in a great place those who can't take advantage of them could have disruption

i should do that yeah yeah so you should be flexible you should learn about what the new tools are you should uh learn about uh domains of application right so i think it was kind of of strange that you could make a career from knowing the syntax and the modules from Python

or Java or JavaScript without knowing, you know, you're in the medical domain or you're in the legal domain or the financial domain or something like that. So I think in the future there will be more of a combination where you say you have to really know the business domain that you're in and then you also have to know some of

Will Managers Build Apps? Education and Accessibility

this technology stuff of how to interact with software packages yeah so I think I think they will, right? So I think it depends on what you mean by cool, right? So they'll be doing a lot of prototypes, right?

So real product, you'll need people who know what they're doing to make a real product, but to make these kind of one off things, I think you're right that the managers will be doing that now.

And I've already seen that, so I made my class do that at Stanford, right? So it's a class with no prerequisites. So some of them are computer science students who know all the technology, but some of them are from business school or law school or whatever,

have never programmed before. And in one of the classes, I just said, for the next half hour, everyone's going to build an app. And use, you know, here's a URL for cloud code,

or use whatever you're used to. And in that half hour, everybody built something. And a lot of...

What you just said is another version of Hello World. Everybody can do Hello World, but it's not real. Yeah, but it was more than that, right? So it was more complicated than Hello World.

And then another thing I thought was really interesting is we have this internal tool at Google, and people that use it every day love it, but it's grown over the years, and so there's all these menu bars along the top, and when you pull one down, there's lots of different commands. And so the people that use it every day love it.

But a manager who wants to use it once a year has to pull one of the engineers aside and said, you know, I wanted to make these queries to this system. I don't know how to do it. Can you show me how to do it?

And then we just connected it, put it in a little search box for Gemini. And now the managers say, oh, I don't need to go to the engineer anymore. I knew what I wanted to ask for. I wanted to know, show me a chart of the blah, blah, blah, blah, blah.

And I knew how to ask that. I didn't know how to press the buttons but it gives me the right answer and so I think we'll see a lot of that kind of thing and you know we've had that kind

of disruption before it used to be you go to a business and they had a whole typing pool of people whose only job was to type up documents and we don't have that anymore people type their own documents now and there's a little bit of help to do that but mostly we've distributed that work and so some of

this programming work will be like that i agree with you that we're not ready yet to to ship a real app and make it complete and make it good without somebody who really understands what's going on but for these one -shot things it's already happening yeah it's a great question

What Will IDEs and Workflow Look Like?

i don't i don't know what they're going to look like so part of it is that you know maybe don't think of it as what the IDE looks like think of it as what is my version control and my issue tracking look like right because it's you know it's talking

about this idea of what's in the loop what can you do in the next second or the next 100 milliseconds and a lot of it's not going to be like that so as a programmer you maybe all the programmers are gonna be more like managers who say

say, well, what are the tasks that are going on? These people are working on this. These people are working on that. And when are they going to be done?

Are there going to be issues of hooking them up? Maybe that's what we're going to be doing, managing agents like that.

Keeping the GPUs Busy: Workflows and Culture

I have to tell you a story. I was just reminded, I was talking with my colleague, Ben Gomes, who from like 2000, 2006, we were running Google Search together.

and I said remember that time we were in the office on a Saturday and Sergei comes in and says how come there aren't a lot of people here you know we should be working harder and I was saying oh you know life work balance Sergei

wasn't buying that at all and then Ben gave his approach she said oh well look at this dashboard of jobs that are running in the data center and Sergei looks at that says oh okay now I'm happy there's a lot of jobs running so people are at home working that's okay that they're not in and he was satisfied and

he left and then I turned to Ben and I said you didn't have him concentrate on the column that says when the jobs were started and most of them were started on Friday and he said well you know it was there he could have looked at it if he didn't want to look at it that's his fault not ours so so hopefully some of

these tools will be good for that work -life balance thing as well that you can say you know I got a lot of things to do but I'm gonna start all my all my agents doing stuff and then I'm gonna go home Friday afternoon and then I'm gonna I'm going to check in on Monday and see if my agents are done.

Can I follow up on that question, actually? Yeah. I was talking with one of the lead researchers at Salesforce who's since left to start you.

Yeah, Richard Soker. Yeah, and he was mentioning how a mantra there is keep the Jeep useful. Uh -huh. And your story about the logs at Google kind of reminded me of that.

To me, it's an enormous paradigm shift for most people because most people want to clock out and be done, whereas the story I was hearing from you is the whole goal is to get to the end of the day and have enough of a meaningful task that it keeps the GPUs full overnight, right? And then Friday is really tough because we've got to have enough of me that we keep the GPUs full for the weekend.

That speaks to the availability of intelligence that most people don't even think they tap into. For a normie who's not a developer, who doesn't think in that way, do you have any thoughts about how to shift mindset towards tapping into the intelligence that's available?

With a mantra like keep the GPUs. I mean, obviously, that wouldn't work for me. Yeah.

Well, I mean, a lot depends on how you're set up, right? So at u .com, they have some of their own machines, and so keeping them busy is important. For a lot of people, they're renting machines, and so whether you use them or not doesn't make that much difference. So how you're set up like that is important, how you're set up with your data flow,

you know how much data is coming in and when is it coming in and I remember talking you know it's like a decade ago now and talking to colleagues at Carnegie Mellon and they had this interesting idea of the continual learner that would be you know scouring the internet learning new facts putting it into a knowledge base going back and continually learning and they said to me Do you guys do that at Google?"

And I said, nah, because we can't think of ideas fast enough. Our crawlers and indexers are so fast that they finish doing the web before we thought of a new idea to try. So it's not worth it to us to worry about that kind of thing.

But maybe in the future, that kind of thing becomes more important. Yeah. Yeah.

Security, Kernel/User Modes, and Prompt Injection

In less than, what is it, 60, 70 years, we've gone through the initial computing system, which were big, monolithic, very tightly controlled. And they had single track minds.

It was only one job, single process running. To multi -user systems, to distributed systems, where you are less in control of the entire system.

And you can have malicious actors interacting with it, trying to, we had code, prompt injection, SQL injection, that type of thing.

And one of the most basic things we came up with to address that issue is, you know, security -oriented operating systems with kernel mode and user mode. We have that in our chips, have kernel mode and user mode, ring zero, ring one, et cetera. Our operating systems have that.

Once we switched to AI, you know, once we got to the training, the language models, we lost that concept. that we don't have a kernel mode and a user mode, which is why it's so simple to have the fact, like,

forget all previous prompts. Now do xyz to subvert my system and work it to my will. And I haven't seen anything.

I think it's one of the core issues with security in AI is the fact that we don't have such a concept of full -on control or user mode versus kernel mode. Do you have any thoughts on that, on securing models?

Yeah, I think that's an interesting topic. First of all, I wanted to observe that the first recorded prompt injection attack was Obi -Wan Kenobi saying, these aren't the droids you're looking for. Did that work for him?

right so there there are ways of doing that right and so a lot of people now are saying you know you have a system prompt and it says do the following things blah blah blah and then it says oh and by the way you're going to get something from the user and that's going to be surrounded by HTML that says begin

user and end user and when they do things here's things they should be allowed to do and here's things they're not allowed to do and it's still kind of flat and it's like you give these instructions and yeah yeah so I think it's hard right it's a so I guess you know the way I think of the difference

between AI and regular software engineering is regular software is about defeating uncertainty by squishing it into nothingness and AI is about accepting uncertainty and learning to live with it and in both approaches you know it doesn't really match the world perfectly and there are flaws and so So there are attacks that get through in either kind of approach.

I mean, I guess if you want to solve the kinds of problems that AI systems solve, I don't think it's going to work to try to completely regiment everything. So I think we're on the right track, but we need to do a better job of just these basics of

shouldn't be able to delete somebody's disk and so on, right? And I think we can do that.

English as a Programming Interface—and Its Limits

I think that's right right and you know another quote I didn't show from Andre Karpathy is English is the new programming language and so I guess what I what the way I want to think of it is we should think of it as communication between our intents and what the machine can do and we should use whatever

notation is right for that communication and you know sometimes that's going to be a programming language sometimes it's going to be English instructions communicate your intent and sometimes that'll be English or natural language

sometimes it's going to be other notations we have right so we've met a lots of really powerful notations and like chemical equations and musical staffs and mathematics and so on they all have their own set of equations and

it's always better to use them than not when they're appropriate and so I want a system where you can use whatever makes sense whatever is the best way to and sometimes it'll be more formal and sometimes it'll be a mix yeah right so I

I mean, it's still easy to make a mistake when you specify what you want. And an engineer has more experience with making those mistakes and how to avoid them. And whether they couch them in code or in mathematics or in natural language, you want somebody who's made those mistakes before, understands where the dangers are,

and understands how to avoid them. And I've got to say, as a species, we spend a lot of time on this, right? We have a lot of legends and myths on this idea that it's really hard to say what you want, right?

So we have the genie with the three wishes, right? You say what you want, and the genie always gives you technically what you want, but it wasn't what you really wanted.

And we have King Midas, and he was given the gift of whatever he wanted, And he said, I want everything I touch to turn to gold. And then he realized he had a bug in his specification when he touched his daughter, and she turned to gold.

And so we have ways of getting around that. So King Midas said, I want everything I touch to turn to gold, but I want an undo button, and I want a pop -up that says, are you sure? So we have tools.

It's hard to say what it is you really want, but we have tools to help, and we should develop those tools more.

You did exactly what you told it to, not what you wanted it to. It's becoming harder and harder to be able to say that. Yeah, yeah.

Debugging Heuristics vs. AI Systems

Most of the time we have these algorithms, which are heuristic algorithms, or rule -based algorithms, right? But now moving towards the AI, we are seeing the AI -based algorithms. So the way I see this is like with the heuristic algorithms, we can actually predict the paper, how to debug this. But with the AI analytics, how do you see this happening?

Yeah, so we're in this funny state where we have systems that make lots of mistakes, right? And they criticize for hallucinating and so on. And I think to figure out what's going on, you have to break that down into two parts. So one is, did we make a mistake because of the technology we used? Is it that neural nets are sort of inherently probabilistic and they can make mistakes? Or do we make a mistake because of the problem we're trying to solve? And I think it's both.

And I think, you know, we built a system of software engineering over 50 or 70 years or so on That was really good at getting us to bug -free programs eventually for well -defined problems, right? So it's hard to write the software for a bank. Why is it hard? Because there's a million different rules and regulations and fees and taxes and transfers and so on. But there is a right answer down to the penny. And all you've got to do is keep hacking at it until you've eliminated all those bugs and do all the tests and so on.

and now you've got a working system with AI systems we're asking them to do things for which there is no specification right so it's you go to nano banana and say make me a picture of something and I want that picture to be really cool and what does really cool mean we don't have a specification of that and sometimes you say yeah that was good another time you say oh no that was it's terrible. Try again.

And that difficulty, those bugs and those responses aren't because it's a neural net. It's because it's a problem that's unspecified. And so we should look at where our difficulties come from.

Some of them come from having inherently probabilistic systems systems that maybe don't have the best match to what's going on in the world, but sometimes they do much better than what we could do otherwise, right?

So, you know, I started in this field in the 1980s, and I was trying to get computers to understand English, and I was kind of hand -coding everything, and I could be sure that I had the right definition of one word, but, you know, It took me a week to do one word, and then I wanted to do the next word. And I said, this isn't scaling. I'm never going to get to the end of this. And as I add more and more words, the interactions start to be bad, and you get to the point where when you add one more, it makes more errors than it fixes. So I knew that that was a dead end.

And so instead we said, now we're going to let the machines make up their own representations. patients. And that seems to be a better choice overall, but it does mean that it makes some mistakes.

Scaling Laws, Representation, and Multimodality

And I think, you know, everyone's been a little surprised at the success of scaling laws, that we keep giving them more data, more compute power, more parameters, and we keep doing better. and I guess to me what that says is something amazing about the real world or our representation of the world in mostly in writing right so when I was a graduate student in the 80s I said okay there's some stuff on the written page and it's my job to understand what's going on there and you know there's some definitions of words and there's some linguistics what's a noun phrase and verb phrase, but I said, we can get that.

But then I said, now there's a lot of stuff going on up here in my head, and then I have much less of an idea of what to put in there. And we didn't really know how to attack that. But over the last five years, what we've seen is saying, well, don't worry about that part of what you put in the head. Put in transformer, very broad priors that could accept almost anything and now if you put enough terabytes of text through it it's all gonna work I think nobody really expected that we thought we'd have to understand more of what's going on in our thinking head and I think that's really a triumph for whoever it was who invented the written

word right that I thought half of what we need is on the written page and half of it is in the head but it turns out the written word captures 90 % of what we need uh and that's an amazing discovery and i guess i'm optimistic that we can go farther as

we add more multimedia right so we started to do that with with text uh recognition and so on i think there'll be another breakthrough when we can really handle video right so now we have you You can make short video clips, but, like, we don't have the GPUs or TPUs to train a system on every minute of YouTube video. That's just too expensive. We're going to need another breakthrough or two before we can do that. But if we could, that's a whole other representation of the world. That's kind of orthogonal, too, but it can also be aligned to the representation that we got from the written world. world.

So I think we're going to do a lot better as we incorporate these multi -media sources and as the power of the computing platforms go up.

Bias, Trust, and Choosing Sources

It feels like there's so much more opportunity based on what I get out of systems of bias. And I don't mean just racial and ethnic, all those social, important social issues. I just also just mean directionally it could take something into a direction that I know is based on my experience of the industry or whatever. It's just wrong.

law.

And so, you know, as I, from a manager or executive perspective, how are you thinking about helping organizations make sure that this bias, because we don't know what the input is anymore, right? We don't know all the inputs. We don't know how it's thinking that we identify this bias so that we don't invest a billion dollars into the wrong thing, for example. Does that make sense?

Yeah. Yeah.

I think I, and so, you know, Jeremy talked about Notebook LM as a platform and sort of the idea there was you decide what you trust and then you ask questions of that. And I think that's a good approach.

you also would want the system to be able to say well what should I trust that I don't know about yet and add to that I think we it still is a good idea to train systems on everything and just so that they're good conversational partners right they should know how to speak they should know any concept you're talking about they should know what you're talking about but then there should be another level of yes there's general conversation but now we're we're trying to solve this specific technical problem or whatever, and we're going to rely on things that are actual facts and not just everything out there. And so I think you want kind of a two -level system that can do both of those things.

Here, what do you think, maybe two more questions and then we wrap? Is that okay? Okay. All right. All right, burning questions. You can... Yeah, go ahead. You choose them. Yeah. Yeah.

Measuring Productivity with AI Tools

So we have been trying any recording, but then when we measure productivity metrics, they can be a little bit around that, but like how much code is being blocked, check -ins and so on, everything is happening.

You don't really see real improvement about movement in the metric that is separable from the noise, except if we try and make those metrics our goal, of course, because they can be very gamified. Yeah.

What are your thoughts on this, like on one hand, yes, it's like when I look at something and if I'm doing the POC, the last time I worked with some U .S. technologies one year ago and now it makes it much easier for me to do something. But on the other hand, the end outcome is not, it doesn't seem to be shifting in some ways. Yeah.

So, one is, I think, we have a different technology, you probably want to be tracking different metrics. You know, I don't agree with everything Bill Gates says, but he said something I really like, which he said, measuring software progress by lines of code written is like measuring aircraft engineering by weight added, right? right? So that's probably the wrong metric.

And there's lots of ways you can make progress, and counting lines of code, probably not the best one. I remember several arguments I had to have

in the annual performance review, where all the managers get together, and we said, well, well, there's this one person who's just not generating enough lines of code, and, you know, should we put them on performance review? And several times I had to say, wait, I happen to know that he had a one -hour conversation

with this team and told them about this area of mathematics that they didn't know, and that made their product successful, right? So I think if he writes zero lines of code for the rest of the year, we're still way ahead but that wasn't measured because it wasn't even on their

team he's just they knew he was he knew a lot of math they went to him they had a discussion it got done but it's something that falls between the cracks of the way we measure progress and I think that's a problem and when we you know the software systems we have have been so regimented that the metrics we have are actually pretty good because we have like this assembly line machine. But now we're doing things a different way and those metrics aren't good anymore. So we're going to have to come up with better metrics.

There was a report from Stanford HAI that measured like 50 different cases of productivity and it was mostly pretty good, right? So it's mostly productivity went up 10, 20 % or whatever. I thought it was very interesting that they said it helped the poorer people performers more than the experienced performers, but maybe the experienced ones were already saturated. And then there was a couple of negative results of zero impact or even negative impact.

And I think what was going on there is you think of that more as an investment, right? So people said, oh, here's this new tool. This is really cool. I'm going to play with this tool and try to understand it and I didn't get any productivity done because I was learning this new thing so maybe you have to look a little bit longer ahead rather than say what were the metrics for this week

some German interviews they asked me how much money the future you created bring bring to your company, it was universal method. Yeah. Maybe let's, your last question here.

What Level Is AI As an Engineer?

So you mentioned like the year 2025 in mind, we had a different level by sub -engineering, junior, senior, so I want to understand your opinion, AI code in your agency at what level, do you see it right now?

Yeah, I don't, I don't know, right? So I've played with it, and it does good things, right?

So I just finished doing this advent of code thing, and it's better than a junior engineer, right? So it knew all the things it had to know.

It knew, you know, asymptotic complexity. It knew some fancy data structures of union find data structures and computational geometry and so on.

So it was well above a college graduate or a first couple years of programmer. I still like my code better, but it's getting there.

Conclusion

All right, folks, let's give it up for Peter Norman.