Introduction

Hello everyone, how's everybody doing tonight? Yeah, so I just want to comment on the last presentation.

Automation Philosophy and the Role of AI

My philosophy around automation, I think humanity in the past 200 years has gone through numerous rounds of this, when we sort of became people who do agriculture and so we don't spend our whole day gathering food, or through the industrial revolution where a lot of those jobs got shifted and became automated by the factories we still found a lot of better

ways to use human intelligence and I think with AI I think for most things there's still a lot of human in the loop but what it does is it frees you to up time to think about how you can use your intelligence for some new task instead of wasting time I guess now is what we call it and doing tasks the I can do so

Speaker Background and Startup Team

So just a little bit about myself, I'm a PhD student, just wrapped up my PhD at University of Toronto, Computer Science Department.

Why there's three people here on the slide is we have a startup and we're a spin out of the lab. And so this is my latest venture.

AI Infrastructure Experience

Myself, I've been working in the space of AI infrastructure, optimizing applications that get deployed across cloud and edge infrastructure. I worked with natural language processing and computer vision applications.

Defense-Funded Research and Applications

Our work was funded by the Department of National Defense. So we were building resilient communication systems for the military, not necessarily for battlefield things, but just generally you can think about like healthcare on the battlefield, things like that, and operational support.

I also co -founded a couple of companies and scale those up.

And so these other two folks are my amazing co -founders, an undergrad who just finished as well. he's XAWS, he's got a large open source following, a lot of products, and my other

co -founder, he's the chair of Computer Science, University of Toronto, so he's one of Canada's top computer scientists, he's one of the early pioneers of the edge computing field itself, so having invented foundational technologies that

we all use, some of his companies are acquired by Google, his notable technology powers the entire memory management system of the Google Cloud, so many of

Talk Focus: Enterprise Generative AI

the apps you run today run on his technology so yeah moving on towards this talk so this talk is

going to be centered around generative ai but more focused on the enterprise and so our startup is focused on solving problems within the enterprise for ai agents being used to do a lot of these

Core Challenge: Fragmented Enterprise Data

automation tasks that we talked about earlier but the fundamental challenge of the ai right now is is that they cannot understand enterprise data because data is in different systems, they have different languages,

and you use humans essentially to work with these systems. So they were not designed for AI agents. And so this is all where the valuable data sits.

And so I think for the past three or four years, since ChatGPT came out, a lot of surface level use cases were being brought into production.

But now the models have gotten really good, and now everybody's really tying it into the depths of their systems, basically. But these systems are very difficult to query because they're fragmented.

So you can think about the average corporation has between 500 and 1 ,000 IT systems, like a very large company. So having humans to stitch things together has been the way we've sort of gone along, but now having AI to do this, it's a challenge.

Adoption Blockers: ROI, Security, and Sensitive Data

So the number one priority in speaking to dozens of large enterprise companies over the past couple of months is return on investment. Of course, people invest a lot of money in AI systems, but security has been the main blocker for things getting adopted.

And what I mean by that is the AI systems get built on very small amounts of data in development. And then when you have questions around how are we going to put our critical supply chain data, our corporate knowledge into the agent to send it to the model, the question becomes that is very sensitive data.

So we have sensitive data that is very valuable. And so how do we leverage public models for that?

A Two-Part Solution for Enterprise AI Agents

And so it's a two -part problem that we're solving.

Part 1: Accurate Reasoning via Institutional Knowledge

and so the first part is accurate reasoning.

I think in the last conversation there was a lot around like okay the models are I guess dumb in a way like they don't capture our institutional knowledge like how our business runs things that only the people know and so what we've built is a system that captures that as you work with the AI it asks you questions and it absorbs that institutional knowledge and starts to compound that into the system.

So this is what sort of the future of systems are starting to look like.

From Prompting to Goals: Agentic Prompt Rewriting and Memory

With respect to prompting, the way the users use these systems now is they don't prompt their way through what they set. A prompt is a goal now, and the system rewrites your prompts behind the scenes.

Even embedded within these systems, everything is agentic. They write their own prompts, they iterate, and they figure out what a memory looks like, what your tribal knowledge of your organization is, and it stores that in a way that it can retrieve it later.

Now, it's not perfect, but this is the beginnings of how the system will start to learn. And all of this is outside of the model. It's not sitting inside the model.

The model, you can think about it now as something like it's a computer processor. It has a bunch of instructions that it can work. And all of these things around it accumulate information that allows it to work with that model.

The model doesn't need to get smarter anymore. It just needs to better leverage these tools. tools.

Guardrails, Governance, and Enterprise Controls

So this platform covers off a lot of the things for large enterprise, of course, like we were talking about the guardrails and the security and governance. So this is part one of the problem that we're solving.

Part 2: Privacy

The second part is privacy.

So the biggest blocker, as I mentioned, for companies is how do we use these really strong frontier models without leaking all of our private information? And so the problem with using these models, user data is not private, and they're being

subsidized like I think there's a report that came out that people are paying $200 for you know the Claude code max that every developer uses but it's really burning like $5 ,000 a month compute so I mean you know we're startups or you know raising money in the VC world

so like all of these initiatives are all being subsidized so the idea here that these prices will stay flatlined it is not the case somebody is paying for that and it's going to come do pretty soon.

Edge AI: Using Powerful Models Without Data Leakage

So an alternative solution to using these public models and the privacy is to use edge AI. So that's why this talk comes to edge AI.

So the work that I did was more around optimizing the deployment. Now we're talking about privacy.

Keeping Models Local (e.g., Factories and Remote Sites)

So privacy means all of this information can be kept local. So in this example here, this is like a manufacturer, maybe they own a lot of factories around the world, you can deploy private models.

So nothing is going to the internet. These things aren't even connected to the internet.

And these models are now capable enough of doing the task with the system I just showed earlier.

On their own, the model itself cannot do the work, but if you put this harness around the model, now you can do a lot of work and you don't need to leak information to the cloud.

So this is the direction the enterprise is moving that is actually getting them off the sidelines to adopt it deep into the the enterprise.

What’s Next: Cheaper Chips and Stronger On-Device Models

So what does the future look like? What's coming in the next couple of years?

Right now, I don't know, I guess how much you follow, like the chip companies, but they're very expensive. There's a lot of other chip companies that are selling, that will be bringing products to the market that allow you to deploy these very sophisticated AI systems very cheaply.

You can imagine the future, you know, everybody has these chips in their phones and their desktops. You already have the ability to run, everybody in this room runs some sort of AI inside your phone.

It's just going to get stronger and stronger to the point where you don't need to send the data outside your phone anymore. And so the models are getting more capable in size. And so that's a very key distinction between whether you can run it locally, because you

might need a lot of power, like GPU power, essentially to run these models, versus having to say, okay, I'm going to use ChatGPT for $20 a month, and that's my only option. So that has changed in the past couple of months.

The models, the really small ones, have gotten very capable. So this is trouble for the large model providers, I think. And we'll see what happens.

Conclusion and Q&A

But yeah, that's sort of like the end of my talk. Not too much philosophy. It's very technical. So yeah, happy to answer any questions around, you know.

A round of applause for Brian. Thank you.