Hi everyone. My name is Vitalik Leban.
I'm doing machine learning before it was actually called machine learning.
So I've started at doing like,
let's say, digital signal processing at
a very, very small computers called microcontrollers.
So how many of you are actually developers here?
Okay, quite some.
So how many of you are actually doing
a machine learning as an everyday job?
Okay. So today,
I'm going to talk about this bit's billions and cents,
and actually on how it's possible to basically
build an interface between
the artificial intelligence or
like large language models in the real world.
So unfortunately,
there is a lot of infrastructure around us that it has been
built purposely to make human populations
sustain and this infrastructure is tremendous.
So we have energy grids,
we have water grids,
gas pipelines, sewage systems, and so on.
All of this infrastructure,
it's impossible to rebuild it.
So the only way for us to
automate anything in this infrastructure is to actually retrofit it.
So to retrofit it,
we are using a small things called sensors that we deploy everywhere.
There are several problems with the sensors,
and with the recent developments,
you could actually see that the machine learning
is actually coming to a very,
very small devices and now it's possible to do
machine learning on the sensor itself.
And the market is going to grow quite significantly.
So you could see that it's projected that we're going to have
almost 2 billion smart sensors or sensors with AI by 2026.
And I know that many of you do not notice these things,
but it's actually happening.
So probably the best example is just the airports.
So it has a small CPU that is capable to do
some ML task to cancel out the noise.
So some of the key players,
because we all know the key players for the LLMs,
these are OpenAI, Anthropic, and all the others.
But also there are these guys that are doing hardware,
and they are not small.
So these are companies like Qualcomm,
Silicon Labs, NXP, Schneider Electric, and so on.
And on the picture,
you could see a chip which is roughly 1 by 2 millimeters.
So it's a very small thing,
but it's a hardware implementation of the fully connected
neural network.
And it's already in hardware.
You could buy it and you could embed it into your products.
And it has almost 600,000 parameters,
and it draws only 140 microwatts of power
in the constant on operations.
So why this always on operation?
Why it's important?
Well, so one of the most interesting use cases for it
is this, you know, like, assistance, like voice assistance,
that they are constantly listening to the microphone
to the trigger word, and then they respond.
So the power consumption is really, really important.
I'm going to show you why.
So although it is kind of theoretically well known
on how to build this sensor and sensors
and how to bring machine learning to the sensors,
there are still a lot of open questions.
And so, for example, if you put all of the intelligence
on the sensor, how do you actually collect more data
for the learning in the future?
How do you run AV tests on the sensors?
How do you do all of this, like shadow and camera deployments?
And basically, how do you do ML ops on the sensors?
And it's an open question, and there is no answer.
And the main challenge here is that most of the patterns
that we have for, like, big ML, they
are not applicable in this very constrained environment.
And everything that is happening with the sensors
could be basically, you could see it
as the following set of processes.
You collect the data from the sensor,
and then you transmit it.
You collect the data.
You compress the data, and you transmit it,
which is a very well known process,
like with MP3 files and so on.
And you collect the data.
You process the data, and you transmit.
And we are here.
So in this talk, I'm not going to talk
about the collection of the data,
and I'm not actually going to be talking a lot about either
compression or processing of the data.
I'm going to be talking about this transmission function,
which is a very interesting thing for me.
So this machine learning-related compute
has been doubling, well, every six months
for the last 10 years, according to Microsoft
and all of the other big players.
And the main reason for most of the project,
for the failure of most of the ML or AI project
is actually lack of data, not lack of algorithms
or lack of smart people or lack of solutions.
So there is no data.
And this problem is even bigger within this offline businesses
that have to do with the infrastructure
of this offline world and the physical world and so on.
It's basically all of the logistics, smart cities,
and on the picture, you could see the New York 5-Mays in 1891.
I believe it's exactly the same 5-Mays now, or even worse.
So how do we monitor all of that stuff?
And what do we do?
So if we go on the slide back, we probably
know how to collect the data so we can install the sensor.
We probably know how to process the data.
We have all of these nice algorithms on the sensor.
But the issue is that the computational efficiency
of CPUs and microcontrollers and sensors,
it was exponentially increasing over the past years.
But the wireless transmission or basically the transmission
of data from the sensor to the cloud or to the internet,
it remains first energy intensive and actually expensive.
So and this is the problem that I'm
trying to solve for the last, let's say, five maybe years.
To solve this problem, there is this protocol called Lotawan.
How many of you have heard anything about it?
OK, so Lotawan is basically the following thing.
So your cell phone or in a 4G mode
could probably throw put 300 megabits per second.
And it's a tremendous amount of data.
But there are no sensors in the world
that probably require you to have such a throughput.
So what you do is you probably need like 300 bits per second
for the sensor if you have a water meter or a gas meter
or something like this.
So what has been done is using the modern radio technology,
these 300 bits per second were converted to 300 megabits
per second were converted to 300 bits per second
in the exchange of the power consumption.
So basically, the device that is using the Lotawan protocol
is consuming 10,000 times less power than any LTE-based device,
which makes it possible to build a sensor that is always on,
has an ML inside, and connected to internet,
and runs from the battery for, let's say, 10 years.
And this is a game changer because there are so many use cases
that could be implemented with this type of sensor.
There are a lot of companies that are doing this.
So the protocol is supported by Amazon, AWS,
by bigger companies like Cisco, SD, and so on.
So the ecosystem is really kind of big.
The use cases are also very different.
So people are doing everything from vaccine monitoring,
they do electricity metering, gas metering,
all kinds of infrastructure monitoring, logistics,
and so on and so on.
So unfortunately, that's not enough.
Well, we have the technology, but to bring it to life,
we need to build the networks, similar to the networks that
exist for LTE or any other cellular networks.
And this is what I'm doing.
So together with my company, we built these networks.
So here is an example of the network in the United States.
And altogether, every net that's the company I work for,
it has access to more than 100,000 towers.
So what we do is we put our equipment on these towers
and enable this low-cost connectivity at scale.
So the companies that are doing AI could interface the sensors
and bring the data from the real world
to their machine learning applications.
These are another example.
So this is our network in Brazil.
And this one is in Indonesia.
I especially like both of these examples,
because Indonesia is, I believe, maybe like 18,000 islands.
And so they have logistical challenges,
and they need to monitor a lot of stuff that is going
from the island to island.
And as you may understand, having all of this infrastructure
in a field is somehow challenging.
And I'm going to show you a couple of examples down there.
So a part of covering the countries itself,
we also cover the busiest ports in the world
to help the logistics companies to track down not only containers,
but also packages within containers and pallets
and smaller items and so on and so on.
So every net has coverage in all of these places,
in Port of Johor, in Jakarta, and so on.
And this is an example of the same coverage in Europe,
because, as you know, everything that is manufactured in Asia
is going to end up in Europe.
Or let's say, for example, the pulp that is used for the paper industry.
It comes from Brazil and may end up, for example, in Finland.
I think Finland's one of the biggest paper manufacturer.
So machine learning and AI, the big ML and BI,
are already helping us to do different kind of things.
First of all, we generate all of the coverage maps
with AI using clutter data and different kind of maps.
And we also do predictive maintenance for the equipment
that we put on the towers using big AI algorithms.
But we are, of course, looking forward to implement something
with the sensors as well.
So I don't know why, but I like this picture.
So this is an example of economy of scale.
So I think this is going to be, this is the end game
for all of the sensors and for all of the hardware or ML enabled hardware.
So these are rice cookers.
And each of them, just take a look.
So it's a button and a lamp or LED.
It's a button and LED button and LED button and LED button
and LED, all of them are the same.
So it's basically the same thing.
And it contains the same electronic components inside.
It contains the same heater element inside.
It is just the same thing.
It's just have the different forms.
So you could take care of different consumer groups.
And right now in the sensor industry,
we have this thing called system on chip.
And I think we need to have the systems on the chip.
So and once we could go down in price with the sensors and connectivity
and batteries and everything else,
I think we're going to see a tremendous growth in this.
And also the economy of scale reduces
the accidental complexity of this project.
And I believe that maybe in several years,
we're going to end up in a situation where basically everything
that could be connected to internet
is going to be connected to internet.
This is street lights.
And by the way, we are very, very happy
because Helsinki, where we are now,
is I think the biggest in the world's museum of failed sensor
technologies.
So you could go around and look everywhere
and you see abandoned antennas, abandoned sensors,
abandoned hardware solutions and so on.
Because Finland is very advanced in it.
So it has been trying to adopt different type of stuff constantly.
So I think that's it for my talk.
And I'm happy to discuss anything that is related to tiny ML
or anything that is related to machine learning in general.
Thank you guys.
This is a lot of information about the field
but what about latency?
Yeah, so let's try to think about it.
So the latency between a sensor and the base station,
it is equal to the delay introduced
by the speed of light, basically.
So there is no latency between these two points.
So there is a latency between the base station and the cloud.
And that's introduced the most significant amount of latency.
So going from 300 megabits per second to 300 bits per second
actually does not introduce a lot of latency in the sensor
terms.
That might not be suitable for maybe gaming
or something that really requires a very precise time
but full sensors is more than enough.