Introduction

I'm the CTO of Logidot.

About Logidot and Our Focus

We're basically a company that works with IoT and in the field of logistics.

We do a few things which are mainly around providing analytics around real-time positions of assets in warehouses and insights around how to improve layouts.

Why Open WebUI for Virtual Assistants

The AI craze is here where there's a lot happening with building AI agents and virtual assistants and we're actually working on a project related to virtual assistants right now.

And what I wanted to show to you today is a bit about open web UI.

Who here has heard of open web UI?

So yeah, not a lot of people.

What Is Open WebUI?

So basically, Open Web UI is an open source framework that allows you to deploy.

It's basically your open source chat GPT.

You can create your own virtual assistant chat GPT kind of app that is essentially uniquely tailored to your company.

Why Use It Over Hosted Assistants

I mean, why would you use it compared to, for example, something like Claude's desktop?

It's because you could actually deploy this to lots of people.

So it's a really interesting tool that you can use for fast prototyping.

If you've got an idea that in your company there is a possibility that the virtual assistant will be helpful, it does not take a lot of work to get something like this

running, at least the basics of it.

A Quick, Nerdy Demo Setup

This is a very nerdy talk, so I'll show you.

So basically, I put this together just for this talk.

I just vibe-coded the whole thing.

And there's not a lot there.

Docker Compose and Minimal Config

It's basically all in this Docker Compose.

And then I added a bit of other things just to be able to showcase some features.

Customizing vs. Using Out-of-the-Box

But essentially, you have the option

if you're technical to actually get into Open Web UI and tailor it a bit more.

But it's such a customizable app by itself that you don't actually need it.

Interface and Models

So what does it look like?

It looks like ChatGPT, but you get a lot on top as well.

that might be useful for your company.

Tailored Assistants per Role

1And I think one of the most interesting things is you get this possibility in your workspace of creating models which can be tailored to a specific use case.

And this is just some random stuff I came up with.

But in the context of a company, you might have, for example, an assistant that's tailored for specific roles within the company, like something for managers, something for operatives, salespeople.

I don't know.

It's going to depend on case by case.

The point is, it's really simple to do.

Base Models, System Prompts, and Tools

You can basically create the model, you choose a base model like GPT-5 or Claude or something, and then you have the possibility of adding a system prompt and tell it to behave in a certain way, and most interestingly then, you get an option to choose tools in a really easy way.

Leveraging the Community Tool Marketplace

And I think this is one of the advantages of this thing, Open Web UI, is the tools, because then they've got a community.

And in the community, you can actually search for tools that might be interesting for your use case.

So you don't need to code it.

It's already there.

And getting it in the actual Open Web UI is very simple.

So you basically just do that.

Import to Open Web UI.

And...

Yeah, normally you have to agree with something, and it's done, and it's there, Wikidata.

Configuring Tool Credentials (Valves)

You do have this thing called valves for tools, which normally is where if you need to put in your API key

user agent or something that's where you'll put it but you know in a lot of cases you don't really need that so we could date to just get stuff from wiki Wikipedia so if you come up with a chance I'm just going to choose like this is basically choose like well we can we can we can basically choose any model I think this so

Example: Adding Wikidata to a Model

because I'm choosing here cheeky chef, which is just like your assistant that comes up with suggestions for foods for you to eat.

So it wouldn't appear there, but if you actually go there to the model, you then have the option

to add wiki data.

There you go.

And then if you save that, if you go there, now you have the option.

Now you have that tool available.

So if you do a query, then you're able to actually search the wiki, if it made sense to search the wiki.

I'm not sure this will work.

But yeah, I'd have to think of a query that involves searching the wiki.

I actually didn't plan this.

Yeah.

Tell me more about pasta.

Maybe that'll prompt it, but it might not work.

Knowledge Bases and Retrieval

Regardless, while it thinks, one of the other things that I wanted to tell you that's very interesting for this is then you've got this tab called knowledge.

Creating Collections for Teams

And knowledge then enables you to create your collections for different types of knowledge that might be relevant.

And in the context of, for example, running a pilot with a company, this is the place where if there's some documentation which is useful for certain types, certain managers, you would put it there.

If it's documentation that's useful for your salespeople, you'll put it there.

But you have the option of creating different types of knowledge bases.

And then... Something like that.

And then...

Linking Knowledge to Specific Assistants

The advantage then would be that when you actually look into the model you then have the option of selecting what knowledge base you want to link to the model.

So you have the option for example of creating an assistant which is dedicated to salespeople and just has the information that salespeople need to know.

And then you create an assistant for managers which has just information managers need to know.

Let's see what's... So, great question.

Let's see if it's... Did it actually go to Wikipedia?

Yeah, there you go.

That's the case of a live demo working.

Model Backends and Connectivity

So you can actually do pretty much anything you want.

Local and Hosted Models

So here I've got some local models.

So I've got Lama and Quen here.

And then I've got the GPTs.

So I've got an open, and I'll show you how that's done actually.

That's an interesting thing to do.

Admin Settings and Provider Links

So basically in admin settings, you have the options of linking.

I'll get to that in a bit.

Of linking, for example, your...

OpenAI API or your LLAMA.

So if it's run locally, you can just link your LLAMA that's running locally and then download whatever models you want to download and just run them.

It does take a while in this machine.

The first thing I wanted to do was to actually get this running in 11 months, then turn on the Wi-Fi after.

But that would actually take me like three minutes.

But if you have a decent, a beefy GPU, then that would be good.

Hardware and Performance Considerations

I was going to ask that question.

So the limitation to speed on running locally is just as GPUs.

Q&A: Capabilities, Knowledge, and Cost

Where Does Knowledge Come From Locally?

When you're running something like this locally, how can it have a breadth of knowledge?

Is it purely down to the knowledge base that it's given, or the access to the internet?

Essentially, if you don't give it a knowledge base, you don't give it access to the internet, it has no knowledge?

It would still have the, well, it would still have the knowledge of the actual model that you're using, whatever model that is.

So you're saying no access to internet, so we're assuming it's not like GPT, it's not open AI or cloud or anything like that, so it would be a QEN or a LAMA, so it's, they're normally not...

You know, they're not as beefy as the models that you run off APIs.

And if you're running locally, the more likely you are not able to even run some of the more beefy versions of those open source models.

You're probably running like 7 billion, which is what I'm running here.

They're just a bit worse models overall, but you get to run them locally.

But you can still give them access to your local knowledge base.

But that would be it.

Extending with MCP and Local Data

I mean, what you could do then is you can link your MCP server, for example.

And within MCP, you could, for example, give it access to something that you have in your laptop.

repository with a lot of data.

You don't need to upload it directly in Open Web UI or come up with some way of using directly the knowledge base for Open Web UI.

You can just resort to the tool usage.

Yeah.

Cost and Deployment Scope

In terms of cost, do you have a limit of the usage of which other community workers

Well, so this whole thing is open source.

So you can run it.

The open web UI itself is open source.

The models, it's model agnostic.

So it's whatever you're doing for it depends on the models that you're running, the underlying models.

So it can be anything depending on what you choose.

But in theory, if you're running something locally, it's nothing.

Yeah.

Yeah, exactly.

Well, you can deploy this.

to whoever you want because this is a web app.

Pilots, Iteration, and Fast Feedback

I think the main use case for the open web UI is when you want to run a pilot in your company and you don't want to spend a lot of money and you want to iterate quickly and try to understand for some reason that's not charging.

and you want to understand what works best and doesn't like before this just dies on me we're almost done on time anyway I'll just show you something else which is

Evaluation and Feedback Loops

So another thing that she have with open web UI is you have the ability to do evaluations and get feedback from whatever happens in your chats.

So again, just going,

Yeah, it's charging now.

Data Access via the Backend

Going directly to the database, you basically get a lot of backend for this, right?

And you get pretty much anything that happened in terms of feedback, in terms of chats, in terms of knowledge, you know, just in your Postgres.

So if you want to do anything directly with that data, you can do that.

A/B Testing and Model Comparisons

but you actually have a way to expose users to whatever you've done, whatever you've come up with, the models that you've come up with, and you have a way of sorting their feedback and actually racing models against each other as well.

That makes sense because it's based on an open source concept, right?

So the first model is going to win.

I'm assuming if contributors keep contributing to the model eventually it's going to, I guess, become more performant.

Yeah.

But remember that this, like the, the goal of this probably isn't going to be to see whether, you know, uh, an open AI model is better than Claude or something like that.

It's going to be checking, uh, you know, racing maybe some of these bespoke models, which you actually build here.

And, um,

And you have basically a base model for them.

But then you have a lot of system prompting, different tool usage, and different, you know, you're able to, there's a lot of technical parameter filling that you can do with this as well, like temperatures and so on.

You can really do a lot with these.

1So you can, what you can do is use this for A-B testing.

So you can have like three or four different types of models

that are very similar to each other, but you deploy them to 50 users, and then you check on their feedback and see how they're performing in the actual field.

And you don't have to actually go out of your way to kind of code this, because it's already there.

Complex Systems and Limitations

What's the most complicated system that you've seen someone set up with this?

What Can You Build with Tools and MCP?

um good question like you can do it's i think that's ends up being limited by the tools because you're able to just link to mcp you can pretty much do anything that you would do with potentially cloud

So you can come up with some clever stuff that involves pretty much any other system.

Obviously, you're still limited that you're communicating via MCP, but you can

You can, for example, use this system to create a background task which monitors the process that's streaming and then have it send, for example, an alert via HTTP in your dashboard or whatever.

So it's kind of limited by the creativity of what you can do.

Built-in Pipelines vs. Custom Stacks

In terms of limitations, I would say it's the fact that this has a rack built in, this has pipelines for actually embedding your knowledge built in, but then you might want to not rely on them and you might want to customize them.

And you can still do that.

You have to work with the API, which you can work independently of the front-ends and maybe come up with some separate systems to actually do those things.

So it gets to a point if you want something that's extremely custom that this might end up being a limitation.

But the point of it is to be able to prototype quicker in the beginning until you get to the point where you actually understand what you need to build.

And then you can go on and just build it.

Customization, Languages, and Frontend Options

I think you may have already answered this question, because I was going to ask how much do you customize the code itself?

I feel like this is a stupid question, but is it language-adapted?

Language?

Yeah, yeah.

You're mentioning, for example, these tools, for example, here.

Yeah.

That's a good question.

Tool Languages and Ecosystem

I do think it supports a few different languages.

But I'm mainly seeing this written in Python.

Probably you can do a few others.

I've seen some JavaScripts.

I actually don't know if you can do any.

But I think this is running a bit as a script, as a one-off script.

It will depend on what, yeah, it depends on the setup, on the internal setup of Open Web UI.

I'm sure you can't run, like, everything, but, yeah.

I think it's probably a way to create a interface between, because you also mentioned probably, like, add on a front-end, like, custom, I don't know, like, maybe a link in, I don't know, a web interface for...

I don't know what the use case might be.

Again, you mentioned you'd probably like to plug into interact with some baseline code, I think.

Are you thinking front end or back end?

Frontend: Out-of-the-Box vs Custom UI

In terms of front end, this is pretty much what you get.

You do have some options for customizing what the user sees, but it is what you get.

But then because you can use the API directly, you can build your own front end if you want, if you get to that point.

And normally that's what ends up happening.

You kind of start with the baseline and then you start replacing components because you need something that's more bespoke and more bespoke and more bespoke until you end up just using, you know, the open web UI bit ends up just being like a fraction of the stack that you've got deployed.

Yeah.

Advanced Prompting and RAG Customization

For example, if you supervise a prompt, that prompt is going to be changed into more of a developer prompt or more of a prompt that I guess will be more AI friendly.

I really customize the code specifically for that.

And I was hoping you could do something along those lines of how much I probably didn't customize the code.

For example, do something like that.

Take the entire prompt and then reduce the shots.

Check to add a few name in that.

The question is basically, yeah, system messages you've seen, but I think your broader question is like how much more complicated can you make this rag?

Is that it?

Like how much more complex can you make it?

And my...

I guess my answer is you either use the parameters that they have.

That's option one.

Forking vs. Parameters: How Deep to Customize

Option two is you fork the whole thing if you're really into customizing code.

You fork the whole thing and you just change it to whatever you want.

It's quite modular, so it's not too complex to do that, but that's obviously, you know, it's got its own problems then.

because you kind of have to maintain that as well, but you always have that option.

If your whole idea is, I need something that looks very much like this, but I just need to change the rag to something that's unique, you can do that.

Okay.

Okay.

As a UI for the user.

You can do it.

That's the thing.

UI Customization Limits in the Stock Image

How much can you customize it?

You end up not having a ton of options for customizing it.

When you fork it, when you actually fork the project, you can customize a lot.

But just like what I did here, and just run it off Docker Compose and just use the image, you don't have a massive amount of options for customizing it.

From Prototype to Production

But likely what ends up happening with this is, because this has a lot of bells and whistles, because the whole goal is that you can play around with everything.

But then in the end, what you realize after your pilot is I actually need only like 5% of these features, but I actually need them to look in a different way.

So you go and build whatever you're going to build.

But in the meantime, you are very quick to actually figure out what you need or don't.