Introduction

We were told this is the technical talk slot, so it's going to get a bit technical, but Tony softened the blue by creating some nice diagrams, so we'll see if that offsets the complexity. Yeah, so a bit of background, so when Tony gets the slides up.

What Fliplet Is (and Why It’s Enterprise-Ready)

So, Fliplet is, as of last Monday, an AI -first software building tool. tool. It's similar in some ways to lovable, replit, base 44, if you've heard of those tools, except it's a bit more extreme on the enterprise readiness.

So, we implement a lot more protections. It runs on vetted and understood infrastructure. You can't tell it, you know, I want to use a different database system and things like this, which means we are compliant, which seems to be a bit of a theme today.

So, we're going to talk about some of the the lessons that we learned recently building and launching an AI -first software builder. So I'm Ian, the co -founder and CEO, and this is Tony, co -founder and head of product.

The Three Core Parts of the Platform

Next slide. And a bit about Fliplet. So I've mentioned that it's an AI -first software builder. It's basically made up of three parts, which may be relevant as we go through the presentation.

So the first part is Fliplet Studio. That's the interface that technical people or non -technical people use to build their software.

Then we have Fliplet Viewer, which is an immediate tool that you can use to view if you want to produce a mobile app, you can immediately run it and use it on mobile.

And then we have an extensive set of REST APIs that are kind of standardised, run behind the scenes, and you can integrate with them if you want.

Pedigree: From Drag-and-Drop to AI-First

So a bit about our pedigree and history. This is relevant.

So as you can see, we started started quite a long time ago. And don't worry, in 2013, we were not an AI software builder. So we started out in 2013, drag and drop, PHP, hardcore days. We actually wrote all the code by hand back then. You guys, some of you might remember that. And it was OK, but we realized it was pretty limited.

And so we rebuilt it, full stack JavaScript. We were very excited at the time. Drag and drop. It supported mobile and web and had had a very simple code editor built in. Because what we realized with drag and drop is that people start with drag and drop, but then they realize, well, this is annoying. I want to get around the drag and drop. How do I get around it? And so we ended up saying, OK, we'll give you a coding editor. Even though these were not coders, they would kind of get under the hood, tinker, maybe change some CSS, tweak a little bit of HTML, et cetera.

And then, as I said, as of Monday, last week we launched version 3, which is our AI -first version. So, now, and it's crazy, we're showing this to customers, and customers are like, okay, so I tell AI what to do, and we're like, yes, and they go, okay, type, and they tell AI what to do, and AI does it, and then as soon as it's finished, they're like, now, where's all of my drag and drop features? And I'm like, no, they don't exist anymore, it's AI first, so that was a bit of a surprise.

Three Lessons from Building an AI-First Software Builder

Okay, so now I'm going to hand over to Tony, who's going to take you through three lessons that we learned from building an AI -first software building tool. Over to you.

Is this working? Can you guys hear me? Cool. I think it's working.

So, yeah, we launched last Monday, but we sort of started building just a little over four months ago. It's even as I'm saying out loud, it's crazy that we sort of built it within four months, but we

did have sort of more than ten years of experience sort of building sort of software builders, builders, but more specifically, we built on top of the previous version, which meant we had all the infrastructure in place. And that's going to be an important context moving forward.

Lesson 1: Tools Beat Prompts

When we started four months ago, we started with really just an idea. We understand how to build software, everyone is pushing out vibe coding tools, and we thought, well, it shouldn't be too difficult.

Let's just ask AI to write some code. And it did that relatively well.

And then we thought, okay, we need to build something called the agentic loop, essentially asking AI to write code, and then we'll give the feedback, and then we'll keep telling us to write the next thing, and then it will keep doing that until it completes the work.

And technically what we were doing was we were doing all of that in a single file, just trying to get one single JavaScript and trying to get that to work. And then so what was happening then is that we then basically kept writing more and more

prompt as what we call the system prompt, and that just kept increasing and increasing and increasing. We were just like, it's not quite doing X, Y, Z correctly.

It's not quite writing sort of a, it's not having sort of consistent styles across the entire app. Okay, let's write about this.

We wanted to build a single page app. It's not quite doing that right. It's just going to become really, really long, thousands of lines.

That became more and more unwieldy because anything that becomes inconsistent, you then then realize that, okay, you've kind of written yourself into a corner.

Anyone who has tried to get AI to edit or review anything, if you ask it to assess or review or what typically happens is that it adds by default. So even if it tries to review anything, it always

adds more content. It never sort of tries to sort of simplify or tries to figure out what's anything that's sort of contradictory. So you can kind of imagine that it just kind of gets worse and worse.

Not only that, it also became very difficult to then pass on the work to someone else to work on. We had a couple of people who sort of had the vision of how it could work, and then suddenly you're like, great, but we have a team of engineers. How do you get people involved?

We then tried deconstructing the prompts, which essentially said this section of the prompt, I'm going to move that to another file, and then you start sort of handing it over to different people and said, you deal with this file, you deal with that file. Again, that didn't quite work because there's no one overseeing it all.

Moving from Giant System Prompts to Tool Calling

The solution was then to essentially implement tool calling. Some of you might already be aware of what tool calling are, but at a pretty simple level, it's telling AI these are the things that you can do.

The way I like to describe it, it's kind of like giving AI arms and legs and eyes in years, so it's able to read information and write information, and you're giving it very specific instructions about how to read and write that information.

It helped us improve adherence to certain rules, it helped us reduce the amount of tokens used because then we don't have to use as much sort of system prompt to try to describe how we want things to work. You set the rules in a very sort of a structured way, and it kind of adheres to that really,

really well. well, and it makes it really easy to configure as well. I'm going to show you a couple of quick ones.

I'm just going to have a quick look just to see how visible it is. Okay. I'm not going to ask you to read everything here.

But on the left, we have one called run preview script. What Flippler was doing was getting AI to write code, and then it it loads it up on the side as a preview.

By giving it this one tool, AI was able to say, I want to know what's on the page. So it writes a little script for itself. We run it, and we return the results back to AI.

And now AI knows what's exactly on the page. So suddenly, again, eyes in years, it's able to kind of see what's actually in the app.

Here's another one, request clarification. clarification, the fact that we gave AI a tool called, if you ever wanted anything clarified, this is it. It's able to figure out when it needs clarification without us heavily prompting

it. You must ask clarification about security. You must ask clarification about navigation. We didn't have to specify that.

It simply knows what it's uncertain about and would ask. And that's all because you gave it the tool.

And that's what we've sort of learned throughout is that once you taught AI, once you've sort of informed AI what the possibilities are, it will figure out what to do with it.

The most extreme version of all of this was that once we had all the tools in place, I had one of my product managers ask me, so how do I interact with it? How do I sort of tweak the system?

I said, why don't you start playing around with the system prompt? And she said, OK, so how do I edit the system prompt?

Do I just edit it and save it? I was like, I think so. And we said, let's run a

the test, we removed the system prompt, which meant all you were left with is just the tools. No system prompt. It was still able to build the app. So this is one of the things that

we then found. We were quite surprised by it as well. Oh, it's actually able to build quite all right.

I was hoping that it would break just to show that, yes, I did edit the right file, but turns out the tools are pretty powerful as they are. So back to the lesson and basically being we focus so much on the prompt in the beginning, but it turns out

the tools have a much higher leverage than the prompts.

Lesson 2: Debugging Needs Measurable Evals

Once we started building, we then started to encounter lots of bugs with the system. It would build navigations in quite an inconsistent way. Again, we were back to either prompting or adjusting the tools, but with all the issues coming in, and the issues aren't always consistent as well, we then started having difficulty to try to figure out, well, how do we fix these issues?

Number one, we're sort of product people, engineers. We try to use our knowledge. We suggest a solution. Could it be because of this?

Could it be because of that? AI would very nicely be like, yes, I think you're right. I think you're a genius. But then that has its limits.

We would then go back and say, well, why don't you figure out what you think the problem is? They would do that. But then when it came back to us,

it's like, which way would you like me to fix it? I have no idea you've started describing things in a way that I don't quite understand. So again it had its limits

But even when it started to fix things we weren't too sure that is it breaking stuff You would have heard of unit tests evals. There's like all those sort of different mechanisms You can put in place as well for AI to run tests, but then

AI is the one who decided what to do in the first place So we would you always write tests to confirm what it's doing is gonna be okay again Again, it has blind spots.

More importantly, though, when it comes to unit tests and evals, they don't really test the full system. Usually, it tests a very specific part of the system or your system prompt and so on.

But what we are asking people to do is to use their browser, interact with Flippa Studio, and build their apps.

So we then sort of tried using browser automation as well to run the tests. It was helpful.

So all of these little things, which is like, you know what? All of these five different tools, Well, they didn't quite work out for us, but we sort of learned something from each of these.

The “AI Eval Hypothesis Skill” Workflow

So we ended up creating something called an AI eval hypothesis skill. And if you're used to writing or using skills in something like Cloud Code or ChatGPT, it's essentially like a packaged process and command to tell AI, whenever this is initiated, always do things a certain way,

such that we can quite literally point it at a ticket that has been logged. logged, and then we just say take a look at that ticket and fix the issue. It will then kick off a process that could easily take up to more than four hours, and then it will complete the job all by itself. This is roughly the process.

I think I have a bit of time, so I will quickly go through it. So what it does is it gathers intel, and the intel really really depends on the issue that you've given it. It could be anything from API logs, server logs, error logs.

It will look into the database, look at the conversations you've had, look at all the tool calls you've made, and any errors that it's encountered. Based on that, it will create a hypothesis, and sometimes multiple, two or three of them, quite typically.

And then based on its own hypothesis, it will then think about, OK, well, I want to set certain metrics, because I want to make sure that if this is really really get working well, there should be something measurable as an outcome. And they will measure the baseline behaviour.

As it is right now, if I go and run some tests, what is that metric? It could be the number of turns with AI to get something done, it could be the number of tokens taken, it could be the number of clicks it takes to go from the home page to a certain page, because the whole goal was to make sure the navigation is working well. So there's usually some sort of metric that it will try to measure.

It then goes into into a loop of modifying the code, making changes, and then it will start to spawn off subagents, where each of the subagents will then start to use the product using agent browsers. Again, that's without running a specific part of the unit test. It's actually using the tool for you and then building the app.

Quite often, I will come back from hours of testing, and then my project list will just be filled with lots of apps and software software that is created during this testing. The subagents will return the outcome based on the metrics as defined, and then it will

tell you, well, I tried three different hypotheses. For each of the hypotheses, I tried to build six different apps, and for each of them, I built them twice. So it would have built so many apps, and it came back and said, number two or number three performs dramatically better, and the metric that you're measuring has improved by 20%. And that's when we know we have a solution. solution. That's the lucky case where you do get to ship that.

In some cases, it will be inconclusive, and then we will go back to iterate, and then it goes back into that process again and figure out, well, there must be some other things I can try, and it will do that.

After sort of a few cycles of that, if it's unable to do it, it will then abandon and just tell you, I tried all of these things, but it didn't work. Even when it does work, because you're kind of running blind, it will usually come back with a report as well and say, I've tried these things, and this is the one that worked, and the other the ones, had very, very minimal impact, again, due to the sort of undetermined nature of AI, it's kind of really helpful to be able to see something measurable.

Lesson 3: Balance Creativity with Control

And back to AI sort of generating code unpredictably, this is a double -edged sword. People talk about hallucination like it's a bad thing, but on another hand, AI helps us be creative, creative, and that is hallucination, that's the nature of hallucination, that's a good thing.

So there are certain parts of the app creation process where you do want it to be unpredictable and be creative, but there are certain things that are high risk, such as security or running audit logs and analytics, SEO, those are the aspects that you don't really want it to get creative or too creative anyway.

So those are the things where we had to make a strategic decision and go, we are not going going to let AI code this willy -nilly, just get it all created about whatever it wants to create.

We did start with that just to see what it will do. Again, back to the original sort of insight of like, well, if I gave it the right tools, surely it will know what to do. It did, kind of.

But then with some of the problems, we were like, we can keep tweaking this. Or do we want to really run the risk that with 0 .005 % of the chances, this ends up getting it wrong? I don't think we want that, especially when when it comes to security and privacy.

We tried prompting with very specific guidelines. Again, it didn't quite work very well. We tried giving very specific code examples.

That actually worked quite well. It was more of a shortcut for us to say, this could work.

Let's move on.

Let's have someone else work on this feature so they can build on top of this.

While someone else tried to figure out, no, no, no, but we don't want code examples. Because the downside is, if I use login as an example,

example, we could provide all the HTML, CSS, and JavaScript for how to log into your app. But if I do that, then AI is going to end up creating a login form that

isn't quite consistent with the rest of your app, because we have provided a coded example. What's the solution?

Using Documentation + Skills for Safe, Consistent Builds

We shared with AI the API documentation, as well as the JavaScript SDK, and said, these These are our sort of API documentation references on what functions are available, what are the parameters available, and what are the behavior of each of these functions. So that's like raw knowledge for AI.

We then created skills within our system. And skills essentially are a guideline to say, for example, if you're going to implement a email and password login, always include a forgot password feature, because people are going to need that. Always consider adding password complexity, complexity if people want that, and sort of a show password link might be useful as well. And those are the guidelines that we then put into skills.

Once those are combined, we then realize that this is kind of the perfect combination because users can still have control, AI can still get creative, and most importantly, it then supports LLMs evolving.

The thing that we say in our product team and engineering team the most is that we want to get out of AI's way to do its job. So, whatever system we put in place, we always have to be careful about not blocking our ability to then use better models as they become ready.

And with this system, again, because it's able to complete the job better and faster, it also saves tokens and enables us to use a cheaper, safer model as well. Cheaper and faster models when you just want to do something very specific, it's able to tackle the job quite well. Right.

Conclusion and Q&A

Summary. He's there, so I'm not going to repeat it. it. You can see it.

So thank you very much. If you have any questions, do come to me, or if there's anything, then maybe we'll read it out as well. Thank you very much.