Introduction

Good evening, everybody. My name is John Brush.

I'm the co-founder of Sigmology and the lead developer of Supervision AI.

Positioning in the Testing Landscape

Supervision is our AI-driven test automation platform that takes you from plain English to fully automated browser tests without writing a single line of code. And over the next few minutes, I'll show you where we fit into the testing landscape, what makes supervision unique, and why it matters for teams that want reliable, maintainable automation at scale.

Four Axes: Driver, Scope, Target, Purpose

So the deluge of QA terms you see represents the wide world of testing, and in a quick tour, I'll map supervision along four axes, driver, scope, target, and purpose.

So historically, teams were either human-driven or script-driven. Now there's a third option, which is AI driven. Our AI identifies elements, adapts to change, and steers execution.

Testing can apply to code or UI. Our focus is browser-based UI flows, the point where the rubber meets the road for the user.

In terms of scope, we go beyond unit and integration testing. We operate end to end, validating complete user journeys across services.

Testing can have many purposes, including validation of performance, scalability, compliance, and so forth.

Our core is functional correctness, answering the question, does it do the right thing reliably?

So in summary, Supervision AI is an AI-driven, end-to-end, functional, browser-based test automation platform.

How Supervision AI Works

Supervision AI works in three simple steps.

Three-Step Workflow

Number one, describe the test case in plain English or French or German or Swiss German or whatever language you like.

Number two, the agent executes the test steps directly in the browser And number three, you receive a report with a full transcript and test status.

Robustness to UI Variations

For example, let's say you ask the agent to enter values for the first and last name fields of an application form. The agent identifies the fields and fills in the specified values. The test then completes successfully.

Alternatively, the agent may identify small discrepancies in the UI. For example, last name might be called surname. In this case, the agent still completes successfully, but also warns you about the deviation.

However, if major changes occur, such as the first and last name fields no longer being on the page at all, then the agent informs you that the test has failed.

Key Advantages

So what are the key advantages to supervision AI?

Zero-Code and Accessibility

1Number one, 1this is a true zero code system, meaning that there is no code whatsoever, not even intermediately generated code. The use instructions are interpreted directly by the agent at runtime.

No special skills and no developers required means that anyone can write tests.

Resilience, Speed, and Coverage

Small changes in the UI do not break tests. This results in fewer false negatives and less overall maintenance.

New tests can be up and running in a matter of minutes rather than hours or days. Basically, you can write tests as fast as you can think.

Use of test development means more tests in less time, resulting in better test coverage and higher quality.

Cost Savings

And finally, eliminating test-related development and maintenance results in significant savings.

Live Demo Walkthrough

So, that's all of my formal presentation, and at this point I would switch over to our demo, and I'm gonna go into Yep, and let me just be sure, yep, connected, good.

Okay, so I have set everything up with a special user for MindStone.

Did you guys know that you had an Ethan Caldwell? Never heard of him? Okay.

So that's gonna be our demo user for today.

Application Overview

This is our application and you have the kind of main things that you have in most applications. You've got organizations and projects and so on and so forth.

The meat of the application is in the test cases and I have prepared a small test case to show you how this works.

Defining a Test Case

Basically, you have, so instead of talking to a chat bot, you're describing your test case in steps, and the steps are just written in whatever language you want. So for example, for this particular test, we're telling our test agent to go to this URL, And then we're going to go through a series of steps where it creates a record. And then finally, at the end of the test, it does an assertion and makes sure that that record is being created.

Stepping Through the Debugger

So I think the best way to show you is just to go into, to step into the debugger. So for anybody that's familiar with development, it's actually quite similar to, to program code except the individual instructions are being executed by an AI instead of by a CPU.

So we step into the test. And you can see up here in the right hand side that there's like a status where you can see that it's thinking or you can see that it's paused now.

So you can see here which test case is currently running. You can see which test step is currently running.

And let me just do a quick refresh here.

And we have a remote browser where, so basically everything is running on our infrastructure. So this system is set up to be either an SAS solution or an on-prem solution, but it's fully self-contained, except for the LLM.

Agent Reasoning and Actions

So you can see here that we told it to go to this URL, and you can see what the agent is thinking up here, and you can see what actions it's decided to take, and you can see that it's taken us to the correct URL.

So I can keep stepping through, and now we're telling it to click on the Create button, which is here, and I don't know if you can see that it's highlighted in orange. And if I step over, then it will take that action, click on the button.

I'll just move this up a little bit so you can see that a little better. Keep going here, you can see that it's thinking.

Data Entry and Interaction Flow

So we tell it to enter the following data, name John Doe, age 27. And it should also click on the date picker. And you can see it's come up with three actions, fill in the name, fill in the age, click on the join date picker. So I'm gonna step through those and you can see in the browser that as I step through in the debugger, it's doing those actions.

And then we'll just continue on. The next thing is to select the current date. And out of the context, it knows what the current date is. And we'll do that.

And then the next thing is to click on the dropdown, on the department dropdown. And it's gonna do that. And then it will select development.

I hope I'm not boring you with all these steps. So, and there we go.

Assertions and Results

And then it's going to click on the create button and that's gonna be the end of creating the record. And then we come back into the overview, and then the last step is to do an assertion, which is we're asking the agent to be sure that that record actually shows up in the overview. And you see that the agent says, yes, it confirmed that it went back to the overview and that John Doe, age 27, is there.

So that would be one run through.

Full-Speed Execution and Retry Policy

So we can actually run it now at full speed. Maybe I just show you the browser view this time. I can try to show both at once.

Up here you can see that it's thinking. It's going through the steps. And when we get to the point where it hits the create button, we should get a validation error. And then the next step, the assertion should fail.

I know it's a little hard to see on such a small screen, but. So there, it's on the assertion. It fails.

Now it's gonna fail three times because of the way the system works, the system doesn't assume that it's seen the correct state of the browser the first time. So there's a fallback mechanism where it will check again and there's a fairly sophisticated thing called a retry policy which you can define in a lot of detail if you need to but there's like default ones built in.

Real-World Adoption

In any case, we have a, we have one large financial institution that I won't name here in Zurich that we're working with. We have a basically a defense contractor in the states that we're working with.

And so far we've had a lot of excitement because, I mean I'll just give you one example.

Customer Example and Impact

One of the customers that we're working with comparable to Hazana, let's say, is spending, they have, we did a calculation, and they have about 30 FTEs right now that are doing nothing but developing test scripts.

So they're working with tools like Selenium and Playwright, that kind of thing. And then there's a whole bunch of business analysts that are feeding them data.

And what we think we can do with this tool is to basically cut out all of that development and just let the business analysts do their test directly. That's kind of, that's the dream and that's what we're targeting.

Conclusion and Call to Action

So, supervision.ai, supervision.ai without the E. We're on X, we're on YouTube, we're kind of on everything, but it would be great to hear from you if anybody's interested and if you have any questions. Thank you.