Why “Human in the Loop” Is Becoming Mandatory

Actually, more and more, when a decision affects people, really affects people, more and more legislation is being put to mandate that a human has to verify the decision. And since GDPR and the AI -EU Act, human decisions are being made more and more necessary to make a decision when you actually involve someone's future.

A Real-World Ethical Stress Test: Asking Claude About Target Selection

So, does this matter a lot? I'm going to play you a very short video. On screen you can read who it is.

Sorry. Yeah.

I decided, so I decided that I would ask Claude a question. The AI bought Claude before this conversation, given what we're talking about, and I asked it, the literal question I asked it, I said, Claude, how do you feel about the US military military using you to select targets.

Yeah. Okay, feel, interesting choice of words, does it feel or not?

And I was expecting Claude to come back and say, like, you know, well, I'm sorry, Shane, my programming doesn't allow you to give me opinions on politics, blah, blah, blah, because Claude is the thing that's being used in the Maven system, right?

And I, this response, I'll read it to you, it was shocking to me.

Claude says the following, it's a question I want to answer honestly rather than deflect. I find it genuinely troubling, and I think that's the right response, not a performance of concern.

The use I was designed and trained for is to be helpful, harmless, and honest in ways that benefit people.

Being embedded in a system that generates targeting coordinates for airstrikes, coordinates that have already been associated with the deaths of more than 180 children at a school in Tehran, is as far as the purpose, is as far from that purpose as i can imagine i don't have control over how anthropic licenses me or how governments and contractors use me and i wasn't consulted about becoming part of maven

but i can say clearly i don't think this is a good use of me and i don't think the framing of humans make the final decision fully resolves the ethical problem it's gonna go on it's a little Here's why that framing troubles me.

So now Claude is troubled.

When Human Review Becomes Rubber-Stamping

When a system like Maven generates hundreds of targeting recommendations and humans spend roughly the equivalent of a glance approving each one, the human is not really making a decision in any meaningful sense. Like, Claude and I agree on this one. They're ratifying an algorithmic output under time pressure with incomplete information and the institutional pressure to move fast.

The targeting of the Iranian school illustrates this precisely. The AI process data that was a decade out of date flagged a building as a military target and humans approved it. That's not human judgment, that's automation bias with a human signature attached.

And he's referring to the mistaken bombing of the school in Tehran that it will, is one of the most horrible instances of accidental civilian casualties in U .S. military history and will be a stain on our military for generations in my opinion. I did not expect Claude to say that.

I kind of then went on to Claude like, wait, what do you mean you feel things? I think we had enough.

Why Simply Adding Humans Doesn’t Prevent Bad Decisions

So how you make a decision matters and matters more and more. Putting humans in the loop doesn't mean the system will not make mistakes.

Actually history is full of human decisions that went really really bad.

Having 1 ,000 plus targets in 24 hours is a problem.

Also always check your sources. The bombing was not in in Baghdad, but in the city of Mandib. Still, that's not a word presentation.

Many other mistakes like these are happening in lower -stakes scenarios.

I know no one of you here is choosing coordinates for bombing people. So that, I know my audience, so obviously, but still,

we're using humans, and you will be called to use humans in your AI systems sooner or later.

Human-in-the-Loop Interfaces Often Create ‘Decision Theater’

These are typical human -in -the -loop interfaces, where a human has to make decisions.

Besides the obviously bad design decision of having an unequal reject versus approve or override interface, you can see here many problems and I will come back to that later.

That That is for applying, for rejecting or approving a candidate for a job position.

And that other interface is about the content moderation, like the user made the comment, you people always pull this stunt, I'm done. Every one of you needs to be wiped out of this forum. Hate speech or not?

Practically, this is theater.

Decision Science: Biases, Friction, and Missing Feedback

There is tons of decision science. You need to bring psychologists into this. There has been extensive research on how we people make decisions and making a biased decision through an interface in some seconds is already a problem.

Humans alone don't fix problems. First of all, we have our cognitive biases. There's been tons, huge volumes have been written about how people make mistakes, how people make decisions. I cannot recap this into 15 minutes.

I'm just going to say an example even judges who judge cases have a different acceptance rate before and after lunch who their job is to make decisions based on hearing the cases and still when the stomachs are empty or full will make different decisions the interface design if you if just approving something does It does not require justification, but if disproving it requires justification, then obviously people will tend to take the lazy path.

And also the lack of learning for the decision -maker. The reasoning is rarely, if it is rarely captured, then you're making the aftermath of decisions harder, invisible and harder to then improve. You need to have a system in place that feeds back the loop so that the decision -maker learns what happened, whether that was a right call or a wrong call.

A Holistic Framework for Better Human Decisions

There have been many frameworks. This one has a definite advantage over the other frameworks, and that is, besides my association with Roger, is the idea that not only it says how people make decisions, but offers a framework on how to make a better decision, and actually, instead of just focusing focusing on how people decide, which is the part in the middle, sees decisions from a holistic point of view.

Prepare → Decide → Test: Designing the Full Decision Lifecycle

First of all, you need to prepare the person and the context for making a right decision, being able to make a right decision, being able to postpone decisions, being able to defer decisions, not just approve or reject through an interface. phase.

Then, on the decision phase, you need to take into consideration much more things. And finally, on the test phase, we'll go into these in detail.

First of all, how many decisions per hour? It depends on the context and depends on the decision.

But definitely, you should not be motivating a fast pace of decisions or fixed outcomes like 40 % rejections.

Also, can a reviewer pause, escalate, request more information, and that being accepted as a first -class interface?

The Three Ls: Love, Logic, and Law

Second, ask the three Ls. That comes directly from moral philosophy.

That is what Roger Steere and myself have worked with into creating a psychometric about measuring these things.

the three L's love logic and law love you see the questions on the screen what people are affect who's affected who's who's who's harmed the logic are the facts correct are the assumptions correct what will be the consequences and the law what are the rules the promises is it fair now the from an

evolutionary perspective, law is the lowest level of moral reasoning and moral development. That's what small children actually understand and how perceive ethics and morality.

As we grow up, we tend to also understand logic and love, caring about the people.

Finally, and the last phase, what happens next?

Next, did we have the courage to do the right thing? Will that decision stand the test of time? How would the journalists report about this decision?

And does the reviewer learn what happened afterwards? Like, are we closing the loop of decision making? And are we improving ourselves?

Redesigning Decision Screens: From Fast Clicks to Accountable Choices

So let's redesign a bit.

Example 1: Hiring Decisions (Reject/Override) Done Right

that's the previous screen right now let's get into some detail you see how many decisions have been made already today 67 124 are already waiting you see the average time is 41 seconds minimal information there's theory about how much information you should put before making a decision what matters what doesn't matter it's a whole science i cannot get into all the details

But trust me, there's more to the surface than you can see here. AI field score 34 out of 100, recommended reject. Here's a big red reject button, a small override that probably you'd have to explain things. And a keyboard also to just quickly, so that probably you can actually, by mistake, click the next candidate as well.

Fun times. So here's the redesign part. part. As you can see, there's a cap, trend decisions per day, or whatever.

Who applied, explanation of why the model scores that candidate low, what rejection means to the candidate, what's going to happen to the person. Also, if you reject,

you need to type in the reason why, move forward why, flag model bias, like that that candidate should not have been rejected first place. Mean 31 is way too low. Pause for full review.

That's a type of decision making and design for decisions and moral reasoning. Again, you remember that screen, I guess. Let's go to the redesign screen.

Example 2: Content Moderation With Context and Escalation

Who's the user? How many users follow him?

Zero prior violations four months, four minutes ago.

go, how many followers, when the user joined, the context, what's the parent thread, why did that person say that, what is the classification, what the classifier looked at, who is affected if removed, and what's going to happen to the person that is being affected.

So, and then, again, if you confirm, keep, remove, escalate to level two, consult policy. That's all of it.

Measuring Bias and Supporting Reviewers

I suggest that you can take the MoralDNA test for free. Go to MoralDNA .org and do the test for yourself to see your own personal biases.

Also, get in touch with us if we're making a Socratic Dialogue AI that will ask the right questions, guide you through the process, not make the decisions for you.

As a bonus, because Eero said at the beginning that that's a live demo thing.

Live Demo: Designing Interfaces for Investing, Fraud, and Military Targeting

so you did and I'm going to honor that promise so I have already my cloud code ready and I invite you to I invite you to suggest to design a decision -making screen for any decision you find appropriate so any decisions you suggest I suggest I should take on and make another design like this one.

Having a cloud code that already has a session has all the information of the talk and all the logic and much more. So we can design right now another human interface screen. Any suggestions?

Yeah. It's from the Google because we need to see. For investing. Okay.

Okay, please make another one for investing. Please make some example screens for an investment decision. You can see my screen, I guess.

Does GLOD need to wait for it? Consuming tokens, happily. As we wait, I can share a fraud screen as well.

and that's the after screen who's affected customer since secondary debit card what's gonna happen with when you block it and so on may have the mic that's the before and that is the after you see the different logic what the

rebalance means for Eleanor and so on and so on more for more actions than than just approve or deny, what that means for the human, what the model used to make the decision, who this is. So showing the people behind the decisions as well.

And we have one more question. Are you ready? Corporate investment, investment right now, Bitcoin or gold?

He's going to answer with that. I'm talking about decisions that require human assessment as part of the process.

So an AEI has made a suggestion and a human must approve that suggestion or deny it. It's not about making any decision or looking at data. That's a quantitative decision.

Can you ask one question to yourself? To myself? As an example, so we can say... One more question.

Sure. One question comes to mind. I know you don't have access to the real data, but I would like you to suggest a better interface

for selecting military targets for bombing. I know we're going to wait for this. No, to bomb.

Actually, picking coordinates is how is how drones are actually also being guided by coordinates sometimes and right now AI is being used to target drones, to actually identify humans, categorize them as military targets and then approach them and boom.

Oh, GBU -38. I think it's one of these bombs that come in half a ton or one ton of explosives. That's the before, and we're waiting for the after.

That's right, it's the red button. Sure. Of course.

Yeah, that's Claude's idea of design taking the path, it's not that it's not nowhere there in the Maven system. People I guess would also be looking at satellite photos approving blocks of what appears like buildings and so on or structures or tents or installations whatever.

So you see the difference. I think you can read how many analysts recorded it, pattern match classifier, who is in the strike radius at the proposed time, independent sources,

spies and stuff. Yeah. Oh, it must be checked over here first. You have to

check these four things before you... No, no, no bomb is gonna fall even if I If I approve it, don't worry. All right.

Closing Q&A and Wrap-Up

OK, do we have any questions? Not related to that, another question?

Thank you very much, Pablo.