Introduction

My name's Vanessa Porter. I've been working in what we used to call data and analytics, and we now call AI, since dinosaurs roamed the earth. And I've done a lot of work in working with enterprises, so financial services, government, large retailers, all of those kinds of systems. And a lot of my work has been in how data is used in customer -facing systems to improve customer journeys.

So I know that most of you won't look at a chatbot and go, oh, my God, thank goodness, it's a chatbot. I'm so happy. I'm going to get so much sense out of this. And it's a massive, massive issue.

you.

The scale and types of customer vulnerability

So 49 % of people, this is the latest FCA figure, experience vulnerability at any point in time.

And when we're talking about vulnerability, we could be talking about a persistent vulnerability, like, you know, you could be a wheelchair user, or we could be talking about a transient vulnerability, like you're having a mental health crisis, or you've been bereaved, or you've lost your job, or you've got a massive hangover because the England game was the night before.

Why automated customer journeys miss vulnerable states

And what customer -facing, digitalized, automated, and AI -driven customer -facing systems don't ever do is think about people in vulnerable states.

Cost pressure, automation, and the fragility of trust

So I was having a chat the other day with the CDAO of Liverpool Combined Authority, and she, like most organizations, is under huge cost pressure, huge financial pressure to automate and AI assist those processes. But she knows that they've basically got a once -in -a -generation opportunity to make these things work. And once they've lost trust, it's gone, and it's completely gone.

So what we're ending up with, I think, is a situation where some organizations are kind of rushing into this stuff and finding that it breaks, but for a lot of organizations, they're under huge financial pressure.

The water companies, I spend a lot of time talking to water companies, they're under huge financial pressure to add AI, but they're terrified of what it's going to do to vulnerable customers. So there's a lot of reasons why, that actually aren't technology -related reasons, why this is a massive challenge.

When KPIs drive the channel instead of the customer

Containment, cost-to-serve, and other misaligned metrics

And one of the biggest challenges that I've seen is that customer experience and these various ways of communicating tend to be driven by either company KPIs or line of business KPIs or even channel KPIs.

So a chatbot would tend to be driven by a KPI around containment. So we want to contain the conversation within the chatbot and we We want to make sure that nobody's speaking to a human.

For a contact center KPI, they're looking at what's my cost to serve? How can I keep the cost to serve low? For a website, they're looking at things like, did somebody hit a particular Google tag? And none of those things are thinking about, did the customer who came to me get what they wanted at the end of it?

Regulators shifting toward “good customer outcomes”

And what's interesting is that the regulators are now starting to think about delivering good outcomes for customers. They haven't described what those good outcomes for customers are, but that's the way that they're beginning to speak or they're beginning to think about things.

so what we've got is a situation where you know everybody's under enormous pressure to save money to um to deliver at least ai assisted systems the technology vendors how many technology vendors have we got in the room this evening i'm not going to put your hands up technology vendors say you say stuff they they say stuff about how things are going to work it's not necessarily always

A platform to test customer-facing systems for vulnerability outcomes

always the way it is so what we're doing is we're building a testing platform that tests customer facing systems against a full range of both persistent and transient vulnerabilities and we're testing to make sure that customers end up with the outcome that they need rather than delivering the outcome that the particular channel owner wants

Why people rarely disclose vulnerability directly

so there's a lot of challenges in this so you know technology is great technology does what technology does but people are much more complicated and people don't ever disclose vulnerability so people don't go to a chat bot and say hello my name's Vanessa and I don't think I can pay my gas bill this month how can you help me that's not what people do so people will They'll kind of go around the houses.

They'll circle the website looking for an answer. They'll use kind of deflecting language. They'll perhaps look at different channels. They'll come back at 3 o 'clock in the morning because they're panicking.

So what they're doing is they're giving off all kinds of signals about their vulnerability without ever actually disclosing what that vulnerability is.

so as i said you know linguistic data being able to understand the signals of distress are really important and that and the interaction data so interaction data things like time stamps metadata um a number of number of uh channels that somebody's visiting number of interactions these are all things that would indicate whether somebody was vulnerable so when you've got so many

any variables so each of you if you were in a position of vulnerability each of you would go to a system in a different way and you would act in a different way and you would disclose your vulnerability in a different way so somehow we've got to turn that into something that's machine

Turning vulnerability into something auditable and machine-readable

readable so there are so many elements of this so first of all we have the rules so the rules are the regulations. So the rules are things like the Equality Act or what Ofgem says or Ofwhat says or any of those things.

Then we've got the evidence. So the evidence is, for us, it's a combination of two things.

So number one is lived experience. So what do people who have specific vulnerabilities, how do they feel when they're in particular situations and how do they behave in particular particular situations and then academic research so academic research but also research by the regulators that's all really useful evidence for us and those things turn into signals so those

signals could be linguistic signals they could be interaction signals they could be signals signals you know that somebody's coming to your site you know in the middle of the night they're trying to engage with a chat bot in the middle of the night all of these different things are signals and because it's so complex and also because what we need to do from a testing

Ontologies: making signals traceable, defensible, and reviewable

perspective is we need to be able to audit this is we have to build an ontology so we have to build a kind of detailed very precise auditable way of connecting all of those different entities has anybody anybody done any work with that or that with ontologies it's a it's a whole thing it's a whole thing where you sit in a room with somebody who's incredibly precise about language until you want to throw stuff at them but that's the way that you build something that is traceable defensible auditable with all of those all of those different elements so the ontology is kind of the structure it's how we organize that data

Real data vs synthetic data for scenario coverage

data how we organize that information and then we add data so there's two ways of doing this so so one is we use real company data and that real company data gives us some scenarios and some of those scenarios allow us to understand how people make their journeys through how people behave in particular situations but that only gives us a small amount of insight so the other thing the

other part of what we do is we create synthetic data that allows us to test against a full range of vulnerabilities and we can kind of look at all of those different edge cases so we can look at the edge cases from something that's going to create a small amount of harm to something that's going to be potentially catastrophic once we have that data we can build that knowledge graph and

Knowledge graphs and how organizations can use them

the knowledge graph which will constantly be updating with more lived experience more academic research more data from our customers will end up giving us this incredibly rich way of being able to kind of categorize categorize view human vulnerability and then we can query it so you

You know, for some customers, what they want to do is they want to test before deployment. For other customers, what they want to do is to do continuous testing.

And then for other people, what they're starting to think about doing is using that data to build agents so you can identify somebody who's vulnerable and take them through that journey. So as we build up that data over time and as we build up that evidence over time,

it becomes richer and richer so um let me just show you a quick view of what i am what we're building so this is good old claude where would we be without him so the way that we look at this

Scenario-based testing: journeys, scoring, and outcomes

is um we would have we would start with an outcome so an outcome could be something really really straightforward, like a customer wants to be able to understand how to make a payment plan following a misdirect debit.

Then we have a number of scenarios. So scenarios would cover a range of vulnerabilities. So in this case, somebody who's bereaved, they've missed a payment

and they've got a debt letter. Then what we do is we recreate those different customer journeys and we score each one of those customer journeys.

Measuring trust, energy, memory, and clarity through a journey

And one of the things that we're looking at is what happens to a customer's trust energy memory and clarity levels as they start to walk through those journeys so somebody who's bereaved for example and has just received a debt letter

might start in a low trust position because you know you i'm sure you you've all experienced this you know where something unexpected happens you don't think it's fair you start with low trust you've got low energy because you know you've got low energy you've got you don't really understand why it's happening to you so we start with you know kind of why with with where where those

people would be against those four elements and then as we step through each one of those journeys we start to look at how those resources get depleted over time so what's physically happening to that person and how is that having an impact on that person then as we look at each

Mapping moments to obligations and creating an exposure score

step we can also look at a regulatory obligation so what's happening to a customer in that particular moment and how does that map to a regulation so mapping to a regulation or or mapping to a regulation doesn't necessarily mean you're breaking the law, doesn't necessarily mean you're going to get reported to a regulator.

But what it does is it allows us to start to kind of almost start to add up all the different times when a company is breaching those regulations so we can start to create a vulnerability exposure score.

So that vulnerability exposure score is a combination of, is the customer getting the outcome they wanted? Are we meeting the regulatory obligations? And what is physically happening to the customer? What's happening to the customer in terms of their emotional state?

So that's what we're building. That's what I wake up at 3 o 'clock in the morning thinking about. I should maybe go on holiday or something.

What the work is teaching: signals in what people don’t say

thing um but the things that i'm learning um you know having spent years and years and years looking at data that people create so you know things people buy and you know things um things people apply for you know all of that that kind of physical stuff what i'm what i'm learning now

is that the the signals that matter most in this in this situation isn't what people say and isn't necessarily what people do it's the things they don't say and it's the way that the kind of round the houses things that they look at in this situation you know we're looking at this

kind of ontology approach which a lot which means we have to be very precise about what we're doing because we do need this to be defensible and traceable so that precision is really important

The hard part: structuring vulnerability data, not just detecting it

it's really annoying but it's really important and making vulnerability machine readable is going to be a never -ending task so it's a really hard problem and it's a harder problem to make that machine readable than it is to detect it so the detection engine itself is actually reasonably straightforward it's about making that data machine readable and that structure needs to exist exist in a defensible form before anybody can read it so if you're if this is something you're

Next steps and where to find the work

interested in then um what we will be doing is um creating a portal in fact where's dan gone it's probably gone what i think i'm going to do now is create a vault of all of those kind of lived experience snippets that people tell me about on a day -to -day basis um you can find us

on LinkedIn, Kamina, and yeah, that's me. Thank you very much.

Q&A

Could the same approach be used to exploit vulnerable customers?

I can see a company using this to take advantage of vulnerable people. Would you be able to flip it around so you can detect if a company is using this to push vulnerable people for a short -term profit?

I mean, I guess they could. I mean, our market is essential services, So our market is the regulated industries, the good guys. But, you know, I suppose there's no reason why that same approach couldn't be used.

Just not by me. I've got enough going on. Yeah, I was just wondering. Oh, yeah, maybe.

I mean, the thing is, I suppose you can see, there's a lot that you can see from the outside. so so yes theoretically it's then you know what what would I do about that great I'll write it down I'll get I'll get on to that thank you

How lived experience and synthetic data are sourced and combined

yeah oh um I was just wondering so you know you said you have a combination of lived experience um that you're getting data from as well as a synthetic data like what percentage like how is that weighed are you getting more lived experience evidence so well so so the so

So the synthetic data is generated from lived experience. So lived experience comes from different sources.

So we run our own lived experience workshops. So we simulate, when we're working with a customer, we would simulate their processes. So we would simulate, you know, kind of two or three outcomes with different types of people, and we would capture all of that data.

We also work with charities who represent people with disabilities capabilities um our delivery partner uh an organization called autocon who are do you know autocon so they're uh they're so great so they're um they're basically tech consultants um who who are people with autism so they were started by a guy who had a very technically gifted but and also autistic son so we we feel it's important to have people with that lived experience you know know design designing for other people with vulnerabilities so it's a combination of lots

of things so you know the lived experience will come from our workshops they'll come from charities they'll come from you know facebook madness a lot of facebook madness where we get a lot of information from reddit um and then we use so so that kind of that that feeds into two channels

one is forensic linguistics so we work with a forensic linguistic analyst who is an expert in distress signals and she helps us to understand what people actually mean I think that the next piece of work that would be really interesting to look at would be people whose first language isn't English because the way that people express distress in in english when english isn't your first language is is is a whole different thing so it's a it's a bit of a never -ending thing but yeah it comes from all sorts of places

yeah i'm quite old though i feel like i feel like one day i just want to sit

Pilots, deployment stage, and the business case

i was just wondering uh what stage you're at do you have any like pilots running with

Yeah, we've got a pilot running with Northumbria Water, who are, you get into this and you find out that you've got a favourite water company, and they're mine. So, yeah, we've got a pilot running with them, and then we're just about to start off with one of the combined authorities as well. But the use cases are kind of multiple.

So what we're looking at in water is debt because the industry is spending 200 million a year just on servicing debt. And if you can get to people early, then you can you can massively change it. So this is not just all about being lovely. There's a you know, there's a big financial gain there as well.

Hi. Thanks. That was interesting.

Technical implementation questions

I was just wondering from a more technical perspective what you're doing with the knowledge graph. like how's that kind of implemented at a high level and what systems are

like writing to and interrogating that do you know what i've i i can't tell you that just because i don't know i'm more on the kind of i'm on the swanning about end of all this

but if you're interested i'll i'll drop you an email one more i've got a quick one i'm

Cohorts vs individuals: granularity, ethics, and diagnosis avoidance

an interest oh sorry um i might not have fully understood um what was going on but uh that's on me then but when i saw the scenario testing oh sorry the scenarios it looked like there was sort of like an average number rating on what we'd expect in the certain levels of distress for each scenario i guess but does that go down to a more granular individual

person level or does it sort of average out no so we're not so we're not looking at individuals so we we look at cohorts and we don't use any data to to make diagnoses so what we do is we look at signals of we look at signals of distress or signals that somebody is you know signals of vulnerability and we group them we group those signals of vulnerability what we what we're not

in the market for is saying you know somebody's behaving like that this and therefore they are like this and we're not going to go ever go down to a granular granular level if an organization wants to use the same techniques which some of them do use the same kind of techniques to do an intervention lovely but that's not that's not what we're what we're doing i think there's too many kind of ethical um considerations for us to really think about that okay because you did say that's

Accounting for neurodiversity, literacy, and language differences

It's like if you have maybe autism on a non -atypical behavioural set compared to English as a non -primary language, these could all muddy that average, right? So what would you get around that?

So we have a series of input parameters, and the input parameters cover a range of vulnerabilities. So one of the input parameters would be low levels of English and low literacy.

um the the neurodiversity thing is that there's there there's no kind of specific there's no one set of characteristics that would identify somebody's being autistic for example we might look at something like low trust um is is something that that we hear a lot from um uh the the people with asd that that we work with so once um once they've had one bad interaction with either that organization or another ai assisted system then their trust levels dip so we would be assessing somebody with low trust rather than somebody with a particular condition

does that make sense yes yes so so we would look at we would look at things like you know we when we're doing lived experience workshops we would look at you know the language they were using but we would also look at you know kind of how do you how do you feel in this moment what what do you you struggling with what's stressing you out what's making this difficult for you all of those all of those things so we so so that the i suppose the the signals are things that we're interested

Wrap-up

in rather than the condition amazing really important are you hanging around yeah can do brilliant not gonna make me watch football are you no vanessa thank you very much