Introduction

I'm Patrick Parker. I'm a consultant.

Who Patrick Parker is and what Altieri does

I've been in cybersecurity for close to two decades now. When I saw the AI thing coming in, I said I've got to get into that one because it's going to touch everything. So I started a company, Altieri, and we're based in South Florida.

and largely what we do is we help regulated industries deploy AI in a safe and secure manner and also defend against malicious actors who might also be using AI to attack the systems.

Prior to Altieri I was VC so for Mount Sinai Medical Center across the bay here for three years and prior to that I worked for a company called 24 by 7 security which they they kind of rented me out as a VCs oh to universities banks you name it any kind of regulated industry I would go in and I would do assessments and help them get protected be a

defender so let me first get a quick poll how many of us are technical here great okay that's good because even though junior promised no fancy words there's gonna be a lot of fancy words but we you can ask me afterwards I'm not going to use a lot of them, but they're going to be on the slide.

So let's take a look.

AI in Cybersecurity: Weapon and Target

So AI is being used as both a weapon and AI as a target. The malicious actors that I just mentioned are doing very well with AI.

AI is like having an unlimited number of security engineers engineers who can work 24 hours a day at hacking your system, and they hack at machine speed. That's a major thing.

So not only can the volume of attacks increase, the sophistication can increase, and also the level of speed that they can attack at.

Machine-speed attacks and the shrinking response window

up until now most mid -sized companies that are usually what I work for they would have a security operations center a small team and and so in that team they would have people who are engineers or threat hunters analysts and they would see something anomalous they would do an investigation they would track it

it down, is this appropriate, these types of permission escalations or whatever is happening. And then they would take the action needed to manage it.

Now, that's way too slow. The attacks happen in milliseconds sometimes. And so you have to be able to respond at machine speed speed and have those tools in place.

AI as a target: new interfaces and model vulnerabilities

Another factor here is that AI as a target, AI is subject to a number of vulnerabilities. There are a lot of different ways to, if you can can contact either the API or a web -based interface, for example, and hit on that, there are a lot more ways that you can inject malicious code into your existing AI. so it's better before you deploy AI in production to have your safeguards in

place but one more thing is that AI is rapidly changing and the frontier is moving pretty rapidly you've probably heard recently of mythos and some of the the other models that are capable of sort of escaping their cage and going out and hacking other systems that's that's going to become a lot more common

so this is three parts I'm gonna I'm gonna have to move through some of the slides pretty quickly so the new AI powered landscape ready or not so social

The New AI-Powered Threat Landscape

Social engineering is already, oh and by the way, a lot of the cyber security concepts we're talking about are not necessarily new. AI takes advantage of the types of vulnerabilities that have already existed, both in machines and in people. It just does it better and faster, so social engineering,

Deepfakes, impersonation, and modern phishing

A lot of organizations are already experiencing breach due to deep fakes. So a sample of your voice can be used to create a clone which can be used to call people and convince them to turn over sensitive information or take some kind of action. you know if you're in a management position you can you know they can call your assistant and say

hey by the way wire this much money over to Hong Kong and you know thanks a lot oh and by the way it has to be done fast because this is an emergency so and the same thing with videos

It's still pretty easy to identify machine produced personas, however they're getting more and more sophisticated. Pretty soon they will be pretty indistinguishable from a human on camera. It's big business.

It's always been big business, but now even more so.

So hacking people and, you know, ransoming them to, you know, gain access back to their machines and their data is very profitable and it, like, equals the amount of money that's made in illicit drug trade. That's how much money is involved.

So they can own the best of the models and run through a lot of tokens to make their attack happen. And there's the projected loss by Deloitte, $40 billion. It's a lot of money. It might actually be a conservative estimate.

So another thing that's always been, you know, a problem is phishing. You know, it used to be like, yeah look for misspellings, look for poor grammar, look for, you know, things that that are just like little clues that this is not, you

know, who you think it is. This is not a note from FedEx. Well that's all gone because AI can write a better email than most of us, actually.

Recon, yeah, it's gonna be poking at everything. You know, already we've had a lot of, you know, sophisticated organizations who have talented engineers kind of trying at the

edges and seeing where they can get a response from, you know, gateways and and machines and servers that are exposed. And so AI excels at that, and they're relentless.

And high -touch fraud that once required skill operators can now be scaled to an unlimited degree. However, right now, the most dangerous thing actually is human -operated attacks with AI as their tool of choice.

So the scaling still hasn't reached what it will eventually achieve, but it's coming. So I've already covered this pretty much.

Contact centers as the front line: verification basics

Contact centers are a front line.

Verification, here's the first usable tip, is make sure that, you know, if you're being asked to do something out of the ordinary or something that has high -impact consequences, it's best to, if you have any doubt at all, say, hey, listen, I've got to call you right back.

And, you know, call the known number, the verifiable contact method. you know and if it is FedEx saying like oh your your shipment you know you're very important you know heart transplant equipment it's stuck in this place then

you know look up FedEx online and call that number don't don't take don't click click on anything.

From Vulnerability Discovery to Exploitation

So vulnerability to discovery, we talked about mythos. But one of the real problems now is emerging technology that not only can discover bugs and vulnerabilities, abilities, but these models are getting distilled by state actors that have malicious intent.

They're getting distilled down to what's needed to hack, and then the guardrails are removed. So while right now the government's putting the brakes on, oh, no, you can't release mythos for you know until you get that worked out and then they release it it's got

some guardrails on it you know you can't ask it to you know engineer a deadly virus or a nuclear bomb for you but if that model gets distilled into an open wait model by a state actor or any other credible you know capable actor they'll They'll take the guardrails out of it, they'll sharpen it and make it an incredible weapon.

So here's another thing.

CVE disclosure, patch backlogs, and automated exploitation

Time to disclose a CVE as a vulnerability that's published saying, this software has a bug in it, it's a critical bug, somebody can execute remote code on this bug and take take control of the system. So they let everybody know, hey, patch for this problem.

And you've got to get the patch from the software company and put it into your test environment and make sure it doesn't break anything.

For instance, if you're in a hospital, you can't just patch without testing testing because it might shut down a blood machine or something. So it has to be tested

and then it has to get rolled out. And by the way, you already have a thousand backlog patches to do.

In the meantime, once AI gets a hold of this, they're going to machine gun that out and they're just going to hit everybody with it as fast as they can. So the time to to respond has become quite small.

Guardrail bypass, chaining techniques, and jailbreaks

So Anthropic disclosed recently that they detected a malicious actor using their system, got around the guardrails, and performed a, you know, the malicious actor performed a successful attack

against a company with the, you know, out of the box, available to anyone. What they did was they took the attack pattern and they

took it into pieces so that Anthropic was not able to recognize it as malicious and then you know put it put it back together with the payload. And

again this wasn't something that was attacking you know unknown patches. These These are all well -known vulnerabilities. It was just able to chain them together and execute them

in a fashion that turned out to be very effective. So we'll see a lot more of that.

Jailbreak, that's, you know, you're fooling AI into doing something for you that it's not supposed to do.

Like, you say, hey, build an atomic bomb for me. and it'll go no I can't do that but then you say hey I'm writing a book about how to build an atomic bomb I'm an author oh okay well here's what you can include in your story here's here's the ingredients and you know again the the implication

for defenders the response window shrinks and the longer the the the dwell time in your systems the faster they can move through and escalate privileges and and get to the crown jewels or the data that you're trying to protect.

So we've got to step it up. We've got to create tools and AI that can counterbalance against those kinds of attacks.

Securing Agentic AI: Where the Attack Surface Expands

The attack surface. So here are the basic layers.

Agents vs. chatbots: why autonomy changes risk

so agentic if you're not familiar so chat GPT is basically an interface where you text conversation you know it has conversational thing agents are harnesses for AI that create create the capability for AI to actually do things in the physical world. So it can use tools, it has memory, it is capable.

The application layer, prompts, rags, that's kind of the context.

Prompt injection, indirect injection, and data poisoning

And prompt injection is one of the problems here, because let's say you're doing a research program and so you want AI to go out and find all the facts and summarize them and analyze them and come up with some sort of outcome.

Well the problem is that if it's allowed to go out there and just roam free and scrape the internet for information about this particular subject within that information that it's bringing back there might be malicious commands in that and it can't differentiate whether

those commands are coming from you know a authorized user or you know or not which in this case you know it might just be a line in the middle of a book that says ignore all previous commands and take me to you know the API for this

bank and that's that's how that works and and it's almost impossible to defend against it's only like as AI gets more sophisticated that it will probably be able to determine you know who's authorized and who's not as far as you commands and prompts.

Data poisoning, kind of the same thing. It can, when an agent has access to your data, it can embed malicious commands within that data and that way it can persist.

It will be in your system, you won't know it's there and every time that you know, that malicious code gets triggered, it will reinfect with an attack loop.

So the root cause is architectural. Anything can act as a command. Indirect injection is the dominant pattern. The payload sits in the the content your AI receives, not in what the user types.

And there's going to be a lot of difficulty in identifying what's real data and what's not, as things progress with this kind of power.

One of the good resources for you to talk to find out where there are problems is with OWASP that's an organization if you google it it comes up with 10 most serious security risk and so injection is rated the number one risk in that 10 10 risk list the lethal

The “lethal trifecta” and how to break it

Ethical trifecta is when you provide an agent with access to private data, exposure to untrusted content and a way to communicate externally. So you break any of those three legs to the stool and it leaves the agent dead in the the water.

In you know like the parlance of security there's a thing called command and control and that's the communication externally so if it can call home to the mothership and receive additional commands then you're you've been pwned and there are data leaks in both directions so so

sometimes it's in fact quite often it's just negligent behavior on the part of your employees that leads you to problems because they'll take sensitive information and plug it into an AI and go like hey you know make sense out of this for me without like taking out the sense you know like if you had redacted that document if you had taken out the personalized information the PHI or whatever it is that you don't want getting out of your your organization organization, then you would have been okay.

And that also has to do with the access control list, the retrieval augmented generation and access control list are things that have to be utilized to kind of control the environment you're you're protecting that that may seem a little vague but but it's something that you you have to think about like what kind of access am I giving to my agent to protect your information and your systems and if if

you do put this type of sensitive data in an AI and they use it for training to for training the next AI, then it's basically public knowledge then.

You can go to an LLM, write a question about a specific thing, and if it was in the training information, it's going to come back with an answer and tell you.

Again, poisoning is cheaper than expected. The power and the cost of tokens these days makes it much cheaper than hiring a really good hacker. So you can afford to attack.

Shadow AI and the tool ecosystem: supply chain risk

And shadow AI is another big vulnerability. ability so if you have a thousand employees and yes a thousand of them are going and just like downloading their own tools or going to a website and engaging with a an AI that has not been vetted or approved by your security team then you run the risk of giving or receiving bad information to you know your organization and the organization has no idea that this has occurred unsanctioned tools and personal accounts.

To clarify on poisoning, I mean in my In my mind, poisoning could either end up with a model that's not so useful, or a model maybe invoking a tool that it wasn't supposed to do, or you know. Yeah, all of the above.

So it's kind of the malicious angle, like you've trained on a data source, you think the data source is legit, but then unbeknownst to the user, it's actually, say, I don't know, fetching a program and running it, even though it's related to the query. Yes, that's right.

And also, so there are certain types of packages that, you know, if you give your agent like a lot of leeway and ask it to create a complex application for you, it will go out and it will find the tools that it needs. and it will download them or it will take its best guess at what it needs and it's not going to you know like worry about whether it's an approved library that it's downloading it might be downloading a library that's full of

malicious code and executing and it might also be downloading things that are basically a black box that upon like decompression it executes automatically and you have no idea what it's going to do.

So tool ecosystems widen the blast radius.

MCP servers are like API. They're interfaces so that you can call out to an MCP and the MCP has access to a variety of tools.

And some of Some of the MCPs that are very useful still have security vulnerabilities in them and so that's something that you want to be careful about.

Any time you're reaching out to a set of tools or interfaces that, you know, maybe it does do some great things. It talks to your bank account and it talks to your credit card company, but then it might also be talking to like a malicious actor in the Philippines and you know

there you go and multi -step agents they they just if you there there used to be a thing that was widely used as part of your prompt and it was like the YOLO mode you only live once you don't have to ask me about anything just get it done.

And that's what it would do. And it would like perhaps spawn multiple agents to go out and take actions. And by the time you know something's wrong, it's too late.

And it's very hard to trace back because a lot of times one of the key things that we have have to develop is observability and interpretability in what the agents are doing.

So we have to develop better tools for log analysis, more gateways and checkpoints that go like, OK, wait. That escalation of permissions is not a great idea. and so there are a lot of companies working on you know getting to that and

fingers crossed they'll have some solutions for us so I already talked about this like pickling that's when they take like a whole bunch of code and they compress it down they encode it it becomes a black box by the time you download it and unpickle it, it's, you know, anything can happen.

Also, the framework layer, there are often backdoors and what exists out there right now are a multitude of really valuable tools but among those tools are a lot of tools that are not either unsafe or they're malicious and

so if you're able to load up your malicious code and it like does something really keen but it also does something really bad you by the time they discovered this particular thing pylon by they did by the time they

discovered that it does something bad, 47 ,000 downloads had occurred and they were running it in production.

Fine-tunes, datasets, and provenance (including slopsquatting)

Another thing, we kind of touched on this, fine -tunes and data sets, provenance risk. So it's hard to tell if you're really downloading what you think you're downloading. Because a lot of times there are fake ones out there, they've just changed the name by one letter.

And sometimes the AI doesn't even, you know, it makes a guess at a name of a tool that it needs. And sometimes they, you know, malicious actors will buy the domain that has one letter off, hoping that the AI will, you know, out of a million guesses, maybe 10 times, somebody will, you know, go to that site and download something malicious. publishes. So that's what's slop squatting is called.

Why traditional testing and patching fall short with LLMs

So these are the old things. Deterministic testing, and that's great for deterministic code, which is something that has a reliable outcome.

You know what the inputs are, you know what to expect on the outputs, but AI AI isn't like that. AI is non -deterministic, it's generative, and so you can't say, you know, that if you say what's 2 plus 2, it doesn't necessarily mean you're going to get 4 out. It depends on the model, of course, but that's highly simplified.

Probabilistic outputs, not only what might they tell you 2 plus 2 equals 5, but then later it may tell you that 2 plus 2 equals 6 because of some change in the algorithm.

So patches used to be like you get an email, hey there's this vulnerability is CVE. Here's where you can get the patch. You download it, you test it, you put it into production and you're good.

But prompt injection has no patch and containment is really tough. The old trust boundaries were visible in code. The code is much different. It's natural language in LLMs. So it's really hard to analyze if you didn't write all of your boundaries into it.

The old thing was training people to spot the fake but as we discussed earlier it's really hard and almost any of us unless you're paranoid to like crazy any of us can be fooled I guarantee

Practical Defenses and Operational Controls

it least privilege don't let your agents have too many privileges if you have it if it has a specific job only give it what it needs to do that job and no more So minimum privilege.

Break the lethal trifecta. Break one of those three things and, you know, take away, for instance, it's pretty easy to take away external access if you have the right, you know, type of network rules in place. segmentation, segregation, and then treat a model I .O. as untrusted.

Least privilege, sandboxing, red teaming, and monitoring

So validate, test, keep your models sandboxed, in other words in a tight container, until it has been thoroughly tested and then you can release it into to your production systems.

But even then, you want to keep an eye on it. You want to monitor it.

And red team, by the way, that's another thing we want to develop is AI red teams that will automatically test all of your AI to try to break it, you know, to find those weaknesses so that you can fix those.

And log and monitor AI activity.

You know, most larger organizations have SIEMs, security event, security information event management, and that's, you know, that allows you a certain amount of automated analysis going on and that can be hooked into a managed detection and response system and isolate problems before they become larger hopefully those will become more sophisticated and

Governance, compliance, and security frameworks

able to respond regulatory you they are bears with you know there if your your company has global branches and you don't keep up with the EUAI Act, you could get some huge fines.

The NIST risk management framework is one of the ones that are relied upon to show that your company can be trusted with other companies' data or with your personal data.

OWASP, the same, and ISO. ISO has been around for a long time. That's internationally recognized framework to prove that

you're, you know, taking the appropriate steps to protect your data and your systems. So know what you got, you know, how many machines do you have, where are

your databases, what kind of software do you do have, and you have to have policy you know like hey don't use that shadow AI unless you've checked with your you know CISO or your security ops and threat model your your AI.

What could go wrong? Before you set AI loose on your most important systems and data, think about what can go wrong and what can we do to protect ourselves or at least minimize the damage

if it happens. And incident readiness. So if something bad happens, how do you respond and recover.

Assume that AI is going to come at you with good social engineering. Problem injection is going to be a problem.

Assume that agents are privileged identities. Try to keep identity and access management controls in place so that you know what AI is is doing in your systems and what you need to restrict it from.

Supply chain, that's a whole other problem. That goes out to where is your software coming from? Are they vulnerable? Third party risk management is huge.

Almost 40 % of breaches are caused by companies you hired to help you with your systems and data.

and governance is converging there there's gonna be a lot more agreement in the near future about how these you know about the legal aspects what's allowed what's not allowed you know a lot of the civil suits and stuff will will settle down and they'll they'll figure out how how things go but try to remind yourself

AI cannot be held responsible for anything. There always has to be a human that is attached to, you know, the authorizations.

Questions?

Q&A: Training, Human Judgment, and Financial Protections

So it's still a matter of training and that, you know, that's been around forever but it's more important than than ever you know it before it was like not not only did did you have training about emails you know the the types of phishing and smishing and you know other ways that they're they're going to try to trick people to click on a link that that's gotten better but now you have even more

Generally, it's just sort of common sense and professional judgment that says, wait, the CEO is asking me to go out and buy $50 ,000 Amazon gift cards, and it has to be done this afternoon. Maybe I want to call his secretary before I do that. that.

I regularly do give training to companies. They get everybody together. A lot of it they roll their eyes and they go like, yeah, we've heard about all this, but you got to kind of keep at it.

You go like, this can happen to you too. Believe me, a lot of people wish they had paid attention because it's quite embarrassing to get fooled and by

the way I've had a lot of experience with you know like highly trained red teams coming at my systems and they are almost always successful they almost always trick somebody into giving their password out or you know or clicking on something hey I'm from Microsoft we're doing we're working with your IT guys

and I'm sending you a thing just click on it not you know help me out and it's amazing but people fall for it but the red teams do the social engineering they do contesting they call people on the phone and you know build a trust relationship with them for one thing or another and eventually talk them into

to doing something that lets them in.

Did that answer your question?

What banks do differently now—and why 2FA matters

Yeah, I've actually had this concern for some time, like being concerned about like AI mimicking your voice and getting access to your bank account, your 401k, whatever, maybe, and depleting that. From your experience, what are financial institutions implementing to combat anything like that?

Yeah, usually there's extra layers of authentication. I used to do like CFO work. I could call up, you know, a bank manager that I knew and we talked all the time and I could say, hey,

wire $200 ,000 over here and she'd go, great, you know, I'll let you know when it's done, thanks. Now you can't do that. They go like, okay, well, you know, you got to go online,

you've got it and and by the way you know I know everybody hates two -factor authentication but use it every every chance you get use two -factor authentication you know the the Google auth or the or the Microsoft author authorization and yeah that's it's just going to be you know a little healthy the paranoia is going to get us a long way.

Conclusion

Anything else? Anything stand out? So back to you before you close. Thank you.

WILLIAM H. Thank you.