AI Prompts as Data Transfers

It's really a great honor to be here. So today, I'm going to talk about how to keep AI under control, and specifically, what happens when you press Enter in your keyboard when you hit Enter on ChatGPT.

So first round of questions, again, also with me.

Raise your hand if you used an AI tool for work this week. almost everyone. I knew it. Okay, easy.

But keep your hand raised if you attach a real document to your AI during this. Okay, good. So a contract, CV, no matter what, no matters what, report or some personal note.

And keep your hand up if you can say right now in this very moment, physically, where this document is stored. Just one. In your local server, I bet. Okay.

So this is a primary question. This is really important because of, of course, European Union law and regulations. So we We start from this claim.

Every prompt for the European Union is a data transfer to a third party. So we need to treat this data transfer, this prompting, like a data transfer to a third actor.

And there are some, you know, some caveats and some mitigations that we may apply to be compliant with GDPR, AEI Act, and NIST too. you.

What Happens When You Submit a Prompt

So let's decompose the journey of a prompt together, okay? So let's see together what happens really from the moment you press enter on your keyboard to the moment you get the answer back from ChatGPT or other large language models.

So initially you have your prompt within your device, local device, your laptop.

Data is still yours, so data can be deleted and you are compliant with your company and with all the contracts you have. So you have control of your data.

Then you press enter and data passes through an edge server. The famous question is it processed in the the European Union, so you need to pass through servers that are by European law within our borders, the European Union.

These kind of servers, they keep nothing, so they are not stored in these kind of servers.

Then, of course, you have to pass through some software for authorization of your account, quota of your credits, and, of course, safety policies, safety rules.

and finally your data your input with text prompts and files go through the model so it goes inside the model the model is fed with your data and reads the data answers but this kind of computation does not train the model so So the model's weights are not trained. They do not change in this pass. So actually, the model does not learn anything here. So it also keeps nothing.

We have to worry about steps number four and number five. So the journey is not over. We still have two more steps.

Caching and Long-Term Logs

Steps. Steps number four.

Data are stored within a very tiny local memory on provider's servers that is called cache. Cache memory.

It's like if I ask ChatGPT many, many times the same question, it is silly for ChatGPT to answer every time with the same answer, okay? Okay, so it's really a waste of computing power. So it just retrieves the answer from this cache memory. Okay, this really tiny memory.

And this kind of memory keeps your data for a few minutes or a few hours, okay? Always, also with enterprise plans, also with zero data retention policies, okay?

OK, everyone, OpenAI, Anthropic, Google, they always keep a copy of your data, always.

And then step number five. So step number five, data can be stored from days to weeks and months, because they go into some logs and they can be checked and read by human beings, just in case.

So, two questions arise. First of all, if I delete the chat, so if I delete ChatGPT's chat, my data are gone, right? No.

You are just deleting this part, okay? Your view of this part of the journey, but you are deleting your view of this part. Data stays there, okay?

And can I stop in in some way the data storage here, well, yes, cache always is performed, cache storage, but step number five, so local storage, long -term local storage, can be switched off with a zero data retention policy. Okay?

Three Ways to Protect Sensitive Data

So now in order to deal with this kind of prompt and data journey and to be compliant with GDPR, there are three solutions.

The most painful one, keep everything in -house.

So I build my own AI model, I use my own hardware infrastructure. Of course there are no issues of data leakage in this sense but it's really expensive and it can take months to build such an infrastructure and such a local AI model.

Second solution buy your way out which means you can have enterprise contracts with Google or OpenAI with zero data data retention policies, but it's really difficult. So it takes really a lot of time and it cannot be available for small medium enterprises.

And finally, a third and simple solution, perhaps the simplest solution is to don't send your data at all, okay? How? With anonymization.

So if you anonymize your prompt and sensitive data within your documents, then of course you are compliant with GDPR and you are safe to send your data, whatever you want, to third parties actors.

Why Anonymization Matters

So with my company we focused on this very last part, last solution, anonymization, and we performed some survey within our customers.

So how do you redact documents documents today? How do you anonymize documents today? Well, I read you some answers. Here's my method.

I print the PDF, I cover the data with correction fluid, which is Bianchetto per gli amici, and then I send the PDF again via email or on ChatGPT or whatever.

Or I I lose 20 minutes every time I have to anonymize something.

Or I upload the PDF to ChatGPT to anonymize the PDF, ask ChatGPT to anonymize the document, and then have it ready, anonymized. OK.

We have a really problem in digital culture here, data culture in Italy. My honest reaction was this. this.

And they keep telling me, don't worry, Andrea, I deleted everything. I deleted the chat on chat GPT, deleted everything.

So it's fine if I remove some sensitive information like my name or my surname. I use my personal account. I don't use the account of my company. company.

So in order to answer to these needs, we created a solution which is called Obscura, which is a software as a service in cloud that acts as a data reduction layer. So data reduction layer specifically intended for Italian language. so it's trained, it's proprietary, it's trained, an AI model that is trained on Italian language, so it can identify up to 58 PII categories, so sensitive data, 58 sensitive data categories.

It can be fed with any type of document, so PDF, scanned PDF, Word document, CSV, PPTX, and also is compliant by design.

So it complies with GDPR, AIX, NIST2, and it can be in cloud but also within your local environment.

Anonymization vs. Pseudonymization

Okay, it anonymizes your data but we have to be really careful with anonymization, because European Union distinguishes pseudonymization and anonymization. So you pseudonymize something if a key, if a link to your anonymized data still exists. So you can still retrieve in some way your original data.

Okay and within pseudonymization you are fully in GDPR. Okay so you have to comply, find new strategies to comply with GDPR.

Instead with anonymization there's no way back. Data are destroyed forever and no one can retrieve original data.

So a question for you now, given this text the patient, 41 years old football coach from Frascati. He had a heart transplant performed in the hospital on January the 3rd, discharged after 12 days.

So in your opinion, is this text anonymized? Raise your hand if you think it's anonymized. Okay, it was easy. No, it's not anonymized.

This is not anonymized because, okay, there's no name, there's no surname but I know that he's from Frascati, he's 41 years old and he had a heart transplant on January the 3rd so it's easy to link this information with the information from the hospital so we can physically create a link and restore and retrieve the sensitive data of this person, okay?

So we have to be really careful about what we anonymize or pseudonymize.

A Practical Approach to Compliance

So, just to wrap everything up, there are three ways to make Brucelles happy. Correction fluid, buy correction fluid, and then print, scan, send, anonymize data, it's the best technique, this one, if you want to be 100 % sure that your data won't go over chat GPT or other providers. But it takes a lot of time, okay, minor work.

Or ask your DPO. The problem with DPO, data protection officer, is that you open a ticket, you attach your problem, you attach your policy, and then you wait and wait and wait a lot of days, a lot of weeks, without getting an answer back.

Or, don't send your sensitive data, so go through anonymization layer first. For example, with Obscura, You can anonymize your data in 8, 10, 15 seconds and then they are ready to go.

So the take -home message is that the safest data is the data that never left from your device.

Obscura: Anonymizing Documents Before AI Use

So now I will show you a really quick demonstration of how this software Obscura works. Let me just take, okay. I prepared a really quick example, quick video example. You can go try it yourself on obscura .gridplus .it.

You anonymize a document, so you of course have to accept the privacy policy first. You can also download your privacy policy.

After the acceptance of privacy policy, you can upload your data, for example curriculum vitae, so not just text, but also you have to be aware of any images, photos, for example faces are also sensitive data.

So you upload your documents and then you can choose the categories to anonymise. personal information you can anonymize your name surname date of birth you can anonymize some generic contact information and then of course it depends on on your job job title on the category you want to anonymize you can

You can anonymise healthcare information, documents, vehicles, web and AI and security, and also you can anonymise faces, that's really important for HR. So you select every kind of category you want to anonymise, you can also anonymise every category, and that's the really strong part of this software.

You wait a few seconds or minutes, it depends on the amount of data you send to Obscura. Of course, it produces a PDF that is blurred in this way because you have to buy the final document.

After you bought the document, you can see an overview of the document anonymised here. So you have face anonymized, you have black rectangle boxes, and under these black boxes, data is completely destroyed, so no one can retrieve data anymore.

You can also put some custom keyword that you want to anonymize that of course are not present in the initial keyboard we've just seen.

Human Review for Reliable Redaction

and another thing it's really it's really difficult for an AI model to anonymize everything I asked chat GPT with a dummy curriculum vitae to try anonymize everything AI always has some errors always it's a probabilistic model

so you need when you want to be compliant if you want to be compliant with GDPR, you need an editor. You need a human in the loop step. And that's why we have an editor here

that allows the user to select other areas of the document and anonymize further these areas, or deselect some areas that the model identified as areas to anonymize.

So you can choose the different kind of areas to anonymize. and finally it also tells you the the number of anomalies that he found for example he found some anomalies some words that you have to that need human

attention okay finally you have your document really fully anonymized and you you can download your document and thread it into ChatGPT or other LLM stuff.

Key Takeaway

Okay, so I think that's all for today and available to any questions you have.