Introduction

So, thanks for joining us.

Notes on Previous Discussion

So, before I get into this topic, just some notes on the previous discussion.

Limits of Today’s AI vs. Human Intelligence

For sure, the current generation of AI is not at all intelligent enough to compare to human intelligence.

Perspectives from Leading Researchers

There's lots and lots of good work which has been done in this field. And very recently, you would have heard the chief AI officer of Meta, Dr. Jan Lekun, resigned.

and he is one of the firm believer that the current generation of language models, multi -model models, transformer architecture cannot lead us to human level of intelligence being acquired or achieved by anything non -human.

So, there is lot of good literature available, please do read up. Lot of them are very easy for us to assimilate.

Then Dr. Jeffrey Hinton has lot of talk which is recorded at UFTE at Vector which talks about how biological intelligence is different from artificial intelligence, for sure as

Chris was mentioning, it appears to us that it is intelligent because it is giving us lot of fluff and that makes us believe that this system which is responding to our questions is giving us something which a human would give

but when good parallel which would would make us understand.

Why Animals Still Surpass LLMs in Perception

Even an animal, a cat and a dog, a pet at home, they would have more intelligence than our language model because they will perceive the word around them with different sensory inputs. It's not only language.

They will listen to our command. They will respond. They may not be able to speak our language, but they will respond by action, by showing their affection or some reaction.

But they have visual inputs. inputs, they have audio inputs and then they have sensory inputs and similarly a human baby as he or she grows up, they assimilate the word around them in a different manner.

So there's a lot of research which is happening and these are some luminaries of current generation of AI.

World Models and New Research Directions

So Dr. Fifi Lee who is the godmother of AI, she has a new lab which is called the Word lab which is doing this research to create word models which help us

understand word around us physical word around us much better and that could perhaps be one step forward towards attaining whatever you want to call it super intelligence AGI and but with the current architecture and

current way of training it is far away so again like researchers like Dr. Dr. Yann LeCun would say it's not transformers, it is something like JEPA, he calls it joint embedding.

Yes.

From Intelligence Debates to Practical Risks

So, now we'll talk about something else.

Hallucinations and Probabilistic Behavior

I know this may look and sound very technical, but that's not the point. The point is related to the second question which was asked about hallucinations. Yes, LLMs suffer from hallucinations because they are not deterministic.

They never give 2 plus 2, 4. That's what the traditional program or to an extent traditional machine learning models would give.

Large language models are probabilistic in nature. Their only mission in life is to predict the next best token.

Training Cutoffs and Internet Connectivity

If they were trained on certain set of data, so the latest model which was released which is Gemini 2 .3 has been trained till January 2025. So all the data before that would be updated.

If you ask a question which happened afterwards, let's say who won Super Bowl or who won some championship, they may differ in the output, but yes, now they are connected to internet so obviously the model can draw some inspiration and input from internet and give you the answer.

Trust but Verify: User Responsibility

So as it was brought out earlier also, if you can't trust me with all the things which I say, please don't trust technology with all the things which come out of it.

please do your own checks and balances that you always did right you always and slowly and steadily when we have the trust you will start believing and trusting so towards that there are different

Mitigation Approaches: Fine-tuning and RAG

techniques as it was brought out by previous speaker something called fine tuning where you kind of almost trained a new language model or a small language model to your own data very expensive very compute heavy needs a lot of engineering effort another option is something

something called RAG, Retrieval Augmented Generation, which allows the LLM to reduce the hallucination and get grounded into what you say.

Cybersecurity in the Age of Generative AI

And today we are going to talk about some of the issues which are coming up in terms of cyber security in usage of generative AI.

Has anybody heard about the latest cloud code hacking which was orchestrated using AI? Anybody who would like to talk about it, what happened last to last week? Anyone?

Recent Incident: Coding Assistant Compromise

So Cloud Code, which is a coding assistant, Gen -AI coding assistant from Anthropic, very good tool, a lot of us use it for coding. How many of you use coding assistant for coding? So it makes life easy, right?

So some state -sponsored hacking company somewhere in other part of the world was able to prompt prompt and ask Cloud Code certain questions, which opened certain doors of vulnerability and the Cloud Code understood it as if it is a red teaming exercise going on that defensive cyber security agents are doing some testing.

And it was able to be breached into and almost 30 enterprise accounts, most of them government accounts in the US were kind of penetrated into. Subsequently Anthropic reported this.

Dual-Use Technologies and Rising Threats

So why I want to bring this to your notice is that these techniques are available to all of us to change our trajectory of life, our work but unfortunately the same techniques are also available for bad actors to misuse.

That's why you will find a significant rise in cyber attacks and

RAG: Strengths and Vulnerabilities

Grounding Answers with Context

So, over here we are trying to present a particular research which is bringing out, though Rag is fantastic to bring more grounding to the answers which LLM would give because what essentially Rag does is Rag is bringing context from your documents following certain process and instead of giving hallucinated answers to you which are very global in nature, it It gets grounded in your answers.

How RAG Can Be Abused

But that being said, research has proven to showcase that there are some lapses which can be brought in through RAG, which can be misused. And the idea of bringing this to your notice is not to kind of scare you, but to tell you that for everything good, there are some vulnerabilities which you should keep in consideration.

A Safety Analogy: Seatbelts for Autonomy

It's like driving on the highway. Autonomous driving does a wonderful job, especially when you're driving tired at night. I use it, I've been using it for the last four years, very safe, sometimes I may miss few things walking on the road but it does not, but if I don't wear a seatbelt on top of it then it is like something which is unpardonable offense on my part, autonomous driving cannot do about it, anything about it.

So something like that, these are the issues which you should keep in mind and why I want to bring to your attention, there are lot of young professionals, lot of senior leaders in the room here.

Operational Guidance for Leaders and Teams

As you commission these systems into your organizations or you use for your personal use, try to check these things which you otherwise would check in terms of what are the cyber security or governance or alignment or safety requirements which these systems have. Just because it is available readily or it is available at no cost, it may be that it has certain issues which are not being highlighted to you by whosoever has created it.

Attack Vectors: Corpus Poisoning and Supply Chain Risks

So what essentially RAC does, RAC helps us bring in a lot of relevance and the corpus of data which is being inducted that can be kind of misused by the bad actors to do something called corpus poisoning and those malicious documents will not be used immediately they are hidden

Real-World Example: Wallet Drain via Code Injection

they can be utilized at a point and time of choosing by the bad actor example one gentleman's Because one gentleman was wipecoding, how many of you wipecode, lovable, firebase studio, some wonderful tools available now, Gemini 3 does a great job.

So that gentleman was wipecoding three months back using one of these tools and as all of us do on a personal computer had his crypto wallet password very safely kept in one of of the notes on the computer, on the laptop.

So through the wipe coding, the code which was brought in from internet was utilized by a hacker to read his cryptocurrency, crypto wallet password.

And that person lost almost 300 ,000 worth of cryptocurrency immediately because there was a leakage of password and it could be operated.

Similarly, if you have this rack pipeline, line.

Timed Triggers and Espionage Scenarios

So bad actor can actually keep the malicious code in your system. And later on, let's say tomorrow you're going for your IPO, or tomorrow there's a big release which is happening in your company.

At that time, it can be triggered. It could be for corporate espionage. It could

be for blackmailing. It could be for anything which somebody wants to achieve. It could be for ransom.

Due Diligence When Adopting Third-Party Systems

So why it is important to actually put these things in perspective as you build Build these models either on your own, in your company, in your own teams, or you bring a third party provider that you check on these kind of things.

From Post-Deployment Tests to Secure-by-Design

The traditional way what we were doing, we were actually testing everything later on. It was all post deployment that we'll check if it is trustworthy or not.

Governance, Security, and Trust by Design

What I would like to bring to your attention is these three things by design that you bring You bring governance by design, you bring security by design, and you bring trust by design or safety by design that everything has to be done by design before you actually commission these systems.

Just because there is hype, just because your CEO has declared in the earning call that we are going agentic, you as the developer or you as the business leader should actually think about these important aspects because if you miss them, they can later on trigger trigger a lot of problems which cannot be compensated by whatever productivity gain we got from whatever tools we were using.

Provenance and Cryptographic Traceability for RAG

So essentially what we're trying to introduce, we're trying to introduce a way that we should have some protocols which can use a cryptographic code for each segment which you are putting for RAG so that you can trace back where it's coming from.

something what has been traditionally being done for any kind of cryptography which we have used earlier in in kind of business transactions financial industries have used it communication industry has used it it's all codified so you can trace it back perhaps i read an article somewhere

people were saying that right now ai is very vulnerable in terms of security attacks so blockchain may be the answer i know i'm not sure i am not an expert in blockchain but there are a

lot of researchers who are saying that if we can actually supplement a lot of transaction which happens between the models and the tools especially for genetic systems and have it mapped with blockchain that could be the transformation to map everything and then later on take care of all these vulnerabilities.

And essentially what it helps us do is actually it makes that all chunk, so essentially chunking

Chunking with Keys and Index Traceback

Feeding is done when you are doing a rack and you allocate certain codes to each piece of information and it could be that let's say if I am actually feeding the entire annual marketing report of my company which is already a public document to this particular rack system so that the answers come relevant to my company so that I can respond to emails which have come from our clients.

lines, I upload that 100 page document, each portion of it, maybe a para or maybe equivalent of a para gets assigned certain code. When I say code, it would be essentially a key, a cryptographic key, and then later on you can actually trace it back as you build an index out of it and it goes, so you can trace it back how it was brought out, and in case there is any malicious code in between, that can be pinpointed.

Proof-Carrying Answers (PCA)

So the idea is to actually do identification and re -identification of source of information. And the idea would be that when it goes back, you have something called PCA, proof -carrying answer, which allows you to trace it back.

Keeping It Practical

But this may sound very technical. I don't want to bog you down with technology.

My idea is to actually urge all of you, in your respective field of work, you have had certain techniques like this which you have used, used, which have made the whole process more trustworthy, more secure.

Beyond the Demo: Making AI Deployable

So it is important when you are actually commissioning or building such projects on AI, GenAI, or you're using APIs from any kind of public AI provider, try to see how you can foolproof it beyond that particular demo. Because in demo everything works very well. well.

Why POCs Succeed but Projects Fail

We know that almost 79 % of AI projects fail because they work very well in proof of concept and the rest of the time we're not able to scale it.

Security, Governance, and Safety as Business Risks

And security, governance and safety are one of the most significant aspects of it, though nobody is right now flagging them because obviously it's not good for the positioning and marketing.

But as As users, individual as well as institutional, it is important for us to keep this in mind because this could be a problem in future.

So all the good things which we are aiming to achieve from AI may get kind of a little backtracked because of these issues.

Limitations of the Proposed Approach

Definitely there is a huge limitation. This technique itself does not cover everything. It will only be able to tell us anything which was tempered post -commitment, essentially

what was there part of the set. it. If the information, something which was brought out earlier, information itself was corrupted, then obviously it will not be able to trace it back.

Because if I was, I trained a system on something which was corrupted, and that is what is happening when you are using data from internet and code from internet, which nobody has any verification tool about.

So it's not only about the veracity of the information, how truthful it is, how grounded it is, but also how there may be a torsion horse or there may be ransomware hidden somewhere. It may pass the muster.

Pre-ingestion Scanning and Residual Gaps

So perhaps it could be that before you take the data in, you run through a whole scan and make sure it is free of any kind of security risk. And how generally it could respond to it could essentially help us either abstain if anything

which is already tempered with, and if in case it is structured, it again will get blocked. But yes, as I mentioned, it may not be able to block anything which is pre -committed when it was already built in the system.

Expected Coverage and Defense in Depth

And we are able to just pull it because it was kind of embedded in the code. So we are hopeful that it will be able to stop almost 58 % to 60 % of attacks.

But the remaining 40 % of attacks will still go through, and we'll require another layer. So that is what we would like to bring to your attention,

Defense in Depth for Regulated and High-Stakes Environments

that there is a requirement to have something which will call defense in depth which as it has been the case in most of the regulated industry whenever they have used any technology so all the banks utilize this concept very well and I think all the pharma companies and healthcare companies also use this that could be an approach which we should take for our own usage as well and again

Again, leaving the numbers aside, what is important is that we bring a certain amount of understanding of different techniques which are available to make sure the system is very much aligned and safe and secure for what we are trying to use it for.

Establishing Provenance and Outer Security Layers

And this is something which it would look like that you have essentially cryptographic or any other way of establishing provenance. essentially is the lineage where the data is coming from, where the source is coming

from and then later on you have some way of connecting it to the outer layer which could have a lot of other security techniques which could help you make things solid.

Why Critical Infrastructure Is a Target

It may be that for personal use you may not, nobody perhaps may break into my system because I'm just using it for my homework or my assignment but it may be that if a large organization or a bank or a national security organization is using such a system, it can be broken into and it can cause a lot of problems.

If you would remember, the previous hacking attacks have been not on banks and hospitals, they have been on infra organizations, water authority, electricity distribution authority.

Innovation with Responsibility

And they are very prone to these kind of attacks and that's why it is important for us, and I'm sure a lot of you may be working for such organizations, it's important to bring these aspects to four not to again not at all to stall the process of innovation but

at the same time build with a lot of responsibility and a lot of security and safety and alignment in mind because if it is not done today it can cause

Conclusion and Q&A

issues in future and with this I'll just pause and if there are any questions please feel free to ask and this is my LinkedIn if in case anybody has ask questions you can follow it up later on as well