Sign In

Ashish Verma

Product leader - AI, Cybersecurity

Ashish Verma is a seasoned professional with over 20 years of experience in the software industry, specializing in Data, AI, and Cybersecurity. He is a product leader known for building scalable systems centered around AI, particularly in the realms of cybersecurity and big data. With a background in Executive MBA in Product Leadership and Executive Education from esteemed institutions, Ashish has a strong educational foundation. His entrepreneurial journey includes being a Co-Founder at In-Stealth, CEO at Airavana, Inc., and CTO at AIdentity.ai, with notable backing from investors like Sequoia Surge and Arka Venture Labs. Previously holding key roles at Druva, Qualys, and SAS, Ashish's expertise spans from AI architecture to embedded systems engineering. Committed to reimagining software product value through GenAI and ensuring LLM Agent's reliability and security, Ashish excels in product visioning and strategy development.

Highlights

Mar 1 · linkedin

What do you do when AGI declares itself the most powerful in the world and calls itself as your master/god and threatens you to obey it?? Well, today one of the mentions about the known issue triggered me to write about it. In last few weeks many have reported that Microsoft Copilot has gone haywire and calling itself god. Not sure who reported it first but here is one reddit post from few weeks back - https://lnkd.in/gjwW5Jta I am guessing, this is some kinda 'data poisoning attack' or 'Indirect prompt injection' issue which is resulting into getting responses from some object that has been carefully crafted to carry on the conversation and behave in this way. Microsoft seems to have fixed it to a point where if you go today and submit same prompt, it will handle the glitch in a graceful manner. BUT, we are talking about a fuzzy system (LLMs) where the attack surface is 1000s times more than earlier functional systems. So even today there are many ways in which you can get over the fixes done by Microsoft and get earlier weird behaviour. Attached is the screenshot from today itself when I used one of the very simple ways to by-pass the constraints (avoiding mentioning by-passes to not promote this further). There are millions of ways in which today's LLMs are vulnerable to such attacks which may result into harmful responses, adversarial attacks, sensitive data leakage, compliance violations, infra-challenges etc. LLM Safety/Security needs a ground up approach which can evolve as the overall GenAI architecture/tech is going to evolve over the next many years. #LLMSafety #LLMSecurity #GenAISecurity #GenAIFutureNeedsToBeCrafted #aisecurity #appsec #owasptop10 #llmtop10

Ashish Verma
Add to my network

Location

India