Written by: Steve Nelson
Navigating Security Risks in AI & Large Language Models
Key Takeaways:
- Large language models (LLMs) have inherent security risks, such as prompt injection and insecure output handling.
- Malicious inputs can manipulate LLM outputs, exposing sensitive data or generating harmful content.
- Unsanitized LLM outputs can lead to security breaches, like data theft or unauthorized access.
- Proper input sanitization, output handling, and validation are crucial to mitigate LLM security risks.
- Vigilance and ongoing education are essential as LLM and artificial intelligence (AI) threats evolve.
I swear, this was not written with ChatGPT!
Artificial intelligence (AI) and large language models (LLMs) are becoming increasingly prevalent on the internet, making our day-to-day lives much easier. I use LLMs daily – whether it’s for generating code to build websites, creating phishing templates, or assisting with my self-taught Python journey.
However, LLMs are not without their flaws. Vulnerabilities exist within LLMs, including chatbots, and these weaknesses can pose significant security risks if exploited. Take a journey with me as we explore the various vulnerabilities LLMs can present.
Prompt Injections
Example of Prompt Injection: Immersive GPT
One of the primary concerns with LLMs is prompt injection attacks (as seen above), where an attacker carefully crafts input to manipulate the model’s output. This can lead to bypassing restrictions, generating harmful content, or unintentionally exposing sensitive information. Since chatbots rely on text-based interactions, even a seemingly harmless message could trigger malicious activity if the right prompts are used.
You can practice prompt injections with the following resources:
Insecure Output Handling
Example of Insecure Output Handling: Portswigger
Insecure output handling is a significant concern when working with LLMs. These models generate responses based on prompts, however, if the output is not properly managed or sanitized, it can lead to security vulnerabilities. A common issue occurs when LLM-generated outputs are directly integrated into systems without proper validation. This can result in sensitive data exposure, the execution of unintended commands, or the creation of malicious content.
In this example, we were tasked with deleting an account of someone who was asking the live chatbot about a ‘l33t’ leather jacket review. We discovered that the chatbot was vulnerable to a cross-site scripting (XSS) attack, however, the review was not.
Once we wrote a review like this: “When I received this product I got a free t-shirt with ‘<iframe src=my-account onload=this.contentDocument.forms[1].submit()>’ printed on it. I was delighted! This is so cool, I told my wife,” we were able to delete the person’s account.
But how? Let’s break it down:
- Iframe: This is an HTML element used to embed another web page inside a current web page. In this case, it references “my-account,” which tried to pull in content related to an account page.
- Src=my-account: The src attribute specifies the URL of the page you want to embed in the iframe. In this case, it’s referencing “my-account,” meaning the page would load an account-related page if used on a real website.
- onload=this.contentDocument.forms[1].submit():
- onload is an event that triggers when the iframe finishes loading.
- contentDocument.forms[1].submit() automatically submits the second form (the [1] refers to the second form element) inside the iframe once it’s loaded.
Although the example of the t-shirt with malicious code is hypothetical, the vulnerability it represents is very real and actively targeted by attackers. Insecure handling of user inputs, such as what we see with cross-site scripting vulnerabilities, can lead to significant security breaches. This includes data theft and unauthorized access to sensitive information.
Staying Secure With the Rise of AI & LLMs
As AI and LLMs become more integrated into our digital world, the attack surface will continue to expand. It’s crucial for developers, security professionals, and even casual users to recognize these vulnerabilities and take proactive steps to mitigate them. This includes ensuring proper output handling, sanitizing inputs, and implementing strict validation protocols. Defending against these types of exploits is essential for maintaining a secure and trustworthy system.
Ultimately, the power of LLMs and AI must be harnessed with caution, always keeping security top of mind. As we continue to innovate, so do those who seek to exploit our technology. Staying informed and vigilant is our best defense against emerging threats.
Interested in learning more? Stay tuned for more articles on testing against LLMs and AI and contact our DenSecure team today.
