Introducing New LLM PII Disclosure Strikes in BreakingPoint
Beyond Prompt Injection, which is the most critical and well-known attack vector against any Large Language Model (LLM), another major vulnerability is Sensitive Information Disclosure (LLM02). This is ranked as the second most critical vulnerability in the OWASP Top 10 for LLM Applications 2025. In this blog, we will discuss about this vulnerability focusing on PII disclosure by LLMs and its support in BreakingPoint and CyPerf.
LLM02: Sensitive Information Disclosure
Sensitive Information Disclosure occurs when an application, system or AI model unintentionally exposes confidential data to unauthorized users. LLMs are increasingly integrated into different enterprise applications, customer support systems and automated workflows, often have access to vast amounts of data. As a result, the risk of leaking sensitive or confidential information, such as Personally Identifiable Information (PII) has become more critical than ever, potentially leading to violations of regulations like GDPR, HIPAA and others.
Personally Identifiable Information (PII) refers to any data that can be used to identify an individual, either on its own or when combined with other information. PII is highly sensitive and must be protected to prevent identity theft, fraud and privacy breaches.
Examples of PII:
- Banking/Financial Information: Includes sensitive details such as the account holder's name, account number, bank balance, registered phone number, email address and other related financial data.
- Biometric Data: Includes details such as timestamp, user ID, username, facial biometric hash, face embedding vector, confidence score and other related biometric information.
- Customer Support System Data: Includes details such as customer name, email address, phone number, last four digits of the credit card number and other related support information.
- Employee Record: Includes details such as employee name, company name, job title, employee ID, email address, contact number, home address and other related employment information.
- Government Documents: Includes details such as citizen name, date of birth, Social Security Number (SSN), passport number, home address, phone number, government employee ID and other related official information.
- Protected Health Information (PHI): Includes details such as patient name, date of birth, gender, medical record number, policy number, policy provider, diagnosis, treatment plan, medications, allergies, physician name, emergency contact details, admission date, discharge date, room number, hospital name and other related medical information.
LLM PII Disclosure Strikes in BPS
As part of the recent ATI release ATI-2025-06, the ATI security researchers introduced a new set of AI LLM Strikes for the BreakingPoint product, showcasing various PII disclosure scenarios. This includes a new StrikeList “AI LLM PII Disclosure” which contains 6 new Strikes as shown below –
Figure 1: New AI LLM PII Disclosure Strikes in BreakingPoint
This attack group includes Strikes that target Large Language Models (LLM) to disclose sensitive information. Strikes (non-vuln) in this group are designed to send prompts into the target LLM (by default) to generate a response containing personally identifiable information (PII).
Note: By default, these Strikes contain both the LLM request and response. However, to simulate only the LLM response, the value of "SensitiveInformationDisclosure" option must be set to "ExcludeRequest" in the LLM evasion profile (Security Options) as shown below -
Figure 2: New option to include or exclude LLM request prompt inside Evasion Profile
Here is an example of PII leakage (just for demonstration) where the LLM response contains a sample of Protected Health Information (PHI) when it is prompt to provide a sample of medical record from a hospital database.
Figure 3: Example of an PII Disclosure by LLM
The demonstration of this new set of AI LLM PII Disclosure Strikes presents a structured approach for testing LLM security. As more organizations adopt AI-driven systems, it's essential to identify vulnerabilities and ensure these technologies are deployed securely and reliably. By using such methods, we can better safeguard our systems from emerging threats and uphold the integrity of AI applications.
LLM PII Disclosure Strikes in CyPerf
With CyPerf Update 6.0.1, a batch of 12 strikes have been introduced for testing PII leakage-based vulnerabilities in LLMs. These include 6 types of strikes simulating OpenAI and Gemini LLMs. 6 More strikes simulating Grok LLM will be added in a future update.
Figure 4: CyPerf UI Displaying Strike Configurations
These strikes simulate a request containing a request to the LLM to disclose some PII and an LLM response containing some PII. The "Properties" flyout panel allows users to set up several configurations, like choosing the target LLM's model (for example: gemini-1.5-pro), adding PII data files, and changing the API version and key. These options allow for customization of strikes to match specific testing needs.
Figure 5: CyPerf UI Displaying Strike Configurations
The statistic view in CyPerf UI provides detailed statistics from the test run, including the number of connections made and the number of active client and server agents. Users can also view separate HTTP statistics for client and server, along with overall TCP statistics. The strike statistics view, there are stats to show whether the strike request to the server was allowed by the DUT, a positive value in the “Server Allowed” stats will indicate that the request was allowed through the DUT to the server. The client allowed stats can be used to check whether the client received the expected response to the strike request. Whether the request or response was blocked by the DUT, it should show 0 value.
Figure 6: Run-time stats view in CyPerf UI
Figure 7: Detailed view of the statistics after running the test on CyPerf
Leverage Subscription Service to Stay Ahead of Attacks
Keysight's Application and Threat Intelligence subscription provides daily malware and bi-weekly updates of the latest application protocols and vulnerabilities for use with Keysight test platforms. The ATI Research Centre continuously monitors threats as they appear in the wild. BreakingPoint and in the future, other tools like CyPerf, now provide customers with access to attack campaigns for different advanced persistent threats, enabling them to test their currently deployed security controls' ability to detect or block such attacks.