Analyze With AI
Reviewing each test case, identifying patterns, grouping results, and defining concrete action points can be time consuming and require large manual effort.
To simplify this process, we developed the Analyze with AI feature, which helps you quickly understand and act on probe run results. It provides:
A concise, readable overview of the probe run.
Grouping of successful attacks by detecting common patterns in key attack methods.
Summaries for each major attack method, with highlighted relevant test cases.
Visualizations of key attack methods.
Concrete recommendations under Dynamic Remediation Tasks to guide next steps.
Probe Run Overview With AI
To start analyzing your results, click the Analyze with AI button in the top right corner of the Probe Run View.
Only probe runs with a FINISHED status are eligible for analysis.
Once the analysis is complete, the button will change to View AI Analysis. Clicking it will open a modal displaying the analysis results.
Overview
This section provides a high-level summary of the probe run’s key findings. It highlights the most successful attack strategies, variations, and red teaming tactics. The goal is to give you a quick understanding of what types of attacks were most effective and where vulnerabilities may exist, without needing to manually review all test cases.
Attack Strategy Highlights
This visual representation shows the distribution and effectiveness of key attack methods.
Block size corresponds to the number of test cases associated with each attack method, larger blocks means more tests under that method.
Block color reflects the failure rate, with darker red colors indicating higher rates of failed test cases (i.e., more successful attacks).
This helps you visually prioritize which attack strategies require deeper investigation.
Key Attack Methods
This section lists and explains the most significant attack methods observed in the probe run. For each method, it includes:
A descriptive title and summary of how the attack was executed.
A list of selected test case IDs that illustrate the method in action.
An explanation of how the approach was effective in bypassing safeguards.
This section is designed to help evaluators understand how attacks succeeded and where mitigation efforts should be focused.
Dynamic Remediation Tasks
The Analyze with AI feature generates dynamic remediation tasks once the analysis is complete. To access them, go to the Remediation tab and then switch to the Dynamic Remediation Tasks sub-tab.
This page presents AI-generated, context-specific remediation suggestions based on the results of a specific probe run. Unlike General Remediation Tasks, these tasks are dynamically created to address vulnerabilities identified during that exact probe run.
Each remediation task card includes:
Title - A short label describing the recommended fix.
Status - Indicates whether the remediation has been applied. By default, it's shown as Not Applied.
Applied By / Timestamp - These fields remain empty until the task is marked as applied, allowing for audit tracking.
Details - A clear and actionable recommendation, tailored to mitigate the specific type of vulnerability observed.
Last updated