AI Accuracy and Trust in SAFE Whitepaper
Outline
Introduction
SAFE uses generative AI selectively and within defined boundaries. Cyber-risk scores, report metrics, likelihood, loss magnitude, and other quantitative risk values are produced by SAFE's deterministic scoring and data services. Generative AI does not calculate or alter these values.
Where generative AI is used, its role is to explain results, produce narrative summaries, classify information, review documents, and assist with analysis based on data supplied by SAFE's governed services.
Executive Summary
1. Risk scores and report metrics are deterministic
SAFE's generative AI does not calculate cyber-risk scores, loss values, or report metrics. These values are produced by SAFE's deterministic scoring and data services. AI is used only to explain, summarize, classify, and assist with analysis where applicable.
2. SafeX respects existing data-access boundaries
SafeX operates within the same tenant-scoped and permission-controlled environment as the authenticated user. It cannot access unrestricted customer data or bypass SAFE's authorization controls.
3. Hallucinations and incorrect relationships are measured
SAFE evaluates SafeX responses for supported, unsupported, and contradicted factual claims against underlying SAFE data. It also checks whether stated relationships are substantiated — for example, whether a finding is actually associated with a specific asset, rather than merely confirming that both records exist. These checks are largely asynchronous quality evaluations, not a guarantee that every claim is validated before the response is displayed.
4. AI responses are traceable and auditable
SAFE can investigate how an AI response was produced by reviewing the retrieved data, API and tool activity, model inputs and outputs, and supporting references. For document-based use cases, answers are grounded in extracted source content and include page-level references so reviewers can trace them back to the original document.
Data Access and Grounding
SafeX operates within SAFE's existing tenant-isolation and authorization model. It retrieves information through the same tenant-scoped, permission-controlled services available to the authenticated user.
This means:
- SafeX does not receive unrestricted access to customer data.
- It cannot bypass SAFE's role-based access controls.
- Data retrieval and tool activity are performed within the user's identity and tenant context.
- References to supported SAFE entities — such as risk scenarios, third parties, findings, and assets — can be linked to the underlying platform records.
SafeX is instructed to base its answers on retrieved information, distinguish between platform data and other permitted sources, and avoid filling missing information with assumptions. However, this grounding instruction should not be interpreted as a guarantee that every factual statement is synchronously validated before the response is shown.
Runtime Guardrails
Runtime guardrails operate while a request is being processed. They govern what SafeX can access, what types of requests it can handle, and whether certain outputs can be used by downstream platform workflows.
These controls include:
- Tenant isolation and permission enforcement
- Harmful-content and misuse screening
- Prompt-injection and jailbreak detection
- Cybersecurity and risk-domain scope checks
- Detection of encoded or obfuscated instructions
- Tool authorization and input validation
- Schema enforcement for structured AI outputs
- Validation of allowed classification and response values
- Verification that referenced record identifiers exist before they are used by applicable downstream workflows
- Safe fallback behavior for low-confidence or invalid structured outputs
For selected higher-risk workflows, such as TPRM answer review, SAFE may also apply an additional model-based review before presenting the result to an analyst.
Note
These are runtime controls, but they do not mean that every narrative claim or entity relationship in every SafeX response is checked against platform data before the user sees it.
Post-Response Hallucination Evaluation
Hallucination and factual accuracy are primarily measured through asynchronous quality evaluations performed after response generation.
SAFE uses LLM-as-judge evaluations to compare factual claims in SafeX responses with the underlying SAFE data. Each evaluated claim is classified as:
- Supported: The available SAFE data substantiates the claim.
- Unsupported: The claim is not substantiated by the available data.
- Contradicted: The available SAFE data conflicts with the claim.
Unsupported and contradicted claims are treated as quality failures. Results are monitored over time and used for regression analysis, root-cause investigation, prompt and workflow improvements, and additions to evaluation datasets.
Note
These evaluations measure and monitor hallucination risk; they should not be described as controls that prevent every hallucination from reaching a user.
Incorrect Relationship Detection
SAFE evaluates relationship accuracy independently from the existence of the entities involved.
For example, if a SafeX response states that Finding X affects Asset A, it is not sufficient for the evaluation to confirm only that Finding X and Asset A both exist. The evaluation checks whether the underlying SAFE data establishes the specific relationship between that finding and that asset.
The same principle applies to other relationships, including:
- A control mitigating a particular risk scenario
- A vulnerability affecting a specific asset
- A third party being associated with a particular assessment or finding
- A threat or event contributing to a stated risk
- A domain or IP address being attributed to a specific organization
A relationship is considered supported only when the retrieved evidence substantiates the connection. Unsupported and contradicted relationships are recorded as quality failures and incorporated into regression monitoring and improvement work.
Document-Based AI
For document-processing use cases, SAFE first extracts and structures relevant source content. Native text extraction is used where possible, while scanned or structurally complex pages may require vision-based processing.
The question-answering workflow receives scoped, extracted content rather than relying on an unrestricted interpretation of the raw file. Generated answers include page-level references so reviewers can trace an answer back to the relevant location in the source document.
This allows reviewers to:
- Inspect the source content supporting an answer
- Confirm whether the response accurately reflects the document
- Identify unsupported conclusions
- Review the reasoning associated with an extracted answer
Uploaded documents are also malware-scanned before processing.
Report Numbers and Risk Values
Report numbers and cyber-risk values come from deterministic SAFE services — not from a language model.
The reporting service obtains its data from SAFE's authenticated, tenant-scoped backend services. These are the same governed data services used by the SAFE user interface.
Where generative AI contributes to a report:
- Calculations and aggregations are completed before the data is sent to the model.
- The model is used only to explain or summarize the supplied results.
- The model is instructed to refer only to information present in its input.
- The AI-generated narrative does not calculate, modify, or replace the underlying figures.
- If narrative generation fails, the report uses a visible fallback rather than inventing replacement values.
As a result, the authoritative report metrics remain the values generated by SAFE's deterministic scoring and data pipelines.
Accuracy Measurement and Regression Monitoring
SAFE measures AI accuracy separately for each use case rather than applying a single platform-wide accuracy score.
The AURA — AI Usage Reliability & Assurance — evaluation approach combines:
- Version-controlled golden-dataset testing
- End-to-end behavioral evaluations
- Human review
- LLM-as-judge monitoring
- Negative and edge-case testing
- Adversarial test scenarios
- Repeated testing for factual consistency
- Regression comparison across model, prompt, and workflow versions
Evaluations run automatically, and material score reductions generate alerts for the responsible team. Sustained quality drift can result in a tracked engineering issue for investigation.
Examples of measured use-case results include:
- Compliance-document assessment: 92.6% overall accuracy against human-validated golden data, including 96.8% for ISO 27001 and 89.9% for SOC 2
- TPRM vendor-answer flagging: improvement from 85.3% to 88.8% between evaluated versions
- Intake-extraction fields: approximately 100% accuracy with 0% measured reproducibility drift in the evaluated dataset
Note
These results are specific to the evaluated use case, dataset, model version, sample size, methodology, and evaluation date. They should not be interpreted as a single accuracy score for all SAFE AI capabilities.
Traceability and Investigation
SAFE can trace how an AI response was produced. Depending on the workflow, investigators can inspect:
- The source data retrieved
- API and tool activity
- Tenant, user, and session context
- Model inputs and outputs
- Supporting entity or document references
- Token and execution information
This traceability enables SAFE to investigate a specific response, determine what information was provided to the model, understand how the workflow behaved, and reproduce the issue when necessary.
AI telemetry is maintained in SAFE's controlled observability environment with access controls, encryption, and regional deployment considerations.
Customer-Data Governance
Under SAFE's AI policy:
- Customer data is not used to train shared or cross-tenant models.
- SAFE's LLM subprocessors do not retain customer inputs and outputs for model training.
- AI processing follows the applicable customer-data region.
- Internet search for AI agents is disabled by default and subject to administrative control.
- AI access remains within the authorization boundaries of the underlying SAFE capability.
Ongoing Improvements
SAFE is extending this assurance model through a product-wide AI inventory and an in-product view of AI quality by capability. The planned experience is intended to present measures such as accuracy, grounding, and hallucination rates together with their methodology, sample size, and evaluation date.
This is a roadmap direction rather than a currently available confidence score on every SafeX response. Today, trust is established through deterministic risk calculations, governed data access, runtime guardrails, source traceability, asynchronous quality evaluation, regression monitoring, and end-to-end auditability.
Did this article help?
Let us know by leaving a star rating at the top of this article.