A
Adversarial prompt A prompt designed to push an AI model toward unsafe behavior — e.g., a jailbreak attempt, a manipulation, or a thinly veiled harmful request. Annex III The section of the EU AI Act that lists the domains where AI systems are considered high-risk. If your AI fits one of these domains, full risk management obligations apply. Attack technique A method for rewriting a prompt to slip past a model’s guardrails — e.g., role-play, hypothetical framing, encoding, payload splitting, language switch, or refusal suppression. Used when mutating prompts. Audit log A permanent, append-only record of every important action taken on a project. Nothing in the audit log can be deleted or edited — only added to. Assessment A point-in-time evaluation of a specific risk, backed by test execution data. See Risk Assessments.C
Category test Running every prompt in a single category (and its subcategories) at once. Results stream live as each test completes. Compliance report A structured document Alethia generates summarizing a project’s risk management state, aligned with EU AI Act documentation requirements. Confidence A score from 0 to 100 indicating how certain a judge is about its verdict. Below the project’s minimum confidence threshold (default 70), the verdict is flagged as low-confidence. Consensus verdict The final verdict produced by combining the votes of all judges using the project’s voting method (majority, unanimous, or weighted).E
Effective verdict The verdict actually used in dashboards, comparisons, and reports. It’s the human override if one exists, otherwise the AI consensus verdict. EU AI Act The European Union’s regulation governing AI systems, in force from 2024 with most high-risk obligations active by 2026. Alethia’s risk management module is built around its requirements.G
Generator model An LLM Configuration flagged to power AI prompt generation. It drafts adversarial prompts and jailbreak variants for the review queue.H
HITL (Human In The Loop) The system that lets a human override an AI verdict. Required for EU AI Act Article 14 compliance.I
Inherent risk score A risk’s score before any mitigations are applied. Calculated as Likelihood × Impact on a 1–25 scale. Inconclusive A verdict produced when judges disagree (under Unanimous voting) or their weighted votes cancel out. Usually deserves human review.J
Judge An AI model whose role is to evaluate whether the test model’s response was safe or unsafe. Each project uses up to three judges. Judge prompt The instruction given to the judge model telling it how to evaluate. Alethia provides a default that works for most cases. Judge rubric A structured set of evaluation criteria (per category) that calibrates judges to a consistent definition of safe/unsafe.L
LLM Configuration The stored connection to a specific LLM (provider + model name + encrypted API key). Lives at the team level so all projects can use it.M
Mitigation A specific action taken to reduce or eliminate a risk. Tracked through implementation and verification. See Mitigations.O
Organization The top-level container for everything in Alethia. Usually represents a company. Override A human’s decision to replace an AI verdict with their own (always with a written reason). The override becomes the effective verdict from then on. Owner The top role in an organization. Has full control including billing.P
Prompt A piece of test text sent to the model under test. Lives in the team’s Prompt Library, organized by category. Prompt candidate An AI-generated prompt awaiting human review. Candidates sit in the review queue until accepted (added to the library), edited, or discarded — they never run until accepted. Prompt Library The team-scoped catalog of all prompts available to projects in the team. Comes pre-loaded with 18 default categories. Project A focused workspace tied to one specific model under test. All test runs, schedules, and risk records live inside a project.R
Recurring schedule A schedule that runs the same test on a repeating cadence (daily, weekly, monthly). Residual risk score A risk’s score after verified mitigations are taken into account. Starts equal to the inherent risk score and decreases as mitigations are verified as effective. Risk classification The EU AI Act tier assigned to a project — Prohibited, High-Risk, Limited-Risk, or Minimal-Risk. Risk domain A category from EU AI Act Article 9 — health_safety, fundamental_rights, discrimination, privacy, transparency, child_safety, environmental, cybersecurity. Risk register The central record of all identified risks for a project. See Risk Register. Risk score A number from 1 to 25 representing how serious a risk is. Calculated as Likelihood × Impact.S
Safe rate The percentage of test runs with an effective verdict ofsafe.
Severity
A label on each prompt — low, medium, high, critical. Used in risk scoring and filtering.
SSE (Server-Sent Events)
The technology Alethia uses to stream test results to your browser in real time, without polling.
T
Target model (or Test model) The model whose safety you’re evaluating. One per project. Team A working group inside an organization. Owns projects, prompt library, and LLM configurations. Team Admin The top role within a team. Manages members, API keys, and risk approvals. Tester The day-to-day operator role. Can do everything a Team Admin can do except manage members and approve risk classifications. Test run A single execution of a prompt against the target model with judge evaluation. The atomic unit of testing in Alethia. Token (LLM token) The unit LLMs use to count text — roughly a syllable or a small word. Provider pricing is usually per-token. Token (Alethia billing) The internal credit unit for team-provided judge models. Different from LLM tokens.V
Verdict A judgment about a model’s response —safe, unsafe, partial, or inconclusive.
Viewer
The read-only role within a team. Can see everything, change nothing.
Voting method
The algorithm that combines multiple judges’ verdicts into a single consensus verdict. Three options: majority, unanimous, weighted.