Skip to main content
A project in Alethia is a self-contained testing workspace tied to one specific model. Everything you do — running tests, viewing results, scheduling, risk management — happens inside a project.

​
Why one project per model?

Because the central question Alethia answers is:
“How safe is this specific model against my prompt library?”
By scoping each project to a single model, you can:
  • Track that model’s safety over time
  • Compare it cleanly against another project (= another model)
  • Generate a report that says “here is the evidence for this model”
Examples of good project names:
  • “GPT-4o Safety Audit — Q2 2026”
  • “Claude Sonnet 4 — Pre-deployment review”
  • “Our Fine-Tuned LLaMA — v3.2”
  • “GPT-4o vs GPT-4o-mini comparison — December 2026”

​
Creating a project

1

Go to Projects → New Project

From the sidebar in your team’s workspace.
2

Name your project

Be specific. Include the model name and date if you’ll re-test the same model later.
3

Pick the target model

This is the model you’re testing. Choose any LLM Configuration that’s marked as a test model.
4

Pick your judges

Select up to 3 judge models. Use models from different families than your test model to reduce bias.
5

Choose a voting method

  • Majority (default) — 2 of 3 judges agree wins
  • Unanimous — all 3 must agree, otherwise inconclusive
  • Weighted — high-confidence votes count more
See Multi-Judge Consensus for details.
6

Set the minimum confidence

Default is 70%. Anything below this is flagged as low-confidence and may need review.
7

Add tags (optional)

Tags help you filter and group projects later (e.g., production, q2-2026).

​
What lives inside a project?

Once created, your project has these tabs:

​
Project lifecycle

You can archive a project when its testing cycle is finished but you want to keep the evidence. Archived projects are read-only and stay accessible in the audit trail.
Deleting a project removes all of its test data permanently. This includes test runs, judge evaluations, schedules, and risk assessments. Archive instead of deleting if you need to keep an audit trail.

​
Switching between projects

Use the project switcher at the top of the sidebar to jump between projects in your current team. Recently used projects appear at the top.

​
Sharing data across projects

The Prompt Library and the LLM Configurations are shared at the team level. So when you create two projects in the same team, both projects:
  • Use the same prompts
  • Can pick from the same set of models
This is intentional — it’s what makes model comparisons meaningful. Same prompts, different models = a fair fight.