Evaluation & Observability
Tracing, evaluations and prompt management.
4 projects · catalog last checked 2026-10-20. "Highlight" is editorial; status, commit and release come from the weekly evidence refresh.
Comparison
| Project | What it does | Highlight (editorial) | Deployment | Hardware | Licence | Status |
|---|---|---|---|---|---|---|
| Langfuse | LLM tracing, evals and prompt management | Tracing platform | library, self-hosted | cpu-ok | MIT (ee/ dirs separate) ⚠ needs review | active |
| Phoenix | AI observability and evaluation | Notebook + server | library, self-hosted | cpu-ok | Elastic-2.0 ⚠ needs review | active |
| promptfoo | Prompt testing and red-teaming CLI | CI evals | library, self-hosted | cpu-ok | MIT | active |
| DeepEval | Unit-test style LLM evaluation | pytest-style | library, self-hosted | cpu-ok | Apache-2.0 | active |
Quick start
npx promptfoo@latest init
npx promptfoo@latest eval
Project details
Langfuse
LLM tracing, evals and prompt management
| Upstream repository | Langfuse on GitHub |
| Setup docs | Langfuse setup guide |
| Licence | MIT (ee/ dirs separate) ⚠ needs review · licence evidence · checked 2026-10-20 |
| Licence detected by GitHub | NOASSERTION |
| Repository state | active |
| Licence review reason | Multiple licences: MIT except ee/ directories, which have separate terms (upstream LICENSE). |
| Last lookup | ok |
| Last commit | 2026-10-20 |
| Latest release | v4.66.0 (2026-10-20) |
| Deployment | library, self-hosted |
| Hardware | cpu-ok |
| Languages | TypeScript |
| Evidence collected | 2026-10-20 |
Phoenix
AI observability and evaluation
| Upstream repository | Phoenix on GitHub |
| Setup docs | Phoenix setup guide |
| Licence | Elastic-2.0 ⚠ needs review · licence evidence · checked 2026-10-20 |
| Licence detected by GitHub | NOASSERTION |
| Repository state | active |
| Licence review reason | Source-available, not OSI: Elastic License 2.0 (upstream LICENSE). |
| Last lookup | ok |
| Last commit | 2026-10-20 |
| Latest release | arize-phoenix-sqlean-v0.2.1 (2026-10-19) |
| Deployment | library, self-hosted |
| Hardware | cpu-ok |
| Languages | Python |
| Evidence collected | 2026-10-20 |
promptfoo
Prompt testing and red-teaming CLI
| Upstream repository | promptfoo on GitHub |
| Setup docs | promptfoo setup guide |
| Licence | MIT · licence evidence · checked 2026-10-20 |
| Licence detected by GitHub | MIT |
| Repository state | active |
| Licence review reason | – |
| Last lookup | ok |
| Last commit | 2026-10-20 |
| Latest release | code-scan-action-0.2.2 (2026-10-08) |
| Deployment | library, self-hosted |
| Hardware | cpu-ok |
| Languages | TypeScript |
| Evidence collected | 2026-10-20 |
DeepEval
Unit-test style LLM evaluation
| Upstream repository | DeepEval on GitHub |
| Setup docs | DeepEval setup guide |
| Licence | Apache-2.0 · licence evidence · checked 2026-10-20 |
| Licence detected by GitHub | Apache-2.0 |
| Repository state | active |
| Licence review reason | – |
| Last lookup | ok |
| Last commit | 2026-10-19 |
| Latest release | python-v4.2.4 (2026-09-22) |
| Deployment | library, self-hosted |
| Hardware | cpu-ok |
| Languages | Python |
| Evidence collected | 2026-10-20 |