Decision Workspace
warble-eval-runner vs eval-core vs swink-agent-eval
Side-by-side comparison of Rust crates
46
warble-eval-runner
experimentalv0.17.0
Warble eval runner: replay golden questions through a dispatched agent under tier→model bindings, score, and produce a Pareto report.
54
eval-core
experimentalv0.4.0
A testing framework for LLM agents: prompt-based test cases with built-in assertions on tool calls, parameters, and text/math output — bring your own harness.
52
swink-agent-eval
experimentalv0.14.3
Evaluation framework for swink-agent: trajectory tracing, golden path verification, and cost governance
Core Metrics
| warble-eval-runner | eval-core | swink-agent-eval | |
|---|---|---|---|
| Health Score | 46 | 54 | 52 |
| Total Downloads | 395 | 236 | 376 |
| 30d Downloads | 0 | 0 | 0 |
| Dependents | 21 | 0 | 0 |
| Releases | 21 | 4 | 19 |
| Last Updated | 3d ago | 110d ago | 19d ago |
| Age | 2m | 3m | 5m |
Health Breakdown
warble-eval-runner
Maintenance
13
Quality
11
Community
9
Popularity
3
Documentation
10
eval-core
Maintenance
18
Quality
12
Community
6
Popularity
3
Documentation
15
swink-agent-eval
Maintenance
18
Quality
13
Community
6
Popularity
3
Documentation
12
Technical Details
| warble-eval-runner | eval-core | swink-agent-eval | |
|---|---|---|---|
| Version | 0.17.0 | 0.4.0 | 0.14.3 |
| Stable (≥1.0) | ✗ No | ✗ No | ✗ No |
| License | Apache-2.0 | MIT OR Apache-2.0 | MIT |
| Dependencies | 8 | 11 | 32 |
| Crate Size | 126KB | 81KB | 291KB |
| Features | 0 | 0 | 25 |
| Yanked % | 0.0% | 0.0% | 0.0% |
| Edition | 2021 | 2024 | 2024 |
| MSRV | — | 1.88 | 1.95 |
| Owners | 1 | 1 | 1 |
Links
Quick Verdict
- •eval-core leads with a health score of 54/100, but none of the options score above 80.
- •warble-eval-runner has the most downloads (395), suggesting wider adoption.