Decision Workspace
pesti-structural-tokenizer vs miktik vs llm-utl
Side-by-side comparison of Rust crates
47
pesti-structural-tokenizer
experimentalv0.1.2
Rust structural tokenizer for LLM code understanding. encodes syntactic boundaries as token boundaries
57
miktik
experimentalv0.4.3
A unified, multi-backend tokenizer library for LLMs
53
llm-utl
experimentalv0.1.5
Convert code repositories into LLM-friendly prompts with smart chunking and filtering
Core Metrics
| pesti-structural-tokenizer | miktik | llm-utl | |
|---|---|---|---|
| Health Score | 47 | 57 | 53 |
| Total Downloads | 37 | 5.6K | 312 |
| 30d Downloads | 0 | 0 | 0 |
| Dependents | 0 | 0 | 2 |
| Releases | 3 | 11 | 6 |
| Last Updated | 15d ago | 36d ago | 296d ago |
| Age | 15d | 7m | 9m |
Health Breakdown
pesti-structural-tokenizer
Maintenance
13
Quality
13
Community
6
Popularity
2
Documentation
13
miktik
Maintenance
17
Quality
16
Community
5
Popularity
4
Documentation
15
llm-utl
Maintenance
15
Quality
13
Community
7
Popularity
3
Documentation
15
Technical Details
| pesti-structural-tokenizer | miktik | llm-utl | |
|---|---|---|---|
| Version | 0.1.2 | 0.4.3 | 0.1.5 |
| Stable (≥1.0) | ✗ No | ✗ No | ✗ No |
| License | AGPL-3.0-or-later | MIT | MIT |
| Dependencies | 7 | 4 | 18 |
| Crate Size | 35KB | 24KB | 72KB |
| Features | 0 | 6 | 2 |
| Yanked % | 0.0% | 0.0% | 0.0% |
| Edition | 2024 | 2024 | 2024 |
| MSRV | — | 1.86 | 1.85.0 |
| Owners | 1 | 1 | 1 |
Links
Quick Verdict
- •miktik leads with a health score of 57/100, but none of the options score above 80.