Decision Workspace
token-count vs tokenx-rs vs utokenizer
Side-by-side comparison of Rust crates
47
token-count
experimentalv0.4.0
Count tokens for LLM models using exact tokenization
47
tokenx-rs
experimentalv0.1.0
Fast token count estimation for LLMs at 96% accuracy without a full tokenizer
45
utokenizer
experimentalv0.1.1
CLI tool for building a local model-tokenizer registry and counting input tokens across model families.
Core Metrics
| token-count | tokenx-rs | utokenizer | |
|---|---|---|---|
| Health Score | 47 | 47 | 45 |
| Total Downloads | 165 | 622 | 29 |
| 30d Downloads | 0 | 0 | 0 |
| Dependents | 0 | 6 | 0 |
| Releases | 5 | 1 | 2 |
| Last Updated | 72d ago | 108d ago | 39d ago |
| Age | 2m | 3m | 1m |
Health Breakdown
token-count
Maintenance
15
Quality
11
Community
6
Popularity
3
Documentation
12
tokenx-rs
Maintenance
7
Quality
14
Community
8
Popularity
3
Documentation
15
utokenizer
Maintenance
12
Quality
12
Community
6
Popularity
2
Documentation
13
Technical Details
| token-count | tokenx-rs | utokenizer | |
|---|---|---|---|
| Version | 0.4.0 | 0.1.0 | 0.1.1 |
| Stable (≥1.0) | ✗ No | ✗ No | ✗ No |
| License | MIT | MIT | MIT |
| Dependencies | 15 | 2 | 9 |
| Crate Size | 339KB | 17KB | 33KB |
| Features | 0 | 0 | 0 |
| Yanked % | 0.0% | 0.0% | 0.0% |
| Edition | 2021 | 2021 | 2021 |
| MSRV | 1.85.0 | 1.70 | — |
| Owners | 1 | 1 | 1 |
Links
Quick Verdict
- •token-count leads with a health score of 47/100, but none of the options score above 80.
- •tokenx-rs has the most downloads (622), suggesting wider adoption.