rustio rustio.net

tokenfold-core vs tokenfold-cli vs llm-tokenizer

Side-by-side comparison of Rust crates

Core Metrics

tokenfold-coretokenfold-clillm-tokenizer
Health Score524765
Total Downloads3561382.0M
30d Downloads000
Dependents26044
Releases10815
Last Updated7d ago7d ago1d ago
Age2m2m9m

Health Breakdown

tokenfold-core
Maintenance
14
Quality
13
Community
10
Popularity
3
Documentation
12
tokenfold-cli
Maintenance
14
Quality
12
Community
6
Popularity
3
Documentation
12
llm-tokenizer
Maintenance
22
Quality
14
Community
11
Popularity
8
Documentation
10

Technical Details

tokenfold-coretokenfold-clillm-tokenizer
Version0.5.10.5.11.9.0
Stable (≥1.0)✗ No✗ No✓ Yes
LicenseApache-2.0Apache-2.0Apache-2.0
Dependencies91024
Crate Size169KB66KB242KB
Features200
Yanked %0.0%0.0%0.0%
Edition202420242021
MSRV1.981.98—
Owners111

Quick Verdict

  • •llm-tokenizer leads with a health score of 65/100, but none of the options score above 80.
  • •tokenfold-core, tokenfold-cli are pre-1.0 — API may change.