rustio rustio.net

tokens vs itok vs simd-tokenizer

Side-by-side comparison of Rust crates

Core Metrics

tokensitoksimd-tokenizer
Health Score405151
Total Downloads272701.2K
30d Downloads000
Dependents080
Releases133
Last Updated112d ago49d ago28d ago
Age3m1m1m

Health Breakdown

tokens
Maintenance
7
Quality
15
Community
3
Popularity
2
Documentation
13
itok
Maintenance
11
Quality
13
Community
9
Popularity
3
Documentation
15
simd-tokenizer
Maintenance
12
Quality
15
Community
5
Popularity
4
Documentation
15

Technical Details

tokensitoksimd-tokenizer
Version0.1.00.3.10.1.2
Stable (≥1.0)✗ No✗ No✗ No
LicenseMIT OR Apache-2.0MITApache-2.0
Dependencies0105
Crate Size7KB286KB35KB
Features043
Yanked %0.0%0.0%0.0%
Edition202420242021
MSRV—1.951.85
Owners111

Quick Verdict

  • •itok leads with a health score of 51/100, but none of the options score above 80.
  • •simd-tokenizer has the most downloads (1.2K), suggesting wider adoption.