rustio rustio.net

dynamo-tokenizers vs llm-tokenizer vs miktik

Side-by-side comparison of Rust crates

Core Metrics

dynamo-tokenizersllm-tokenizermiktik
Health Score636557
Total Downloads818.0K2.0M5.6K
30d Downloads000
Dependents51440
Releases241511
Last Updated3d ago1d ago36d ago
Age5m9m7m

Health Breakdown

dynamo-tokenizers
Maintenance
20
Quality
14
Community
12
Popularity
7
Documentation
10
llm-tokenizer
Maintenance
22
Quality
14
Community
11
Popularity
8
Documentation
10
miktik
Maintenance
17
Quality
16
Community
5
Popularity
4
Documentation
15

Technical Details

dynamo-tokenizersllm-tokenizermiktik
Version3.1.01.9.00.4.3
Stable (≥1.0)✓ Yes✓ Yes✗ No
LicenseApache-2.0Apache-2.0MIT
Dependencies16244
Crate Size76KB242KB24KB
Features006
Yanked %0.0%0.0%0.0%
Edition202420212024
MSRV——1.86
Owners411

Quick Verdict

  • •llm-tokenizer leads with a health score of 65/100, but none of the options score above 80.
  • •miktik is pre-1.0 — API may change.