Decision Workspace
wordchipper vs tiktoken-rs vs tiktoken
Side-by-side comparison of Rust crates
47
wordchipper
experimentalv0.9.0
HPC Rust LLM Tokenizer Library
59
tiktoken-rs
growingv0.9.1
Library for encoding and decoding with the tiktoken library in Rust
55
tiktoken
growingv3.1.2
A high-performance pure-Rust implementation of OpenAI's tiktoken BPE tokenizer
Core Metrics
| wordchipper | tiktoken-rs | tiktoken | |
|---|---|---|---|
| Health Score | 47 | 59 | 55 |
| Total Downloads | 442 | 5.3M | 5.7K |
| 30d Downloads | 117 | 823.9K | 1.3K |
| Dependents | 8 | 3.8K | 12 |
| Releases | 27 | 32 | 8 |
| Last Updated | 4d ago | 138d ago | 20d ago |
| Age | 1m | 3y 1m | 2y 7m |
Health Breakdown
wordchipper
Maintenance
13
Quality
13
Community
6
Popularity
3
Documentation
12
tiktoken-rs
Maintenance
12
Quality
12
Community
12
Popularity
8
Documentation
15
tiktoken
Maintenance
17
Quality
10
Community
9
Popularity
4
Documentation
15
Technical Details
| wordchipper | tiktoken-rs | tiktoken | |
|---|---|---|---|
| Version | 0.9.0 | 0.9.1 | 3.1.2 |
| Stable (≥1.0) | ✗ No | ✗ No | ✓ Yes |
| License | MIT | MIT | MIT |
| Dependencies | 28 | 9 | 9 |
| Crate Size | 259KB | 3.8MB | 6.1MB |
| Features | 10 | 2 | 2 |
| Yanked % | 0.0% | 0.0% | 87.5% |
| Edition | 2024 | 2021 | 2024 |
| MSRV | 1.93.0 | 1.61.0 | 1.94 |
| Owners | 1 | 1 | 1 |
Links
Quick Verdict
- •tiktoken-rs leads with a health score of 59/100, but none of the options score above 80.
- •tiktoken-rs is depended on by 3.8K crates — strongest ecosystem trust.
- •wordchipper, tiktoken-rs are pre-1.0 — API may change.