Decision Workspace
bpetok vs tok3niz3r-cli vs morpheme-cli
Side-by-side comparison of Rust crates
35
bpetok
growingv0.1.2
A simple CLI for tokenizing text input using Byte Pair Encoding (BPE).
43
tok3niz3r-cli
experimentalv0.2.0
Command-line tool to train, run, and inspect byte-level BPE tokenizers. Installs the `tok3` binary.
53
morpheme-cli
experimentalv0.5.0
Command-line tool to train, inspect and run Hugging Face-compatible tokenizers with morpheme.
Core Metrics
| bpetok | tok3niz3r-cli | morpheme-cli | |
|---|---|---|---|
| Health Score | 35 | 43 | 53 |
| Total Downloads | 3.1K | 39 | 68 |
| 30d Downloads | 0 | 0 | 0 |
| Dependents | 0 | 0 | 0 |
| Releases | 3 | 2 | 6 |
| Last Updated | 744d ago | 125d ago | 8d ago |
| Age | 2y | 4m | 9d |
Health Breakdown
bpetok
Maintenance
5
Quality
15
Community
6
Popularity
4
Documentation
5
tok3niz3r-cli
Maintenance
11
Quality
12
Community
6
Popularity
2
Documentation
12
morpheme-cli
Maintenance
18
Quality
15
Community
6
Popularity
2
Documentation
12
Technical Details
| bpetok | tok3niz3r-cli | morpheme-cli | |
|---|---|---|---|
| Version | 0.1.2 | 0.2.0 | 0.5.0 |
| Stable (≥1.0) | ✗ No | ✗ No | ✗ No |
| License | non-standard | MIT | MIT |
| Dependencies | 3 | 14 | 6 |
| Crate Size | 11KB | 38KB | 27KB |
| Features | 0 | 0 | 2 |
| Yanked % | 0.0% | 0.0% | 0.0% |
| Edition | 2021 | 2024 | 2024 |
| MSRV | — | 1.85 | 1.85 |
| Owners | 1 | 1 | 1 |
Links
Quick Verdict
- •morpheme-cli leads with a health score of 53/100, but none of the options score above 80.
- •bpetok has the most downloads (3.1K), suggesting wider adoption.
- •⚠ bpetok has not been updated in over a year.