Decision Workspace
chunk_norris vs text-splitter vs tokengeex
Side-by-side comparison of Rust crates
40
chunk_norris
growingv0.2.1
A Rust library for splitting large text into smaller batches for LLM input.
59
text-splitter
growingv0.29.3
Split text into semantic chunks, up to a desired chunk size. Supports calculating length by characters and tokens, and is callable from Rust and Python.
44
tokengeex
growingv1.1.0
TokenGeeX is an efficient tokenizer for code based on UnigramLM and TokenMonster.
Core Metrics
| chunk_norris | text-splitter | tokengeex | |
|---|---|---|---|
| Health Score | 40 | 59 | 44 |
| Total Downloads | 2.4K | 1.1M | 13.6K |
| 30d Downloads | 5 | 110.8K | 15 |
| Dependents | 0 | 654 | 0 |
| Releases | 3 | 60 | 11 |
| Last Updated | 427d ago | 87d ago | 663d ago |
| Age | 1y 2m | 2y 10m | 2y 1m |
Health Breakdown
chunk_norris
Maintenance
6
Quality
15
Community
5
Popularity
4
Documentation
10
text-splitter
Maintenance
14
Quality
13
Community
13
Popularity
7
Documentation
12
tokengeex
Maintenance
7
Quality
16
Community
3
Popularity
5
Documentation
13
Technical Details
| chunk_norris | text-splitter | tokengeex | |
|---|---|---|---|
| Version | 0.2.1 | 0.29.3 | 1.1.0 |
| Stable (≥1.0) | ✗ No | ✗ No | ✓ Yes |
| License | MIT OR Apache-2.0 | MIT | Apache-2.0 |
| Dependencies | 1 | 21 | 20 |
| Crate Size | 6KB | 59KB | 52KB |
| Features | 0 | 4 | 0 |
| Yanked % | 0.0% | 1.7% | 0.0% |
| Edition | 2021 | 2021 | 2021 |
| MSRV | — | 1.83.0 | — |
| Owners | 1 | 1 | 1 |
Links
Quick Verdict
- •text-splitter leads with a health score of 59/100, but none of the options score above 80.
- •text-splitter is depended on by 654 crates — strongest ecosystem trust.
- •⚠ chunk_norris, tokengeex have not been updated in over a year.
- •chunk_norris, text-splitter are pre-1.0 — API may change.