Decision Workspace
sentience-tokenize vs regex-lexer vs langbox
Side-by-side comparison of Rust crates
51
sentience-tokenize
experimentalv0.2.3
Tiny Rust zero-dep tokenizer (ident, number, string, parens, operators, keywords).
49
regex-lexer
growingv0.2.1
A regex-based lexer (tokenizer)
42
langbox
growingv0.6.0
A simple framework to build compilers and interpreters
Core Metrics
| sentience-tokenize | regex-lexer | langbox | |
|---|---|---|---|
| Health Score | 51 | 49 | 42 |
| Total Downloads | 2.6K | 19.7K | 12.0K |
| 30d Downloads | 0 | 0 | 0 |
| Dependents | 0 | 22 | 0 |
| Releases | 8 | 4 | 9 |
| Last Updated | 269d ago | 121d ago | 680d ago |
| Age | 9m | 6y | 3y 10m |
Health Breakdown
sentience-tokenize
Maintenance
11
Quality
15
Community
6
Popularity
4
Documentation
15
regex-lexer
Maintenance
10
Quality
15
Community
9
Popularity
5
Documentation
10
langbox
Maintenance
5
Quality
15
Community
5
Popularity
5
Documentation
12
Technical Details
| sentience-tokenize | regex-lexer | langbox | |
|---|---|---|---|
| Version | 0.2.3 | 0.2.1 | 0.6.0 |
| Stable (≥1.0) | ✗ No | ✗ No | ✗ No |
| License | MIT | MIT OR Apache-2.0 | MIT |
| Dependencies | 4 | 1 | 3 |
| Crate Size | 18KB | 10KB | 13KB |
| Features | 2 | 0 | 0 |
| Yanked % | 0.0% | 0.0% | 0.0% |
| Edition | 2021 | 2024 | 2021 |
| MSRV | 1.81 | — | 1.79 |
| Owners | 1 | 1 | 1 |
Links
Quick Verdict
- •sentience-tokenize leads with a health score of 51/100, but none of the options score above 80.
- •regex-lexer has the most downloads (19.7K), suggesting wider adoption.
- •⚠ langbox has not been updated in over a year.