Decision Workspace
sentience-tokenize vs regex-lexer vs sil-lexer
Side-by-side comparison of Rust crates
50
sentience-tokenize
growingv0.2.3
Tiny Rust zero-dep tokenizer (ident, number, string, parens, operators, keywords).
51
regex-lexer
growingv0.2.2
A regex-based lexer (tokenizer)
50
sil-lexer
experimentalv0.7.0
Silc lexical analysis (Raku-inspired surface)
Core Metrics
| sentience-tokenize | regex-lexer | sil-lexer | |
|---|---|---|---|
| Health Score | 50 | 51 | 50 |
| Total Downloads | 2.6K | 22.5K | 46 |
| 30d Downloads | 0 | 0 | 0 |
| Dependents | 0 | 22 | 3 |
| Releases | 8 | 5 | 2 |
| Last Updated | 406d ago | 66d ago | 4d ago |
| Age | 1y 1m | 6y 4m | 5d |
Health Breakdown
sentience-tokenize
Maintenance
9
Quality
16
Community
6
Popularity
4
Documentation
15
regex-lexer
Maintenance
12
Quality
15
Community
9
Popularity
5
Documentation
10
sil-lexer
Maintenance
14
Quality
14
Community
8
Popularity
2
Documentation
12
Technical Details
| sentience-tokenize | regex-lexer | sil-lexer | |
|---|---|---|---|
| Version | 0.2.3 | 0.2.2 | 0.7.0 |
| Stable (≥1.0) | ✗ No | ✗ No | ✗ No |
| License | MIT | MIT OR Apache-2.0 | Apache-2.0 |
| Dependencies | 4 | 1 | 1 |
| Crate Size | 18KB | 11KB | 4KB |
| Features | 2 | 0 | 0 |
| Yanked % | 0.0% | 0.0% | 0.0% |
| Edition | 2021 | 2024 | 2021 |
| MSRV | 1.81 | — | 1.82 |
| Owners | 1 | 1 | 1 |
Links
Quick Verdict
- •regex-lexer leads with a health score of 51/100, but none of the options score above 80.
- •⚠ sentience-tokenize has not been updated in over a year.