44
text-tokenizer
v0.6.16 GrowingCustom text tokenizer
MIT Edition 2024
Quick Verdict
- !Pre-1.0: API may have breaking changes
- ✓Permissive license (MIT)
Security
Checking security advisories...
Downloads
18.4K
Dependents
0
Releases
31
Size
59KB
Deep Insights
📈
Strong growth momentum
36 downloads in the last 30 days (1/day), up 3500% from the previous period.
🔬
Pre-1.0 for over a year
Despite being 2+ years old, text-tokenizer hasn't reached 1.0 yet. Expect potential API changes between versions.
Health Breakdown
Maintenance 12/25
Recency, release consistency, active ratio
Quality 16/25
Yanked ratio, deps, size, maturity, features
Community 1/20
Reverse deps, ownership, ecosystem
Popularity 5/15
Downloads, momentum, growth trend
Documentation 10/15
Docs, repo, license, metadata
Download Trend
Daily downloads · last 90 days
1/day avg+4%
Version Adoption
v0.4.1
24%
v0.5.3
19%
v0.5.6
19%
v0.5.0
19%
v0.5.1
19%
Release Timeline
31 releasessince 2023
J
F
M
A
M
J
J
A
S
O
N
D
2023
1
2024
11
2025
19
2026
LessMore
Feature Flags
strings
README
Loading README...
Maintainers
Dependencies
4
direct dependencies
Dependents
0
crates depend on text-tokenizer