55
pretokie
v0.0.4 ExperimentalFast, zero-allocation pretokenizers for BPE tokenizers
MIT OR Apache-2.0 Edition 2021
Quick Verdict
- ✓Actively maintained (updated 56d ago)
- !Pre-1.0: API may have breaking changes
- ✓Tiny footprint (58KB, 3 deps)
- ✓Permissive license (MIT OR Apache-2.0)
Security
Checking security advisories...
Downloads
7.9K
Dependents
2
Releases
4
Size
58KB
Deep Insights
📈
Strong growth momentum
7.7K downloads in the last 30 days (258/day), up 4061% from the previous period.
🪶
Minimal dependencies
Only 3 direct dependencies. Lean dependency tree means faster builds and lower supply chain risk.
Health Breakdown
Maintenance 16/25
Recency, release consistency, active ratio
Quality 15/25
Yanked ratio, deps, size, maturity, features
Community 6/20
Reverse deps, ownership, ecosystem
Popularity 5/15
Downloads, momentum, growth trend
Documentation 13/15
Docs, repo, license, metadata
Download Trend
Daily downloads · last 90 days
88/day avg
Top Dependents
Most downloaded crates that depend on pretokie
Version Adoption
v0.0.4
97%
v0.0.3
2%
v0.0.1
0%
v0.0.2
0%
Release Timeline
4 releasessince 2026
J
F
M
A
M
J
J
A
S
O
N
D
2026
4
LessMore
Feature Flags
regex
README
Loading README...
Maintainers
Dependencies
3
direct dependencies
Dependents
2
crates depend on pretokie