56
pretokie
v0.1.3 ExperimentalFast, zero-allocation pretokenizers for BPE tokenizers
MIT OR Apache-2.0 Edition 2021
Quick Verdict
- โActively maintained (updated 78d ago)
- !Pre-1.0: API may have breaking changes
- โTiny footprint (80KB, 3 deps)
- โPermissive license (MIT OR Apache-2.0)
Security
Checking security advisories...
Downloads
11.0K
Dependents
10
Releases
6
Size
80KB
Deep Insights
๐
Steady growth
708 downloads in the last 30 days (24/day), up 3% from the previous period.
๐ชถ
Minimal dependencies
Only 3 direct dependencies. Lean dependency tree means faster builds and lower supply chain risk.
Health Breakdown
Maintenance 15/25
Recency, release consistency, active ratio
Quality 15/25
Yanked ratio, deps, size, maturity, features
Community 8/20
Reverse deps, ownership, ecosystem
Popularity 5/15
Downloads, momentum, growth trend
Documentation 13/15
Docs, repo, license, metadata
Download Trend
Daily downloads ยท last 90 days
0/day avg
Top Dependents
Most downloaded crates that depend on pretokie
Version Adoption
v0.0.4
90%
v0.1.3
8%
v0.0.3
1%
v0.0.1
0%
v0.0.2
0%
Release Timeline
6 releasessince 2026
J
F
M
A
M
J
J
A
S
O
N
D
2026
6
LessMore
Feature Flags
regex
README
Loading README...
Maintainers
Dependencies
3
direct dependencies
Dependents
10
crates depend on pretokie