65
llm-tokenizer
v1.9.0 GrowingLLM tokenizer library with caching and chat template support
Apache-2.0 Edition 2021
Quick Verdict
- โActively maintained (updated 1d ago)
- !Heavy dependency tree (24 direct deps)
- โPermissive license (Apache-2.0)
Security
Checking security advisories...
Downloads
2.0M
Dependents
44
Releases
15
Size
242KB
Deep Insights
๐
Steady growth
391.6K downloads in the last 30 days (13.1K/day), up 5% from the previous period.
๐ฆ
Heavy dependency tree
24 direct dependencies. Consider the impact on compile times and supply chain complexity.
๐
Used by top crates
Notable dependents include dynamo-llm, smg, engine-zmq-adapter, engine-servicer. When high-quality crates choose llm-tokenizer, it's a strong quality signal.
Health Breakdown
Maintenance 22/25
Recency, release consistency, active ratio
Quality 14/25
Yanked ratio, deps, size, maturity, features
Community 11/20
Reverse deps, ownership, ecosystem
Popularity 8/15
Downloads, momentum, growth trend
Documentation 10/15
Docs, repo, license, metadata
Download Trend
Daily downloads ยท last 90 days
1K/day avg-100%
Top Dependents
Most downloaded crates that depend on llm-tokenizer
Version Adoption
v1.3.2
81%
v1.4.1
10%
v1.6.0
8%
v1.7.0
1%
v1.8.0
1%
Release Timeline
10 releasessince 2026
J
F
M
A
M
J
J
A
S
O
N
D
2026
10
LessMore
README
Loading README...
Maintainers
Dependencies
24
direct dependencies
Dependents
44
crates depend on llm-tokenizer