tiktoken
v3.1.2 GrowingA high-performance pure-Rust implementation of OpenAI's tiktoken BPE tokenizer
Quick Verdict
- ✓Actively maintained (updated 20d ago)
- ✓Stable API (3.x for 2+ years)
- ✓Permissive license (MIT)
- !88% of releases were yanked
Security
Deep Insights
1.3K downloads in the last 30 days (44/day), up 203% from the previous period.
The API has been stable (1.x) for over 2 years with 8 releases. This level of maturity means you can depend on it without worrying about breaking changes.
6.1MB is relatively large. May impact download and compile times.
88% of releases have been yanked. Check version history before pinning.
Notable dependents include instructors, embedrs, chunkedrs, tiktoken-wasm. When high-quality crates choose tiktoken, it's a strong quality signal.
Health Breakdown
Recency, release consistency, active ratio
Yanked ratio, deps, size, maturity, features
Reverse deps, ownership, ecosystem
Downloads, momentum, growth trend
Docs, repo, license, metadata
Download Trend
Top Dependents
Most downloaded crates that depend on tiktoken
