rustio rustio.net

Decision Workspace

llm-tokenizer

Side-by-side comparison of Rust crates

54
llm-tokenizer
experimentalv1.3.0

LLM tokenizer library with caching and chat template support

Core Metrics

llm-tokenizer
Health Score54
Total Downloads189.4K
30d Downloads121.1K
Dependents14
Releases5
Last Updated17d ago
Age2m

Health Breakdown

llm-tokenizer
Maintenance
16
Quality
13
Community
9
Popularity
6
Documentation
10

Technical Details

llm-tokenizer
Version1.3.0
Stable (≥1.0)✓ Yes
LicenseApache-2.0
Dependencies23
Crate Size101KB
Features0
Yanked %0.0%
Edition2021
MSRV—
Owners1

Links

llm-tokenizer detail →GitHub ↗crates.io ↗

Quick Verdict

  • •llm-tokenizer leads with a health score of 54/100, but none of the options score above 80.