rustio rustio.net
42

toktier

v0.2.9 Experimental

Rust API preview for TokTier's exact, stateful tokenization runtime

Apache-2.0 Edition 2021 MSRV 1.93.1
Text processing #cuda#tokenization#bpe#llm#serving

Quick Verdict

  • โœ“Actively maintained (updated 39d ago)
  • !Pre-1.0: API may have breaking changes
  • โœ“Permissive license (Apache-2.0)

Security

Checking security advisories...
Downloads
156
Dependents
0
Releases
9
Size
1.4MB

Deep Insights

๐Ÿ“‰
Download decline

4 downloads in the last 30 days, down 97% from the previous period. May indicate migration to alternatives.

โš ๏ธ
Notable yanked versions

11% of releases have been yanked. Check version history before pinning.

Health Breakdown

Maintenance 11/25

Recency, release consistency, active ratio

Quality 8/25

Yanked ratio, deps, size, maturity, features

Community 5/20

Reverse deps, ownership, ecosystem

Popularity 3/15

Downloads, momentum, growth trend

Documentation 15/15

Docs, repo, license, metadata

Download Trend

Daily downloads ยท last 90 days
0/day avg
08/229/99/2710/1511/211/19

Version Adoption

v0.2.5
23%
v0.2.4
22%
v0.2.0
20%
v0.2.1
19%
v0.2.6
16%

Release Timeline

9 releases1 yankedsince 2026
J
F
M
A
M
J
J
A
S
O
N
D
2026
9
Less
More|
Yanked

Feature Flags

default =["sqlite", "prebuilt-gpu", "serving"]

jitserdesqlite*networkserving*prebuilt-gpu*

README

Loading README...

Maintainers

Dependencies
17
direct dependencies
Dependents
0
crates depend on toktier

Similar Crates