rustio rustio.net
34

gemini-tokenizer

v0.2.0 Experimental

Authoritative Gemini tokenizer for Rust, ported from the official Google Python GenAI SDK

Apache-2.0 Edition 2021 MSRV 1.70
EncodingText processing #tokenizer#google#gemini#sentencepiece#llm

Quick Verdict

  • !Pre-1.0: API may have breaking changes
  • โœ“Permissive license (Apache-2.0)

Security

Checking security advisories...
Downloads
4.9K
Dependents
2
Releases
1
Size
2.2MB

Deep Insights

๐Ÿ“ˆ
Strong growth momentum

1.2K downloads in the last 30 days (41/day), up 48% from the previous period.

๐Ÿ“
Large crate size

2.2MB is relatively large. May impact download and compile times.

Health Breakdown

Maintenance 4/25

Recency, release consistency, active ratio

Quality 10/25

Yanked ratio, deps, size, maturity, features

Community 7/20

Reverse deps, ownership, ecosystem

Popularity 4/15

Downloads, momentum, growth trend

Documentation 9/15

Docs, repo, license, metadata

Download Trend

Daily downloads ยท last 90 days
0/day avg
08/239/109/2810/1611/311/20

Top Dependents

Most downloaded crates that depend on gemini-tokenizer

Version Adoption

v0.2.0
100%

Release Timeline

1 releasessince 2026
J
F
M
A
M
J
J
A
S
O
N
D
2026
1
Less
More

README

Loading README...

Maintainers

Dependencies
5
direct dependencies
Dependents
2
crates depend on gemini-tokenizer

Similar Crates