rustio rustio.net
56

memra-tokenizer

v0.138.0 Experimental

GGUF-native BPE/SPM tokenizer with chat-template support for the memra inference engine

FSL-1.1-ALv2 Edition 2024 MSRV 1.97
Hardware supportArtificial intelligence #cuda#inference#safetensors#llm#gguf

Quick Verdict

  • โœ“Actively maintained (updated 28d ago)
  • !Pre-1.0: API may have breaking changes
  • โœ“Trusted by 170 crates
  • โœ“Tiny footprint (128KB, 1 deps)

Security

Checking security advisories...
Downloads
2.2K
Dependents
170
Releases
87
Size
128KB

Deep Insights

๐Ÿ“‰
Download decline

109 downloads in the last 30 days, down 94% from the previous period. May indicate migration to alternatives.

๐Ÿ”—
Moderate adoption

170 crates depend on memra-tokenizer. Reasonable ecosystem adoption, though not yet a core dependency.

๐Ÿ‘ค
Prolific maintainer

The primary maintainer publishes 61 crates. This suggests deep Rust expertise and long-term commitment to the ecosystem.

๐Ÿชถ
Minimal dependencies

Only 1 direct dependencies. Lean dependency tree means faster builds and lower supply chain risk.

๐ŸŒŸ
Used by top crates

Notable dependents include memra-engine, memra-server, memra-reference. When high-quality crates choose memra-tokenizer, it's a strong quality signal.

Health Breakdown

Maintenance 17/25

Recency, release consistency, active ratio

Quality 13/25

Yanked ratio, deps, size, maturity, features

Community 12/20

Reverse deps, ownership, ecosystem

Popularity 4/15

Downloads, momentum, growth trend

Documentation 10/15

Docs, repo, license, metadata

Download Trend

Daily downloads ยท last 90 days
0/day avg
08/239/109/2810/1611/311/20

Top Dependents

Most downloaded crates that depend on memra-tokenizer

Version Adoption

v0.138.0
27%
v0.137.0
21%
v0.129.0
19%
v0.135.0
17%
v0.133.0
16%

Release Timeline

10 releasessince 2026
J
F
M
A
M
J
J
A
S
O
N
D
2026
10
Less
More

README

Loading README...

Maintainers

Dependencies
1
direct dependencies
Dependents
170
crates depend on memra-tokenizer

Similar Crates