rustio rustio.net
55

trustformers-tokenizers

v0.2.1 Experimental

Tokenizers for TrustformeRS

Apache-2.0 Edition 2021 MSRV 1.89
ScienceText processing #tokenizer#nlp#text-processing#wordpiece#bpe

Quick Verdict

  • โœ“Actively maintained (updated 45d ago)
  • !Pre-1.0: API may have breaking changes
  • !Heavy dependency tree (35 direct deps)
  • โœ“Permissive license (Apache-2.0)

Security

Checking security advisories...
Downloads
793
Dependents
18
Releases
10
Size
571KB

Deep Insights

๐Ÿ“‰
Download decline

37 downloads in the last 30 days, down 49% from the previous period. May indicate migration to alternatives.

๐Ÿ‘ค
Prolific maintainer

The primary maintainer publishes 1116 crates. This suggests deep Rust expertise and long-term commitment to the ecosystem.

๐Ÿ“ฆ
Heavy dependency tree

35 direct dependencies. Consider the impact on compile times and supply chain complexity.

๐ŸŒŸ
Used by top crates

Notable dependents include trustformers, trustformers-mobile, trustformers-serve. When high-quality crates choose trustformers-tokenizers, it's a strong quality signal.

Health Breakdown

Maintenance 17/25

Recency, release consistency, active ratio

Quality 11/25

Yanked ratio, deps, size, maturity, features

Community 9/20

Reverse deps, ownership, ecosystem

Popularity 3/15

Downloads, momentum, growth trend

Documentation 15/15

Docs, repo, license, metadata

Download Trend

Daily downloads ยท last 90 days
0/day avg
08/239/109/2810/1611/311/20

Top Dependents

Most downloaded crates that depend on trustformers-tokenizers

Version Adoption

v0.1.0-alpha.1
42%
v0.2.0
16%
v0.1.1
15%
v0.1.0-alpha.2
14%
v0.2.1
13%

Release Timeline

10 releasessince 2025
J
F
M
A
M
J
J
A
S
O
N
D
2025
1
2026
9
Less
More

Feature Flags

gpujaxonnxpyo3mecabpythonpytorchtensorflow

README

Loading README...

Maintainers

Dependencies
35
direct dependencies
Dependents
18
crates depend on trustformers-tokenizers

Similar Crates