rustio rustio.net

Decision Workspace

language-tokenizer

Side-by-side comparison of Rust crates

53
language-tokenizer
experimentalv0.3.0

Text tokenizer for linguistic purposes, such as text matching. Supports more than 40 languages, including English, French, Russian, Japanese, Thai etc.

Core Metrics

language-tokenizer
Health Score53
Total Downloads4.9K
30d Downloads4.1K
Dependents2
Releases3
Last Updated45d ago
Age4m

Health Breakdown

language-tokenizer
Maintenance
16
Quality
16
Community
4
Popularity
4
Documentation
13

Technical Details

language-tokenizer
Version0.3.0
Stable (≥1.0)✗ No
Licensenon-standard
Dependencies12
Crate Size27KB
Features11
Yanked %0.0%
Edition2024
MSRV1.88.0
Owners1

Links

language-tokenizer detail →GitHub ↗Docs ↗crates.io ↗

Quick Verdict

  • •language-tokenizer leads with a health score of 53/100, but none of the options score above 80.